Target trajectory generation method based on infrared monitoring, product, equipment and medium

Abstract

The invention relates to the technical field of target monitoring, in particular to a target track generation method based on infrared monitoring, a product, equipment and a medium. The method comprises the steps that infrared monitoring data collected by an infrared detection device, image data collected by a camera device and radar data collected by a radar are obtained in the process that an unmanned aerial vehicle conducts routing inspection according to an initial routing inspection route, and the infrared detection device, the camera device and the radar are all arranged on the unmanned aerial vehicle; determining a first position of the target based on the infrared monitoring data, determining a second position of the target based on the image data, and determining a third position of the target based on the radar data; inputting the first position, the second position and the third position into a preset fusion model, and obtaining a target position output by the preset fusion model; and generating a target moving track based on the target position at each moment, and predicting a future moving track of the target based on the target moving track. According to the invention, the accuracy of target tracking can be improved.

Description

Technical Field

[0001] The present application relates to the field of target monitoring technology, and in particular to a target trajectory generation method, product, device and medium based on infrared monitoring. Background Art

[0002] With the rapid development of contemporary science and technology, especially breakthroughs in avionics and sensor technology, the functions of drones have expanded to many fields such as environmental monitoring and search and rescue operations. In these applications, drones are usually equipped with a variety of sensors to collect detailed information about the surrounding environment.

[0003] Related drone target tracking technologies mainly rely on visible light cameras, lidar and other types of optical sensors. However, these technical solutions often perform poorly in low-light or completely dark environments, making it difficult to ensure high accuracy and reliability of target tracking. Summary of the invention

[0004] In order to solve the problem of low target tracking accuracy in the prior art, the present application provides a target trajectory generation method, product, device and medium based on infrared monitoring.

[0005] In the first aspect, the present application provides a method for generating target trajectories based on infrared monitoring, which adopts the following technical solutions: A target trajectory generation method based on infrared monitoring, comprising: During the inspection process of the UAV along the initial inspection route, infrared monitoring data collected by the infrared detection device, image data collected by the camera device, and radar data collected by the radar are obtained, wherein the infrared detection device, the camera device, and the radar are all arranged on the UAV; Determine a first position of the target based on the infrared monitoring data, determine a second position of the target based on the image data, and determine a third position of the target based on the radar data; Inputting the first position, the second position and the third position into a preset fusion model to obtain a target position output by the preset fusion model; A target movement trajectory is generated based on the target position at each moment, and a future movement trajectory of the target is predicted based on the target movement trajectory.

[0006] By adopting the above-mentioned technical solution and fusing infrared monitoring data, image data and radar data, it is possible to overcome the limitations of a single sensor in specific environments, such as the effectiveness of infrared sensors at night or in low light conditions, the penetration ability of radar in foggy days, and the advantages of cameras in image recognition. This multi-sensor fusion method can provide more accurate and reliable target location information. By using a preset fusion model to process multiple sensor data, the target position can be accurately determined. Based on the target position at each moment, the target movement trajectory can be generated and the future trajectory can be predicted, thereby achieving accurate tracking of the target.

[0007] In a preferred example, the present application may be further configured as follows: after predicting the future movement trajectory of the target based on the movement trajectory of the target, the method further includes: Acquire three-dimensional data of the inspection area, and dynamically determine the expected trajectory range of the drone based on the three-dimensional data and the future movement trajectory of the target; Comparing the expected trajectory with the initial inspection path to determine whether the initial inspection path exceeds the expected trajectory range; If there is an inspection section in the initial inspection path that exceeds the expected trajectory range, adjusting the inspection section based on the expected trajectory range to obtain a dynamic target inspection path; The UAV is controlled to perform inspections along the dynamic target inspection path.

[0008] By adopting the above technical solutions, the three-dimensional data provides detailed terrain, obstacles and height information of the inspection area, which helps the drone to understand the inspection environment more accurately, avoid collisions between drones during flight, and improve flight safety. Combined with the future movement trajectory of the target, the expected trajectory range of the drone is dynamically determined, which can ensure that the drone is inspected at the best position and angle, thereby improving inspection efficiency. Since the target movement trajectory is dynamically predicted, the expected trajectory range of the drone will also be adjusted accordingly, which enables the drone to adapt to changes in the target in real time, verify whether the initial inspection path meets the inspection requirements, whether it deviates too much from the target or cannot effectively cover the target movement area, and flexibly adjust the initial inspection path according to the expected estimated range to ensure that the drone is inspected at the best position and angle, which can reduce useless flight time and distance and improve inspection efficiency.

[0009] In a preferred example, the present application may be further configured as follows: dynamically determining the expected trajectory range of the drone based on the three-dimensional data and the future moving trajectory of the target includes: Based on the future moving trajectory of the target, the preset angle and the preset distance, determining the initial expected trajectory range within which the drone can monitor the target; An overlapping area between the three-dimensional data and the initial expected trajectory range is determined, and the overlapping area is removed from the initial expected trajectory range to obtain the expected trajectory range of the UAV.

[0010] By adopting the above technical solution, combined with the target's future movement trajectory, preset angle and preset distance, it can be ensured that the initial expected trajectory range of the UAV closely surrounds the target's possible movement path, allowing the UAV to monitor the target more effectively. The three-dimensional data provides detailed terrain and obstacle information in the inspection area. The overlapping area between the initial expected trajectory range and the three-dimensional data is determined, and these overlapping parts are removed, which can ensure that the UAV avoids obstacles and collisions during flight, thereby improving flight safety.

[0011] In a preferred example, the present application may be further configured as follows: the step of obtaining infrared monitoring data collected by the infrared detection device includes: Obtain visibility at the current time and in the current environment, and determine whether the current time is within a preset time period and whether the visibility is lower than a preset visibility threshold; If the current moment is within the preset time period and the visibility is lower than the preset visibility threshold, repeatedly performing the quality detection step until the quality parameter meets the preset quality standard; The quality detection step includes: obtaining initial infrared monitoring data collected by the infrared detection device, determining whether the quality parameters of the initial infrared monitoring data meet the preset quality standards, and if not, adjusting the equipment parameters of the infrared detection device; The initial infrared monitoring data whose quality parameters meet the preset quality standard is used as the infrared monitoring data collected by the infrared detection device.

[0012] By adopting the above technical solution and introducing the judgment of the visibility at the current time and the current environment, the data quality detection process can be automatically triggered in a specific time period and under low visibility conditions. The adaptive mechanism based on environmental conditions ensures that the infrared monitoring data can still maintain a high quality level in an unfavorable environment. By repeatedly executing the quality detection steps and adjusting the equipment parameters of the infrared detection device when the quality parameters do not meet the preset standards, low-quality data caused by poor equipment status or environmental interference is effectively avoided, ensuring that the infrared monitoring data finally obtained meets the preset quality standards, thereby improving the accuracy and reliability of the data.

[0013] In a preferred example, the present application may be further configured as follows: the quality parameters include signal-to-noise ratio and image clarity; The determining whether the quality parameter of the initial infrared monitoring data meets the preset quality standard, and if not, adjusting the equipment parameter of the infrared detection device, comprises: Comparing the signal-to-noise ratio and the image clarity with corresponding preset quality standards respectively; If any quality parameter does not meet the corresponding preset quality standard, the device parameter corresponding to the quality parameter that does not meet the preset quality standard is adjusted according to the preset corresponding relationship. The preset corresponding relationship represents the corresponding relationship between the quality parameter and the device parameter, including the signal-to-noise ratio corresponding to the gain, and the image clarity corresponding to the focal length.

[0014] By adopting the above technical solution, the signal-to-noise ratio and image clarity are used as quality parameters, and corresponding preset quality standards are set, thereby achieving precise control of the quality of infrared monitoring data.

[0015] In a preferred example, the present application can be further configured as follows: the method further includes: Acquire multiple groups of sample data, each group of sample data includes a first sample position of a target sample collected by an infrared detection device, a second sample position of the target sample collected by a camera device, a third sample position of the target sample collected by a radar, and an actual sample position of the target sample; The first sample position, the second sample position and the third sample position are used as model inputs, and the actual sample position is used as a model output to train the model, thereby obtaining a preset fusion model after training.

[0016] By adopting the above technical solution, a model is constructed to fuse data from different sensors (infrared detection, camera, radar). The model can comprehensively utilize the advantages of each sensor to more accurately determine the location of the target sample. Each sensor has its own unique detection principle and limitations. The fusion of data from multiple sensors can make up for the shortcomings of a single sensor and improve the overall positioning accuracy.

[0017] In a preferred example, the present application may be further configured as follows: the step of inputting the first position, the second position, and the third position into a preset fusion model includes: Determine the distance between each two positions among the first position, the second position and the third position, and compare each obtained distance with a preset distance threshold; If each distance does not exceed the preset distance threshold, the first position, the second position and the third position are input into a preset fusion model.

[0018] By using the above technical solution, calculating and comparing the distances between the locations collected by different sensors, the consistency between these data can be preliminarily verified. If there is a significant difference between the data of a certain sensor and the data of other sensors (that is, the distance exceeds the preset threshold), it may indicate that the sensor data is abnormal or has a large error. This verification helps to identify and eliminate potential data quality issues in advance, thereby improving the accuracy and reliability of subsequent fusion models.

[0019] In a second aspect, the present application provides a computer program product, which adopts the following technical solution: A computer program product includes a computer program. When the computer program is executed by a processor, the method for generating a target trajectory based on infrared monitoring as described in any one of the first aspects is implemented.

[0020] In a third aspect, the present application provides an electronic device, which adopts the following technical solution: one or more processors; Memory; At least one application, wherein the at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute the target trajectory generation method based on infrared monitoring as described in any one of the first aspects.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium stores a computer program, which, when executed in a computer, causes the computer to execute the target trajectory generation method based on infrared monitoring as described in any one of the first aspects.

[0022] In summary, this application includes the following beneficial technical effects: By fusing infrared monitoring data, image data and radar data, the present application can overcome the limitations of a single sensor in specific environments, such as the effectiveness of infrared sensors at night or in low light conditions, the penetration ability of radar in foggy days, and the advantages of cameras in image recognition. This multi-sensor fusion method can provide more accurate and reliable target location information. By using a preset fusion model to process multiple sensor data, the target position can be accurately determined, and the target movement trajectory can be generated based on the target position at each moment and the future trajectory can be predicted, thereby achieving accurate tracking of the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a flow chart of a method for generating target trajectory based on infrared monitoring provided in an embodiment of the present application; Figure 2It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following is combined with Figure 1 -Attached Figure 2 This application is described in further detail.

[0025] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, those skilled in the art may make modifications to the present embodiment without any creative contribution as needed, but such modifications are protected by the patent law as long as they are within the scope of the claims of the present application.

[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0028] It should be noted that in the optional embodiments of the present application, the object information and other related data involved, when the embodiments in the present application are applied to specific products or technologies, need to obtain the permission or consent of the object, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. In other words, if the embodiments of the present application involve data related to the object, it needs to be obtained with the authorization and consent of the object, the authorization and consent of the relevant departments, and in compliance with the relevant laws, regulations and standards of the country and region. If personal information is involved in the embodiments, the acquisition of all personal information needs to obtain the consent of the individual. If sensitive information is involved, the separate consent of the information subject needs to be obtained. The embodiments also need to be implemented with the authorization and consent of the object.

[0029] The present application embodiment provides a method for generating a target trajectory based on infrared monitoring, such as Figure 1As shown, the method provided in the embodiment of the present application is executed by an electronic device, which can be a server or a terminal device, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present application. The method includes steps S101 to S104, wherein: S101. During the inspection process of the UAV along the initial inspection route, infrared monitoring data collected by the infrared detection device, image data collected by the camera device, and radar data collected by the radar are obtained. The infrared detection device, the camera device, and the radar are all arranged on the UAV.

[0030] The initial inspection route can be pre-set by the technician and stored in the drone or the electronic equipment set up on the ground. The drone will conduct the inspection according to the initial inspection route. During the drone inspection process, the infrared detection device, the camera device and the radar collect data at the same collection frequency and time. The infrared detection device collects the infrared radiation data of the surrounding environment and converts it into an infrared image or a temperature distribution map. The camera device (such as a high-definition camera) simultaneously captures the visible light image of the surrounding environment to capture the image information of the target. The radar (such as a millimeter wave radar or a laser radar) sends and receives signals, and determines the distance, speed, direction and other information of the target by measuring the time difference or phase difference of the signal.

[0031] S102: Determine a first position of the target based on the infrared monitoring data, determine a second position of the target based on the image data, and determine a third position of the target based on the radar data.

[0032] Specifically, for any collection moment, the infrared monitoring data collected at that collection moment is processed to identify the initial first position of the target in the infrared image or temperature distribution map, the image data collected at that collection moment is processed by image recognition or target tracking algorithm to identify the initial second position of the target in the image, the radar data collected at that collection moment is analyzed, and the initial third position of the target is calculated through the measured distance, speed and direction information.

[0033] Furthermore, the obtained initial first position, initial second position and initial third position are respectively converted to the geographic coordinate system through coordinate conversion to obtain a first position corresponding to the initial first position, a second position corresponding to the initial second position, and a third position corresponding to the initial third position. Thus, each acquisition moment corresponds to a set of position data, including the first position, the second position and the third position.

[0034] S103: Input the first position, the second position and the third position into a preset fusion model to obtain a target position output by the preset fusion model.

[0035] After obtaining the position data at any collection time, the first position, the second position and the third position corresponding to the collection time are input into the preset fusion model, and the target position corresponding to the collection time can be obtained.

[0036] The preset fusion model is obtained by pre-training, and the training process is: obtain multiple groups of sample data, each group of sample data includes the first sample position of the target sample collected by the infrared detection device, the second sample position of the target sample collected by the camera device, the third sample position of the target sample collected by the radar, and the actual sample position of the target sample. The first sample position, the second sample position and the third sample position are used as model inputs, and the actual sample position is used as the model output to train the model, and the trained preset fusion model is obtained.

[0037] S104: Generate a target movement trajectory based on the target position at each moment, and predict the target's future movement trajectory based on the target movement trajectory.

[0038] Arrange the corresponding target positions in chronological order according to the collection time to generate the target movement trajectory. Based on the existing target movement trajectory, use time series analysis, machine learning or deep learning methods to predict the target's future movement trajectory. The prediction algorithm can be flexibly selected according to actual needs, and this embodiment does not make specific limitations.

[0039] This embodiment can overcome the limitations of a single sensor in a specific environment by fusing infrared monitoring data, image data and radar data, such as the effectiveness of infrared sensors at night or in low light conditions, the penetration ability of radar in foggy days, and the advantages of cameras in image recognition. This multi-sensor fusion method can provide more accurate and reliable target position information. By using a preset fusion model to process multiple sensor data, the target position can be accurately determined, and the target movement trajectory can be generated based on the target position at each moment and the future trajectory can be predicted, thereby achieving accurate tracking of the target.

[0040] In a possible implementation manner of the embodiment of the present application, after predicting the future movement trajectory of the target based on the movement trajectory of the target, the method further includes: Obtain 3D data of the inspection area and dynamically determine the expected trajectory range of the drone based on the 3D data and the target's future movement trajectory; Compare the expected trajectory with the initial inspection path to determine whether the initial inspection path exceeds the expected trajectory range; If there is an inspection section in the initial inspection path that exceeds the expected trajectory range, the inspection section is adjusted based on the expected trajectory range to obtain a dynamic target inspection path; Control the drone to inspect along the dynamic target inspection path.

[0041] In this embodiment, the inspection area can be scanned in advance using a drone equipped with sensors (such as a laser radar, a stereo camera, etc.) to collect three-dimensional data of the inspection area, and the three-dimensional data includes information such as three-dimensional terrain, buildings, vegetation, obstacles, etc. The collected three-dimensional data is converted into a format suitable for processing, such as point cloud data, three-dimensional grid, or digital elevation model.

[0042] The target's future moving trajectory includes the coordinates, speed, direction and other information of each point in the trajectory. According to the requirements of the drone monitoring task, the drone's monitoring parameters are preset. The monitoring parameters include preset angles and preset distances. The preset angles include monitoring top-down angles and monitoring side angles. The preset distance is used to represent a safe and effective monitoring distance. With the target's future moving trajectory as the center, a cylinder or fan-shaped area that meets the monitoring parameters is drawn as the drone's expected trajectory range. The expected trajectory range indicates the flight range within which the drone can monitor the target.

[0043] Furthermore, there may be one or more inspection sections that exceed the expected trajectory range. For any of the inspection sections, the point closest to the expected trajectory range for each point in the inspection section is determined through a spatial search algorithm (KD tree, R tree or spatial hash table), and the points closest to the expected trajectory range constitute the adjusted inspection section.

[0044] In this embodiment, the three-dimensional data provides detailed terrain, obstacles and height information of the inspection area, which helps the UAV to understand the inspection environment more accurately, avoid collisions between UAVs during flight, and improve flight safety. Combined with the future movement trajectory of the target, the expected trajectory range of the UAV is dynamically determined, which can ensure that the UAV is inspected at the best position and angle, thereby improving inspection efficiency. Since the target movement trajectory is dynamically predicted, the expected trajectory range of the UAV will also be adjusted accordingly, which enables the UAV to adapt to changes in the target in real time, verify whether the initial inspection path meets the inspection requirements, whether it deviates too much from the target or cannot effectively cover the target movement area, and flexibly adjust the initial inspection path according to the expected estimated range to ensure that the UAV is inspected at the best position and angle, which can reduce useless flight time and distance and improve inspection efficiency.

[0045] A possible implementation of the embodiment of the present application is to dynamically determine the expected trajectory range of the drone based on the three-dimensional data and the future movement trajectory of the target, including: Based on the target's future moving trajectory, preset angle, and preset distance, determine the initial expected trajectory range that the drone can monitor the target; The overlapping area between the three-dimensional data and the initial expected trajectory range is determined, and the overlapping area is removed from the initial expected trajectory range to obtain the expected trajectory range of the UAV.

[0046] In this embodiment, for each point on the target's future moving trajectory, a fan-shaped area is drawn with the point as the center according to a preset angle and a preset distance. The preset distance represents the radius of the fan-shaped area. The fan-shaped areas of each point are spliced ​​along the direction of the target's future moving trajectory to form a continuous fan-shaped area. This area is the initial expected trajectory range, which covers the predicted path of the target's future moving trajectory and takes into account the monitoring angle and distance restrictions of the drone. Then, GIS software or spatial analysis libraries (such as ArcGIS, QGIS, GDAL, Shapely, etc.) are used to perform spatial alignment and remove overlapping areas from the initial expected trajectory range.

[0047] This embodiment combines the target's future movement trajectory, preset angle and preset distance to ensure that the drone's initial expected trajectory range closely surrounds the target's possible movement path, allowing the drone to monitor the target more effectively. The three-dimensional data provides detailed terrain and obstacle information in the inspection area, determines the overlapping area between the initial expected trajectory range and the three-dimensional data, and removes these overlapping parts, which can ensure that the drone avoids obstacles and collisions during flight, thereby improving flight safety.

[0048] A possible implementation of the embodiment of the present application is to obtain infrared monitoring data collected by an infrared detection device, including: Obtain visibility at the current time and in the current environment, and determine whether the current time is within a preset time period and whether visibility is lower than a preset visibility threshold; If the current time is within the preset time period and the visibility is lower than the preset visibility threshold, the quality detection step is repeated until the quality parameter meets the preset quality standard; The quality detection step includes: obtaining initial infrared monitoring data collected by the infrared detection device, determining whether the quality parameters of the initial infrared monitoring data meet the preset quality standards, and if not, adjusting the equipment parameters of the infrared detection device; The initial infrared monitoring data whose quality parameters meet the preset quality standards is used as the infrared monitoring data collected by the infrared detection device.

[0049] In this embodiment, the visibility data of the current environment is obtained by using weather station data, environmental sensors or visibility sensors integrated in infrared detection devices. The current time is compared with a preset time period (used to characterize low visibility high-incidence periods such as nighttime and early morning) to determine whether the current time falls within the time period. At the same time, the visibility of the current environment is compared with a preset visibility threshold (such as less than 500 meters indicates low visibility) to determine whether the visibility is lower than the preset visibility threshold. If the current time is within the preset time period and the visibility is lower than the preset visibility threshold, the quality inspection process is repeated.

[0050] This embodiment introduces the judgment of the visibility at the current time and the current environment, and can automatically trigger the data quality detection process in a specific time period and low visibility conditions. The adaptive mechanism based on environmental conditions ensures that the infrared monitoring data can still maintain a high quality level in an unfavorable environment. By repeatedly executing the quality detection steps and adjusting the equipment parameters of the infrared detection device when the quality parameters do not meet the preset standards, low-quality data caused by poor equipment status or environmental interference is effectively avoided, ensuring that the infrared monitoring data finally obtained meets the preset quality standards, thereby improving the accuracy and reliability of the data.

[0051] In a possible implementation manner of the embodiment of the present application, the quality parameters include signal-to-noise ratio and image clarity; Determine whether the quality parameters of the initial infrared monitoring data meet the preset quality standards. If not, adjust the equipment parameters of the infrared detection device, including: Comparing the signal-to-noise ratio and image clarity with corresponding preset quality standards respectively; If any quality parameter does not meet the corresponding preset quality standard, the device parameter corresponding to the quality parameter that does not meet the preset quality standard is adjusted according to the preset corresponding relationship. The preset corresponding relationship represents the correspondence between the quality parameter and the device parameter, including the signal-to-noise ratio corresponding to the gain, and the image clarity corresponding to the focal length.

[0052] In this embodiment, the signal-to-noise ratio can be extracted from the initial infrared monitoring data through a software analysis tool. The gain is a parameter that directly controls the degree of signal amplification, so it has a more significant impact on the signal-to-noise ratio. By adjusting the gain, the signal strength can be improved to a certain extent and the impact of noise can be relatively reduced. For image clarity, the image clarity can be expressed by calculating the image sharpness, which usually involves high-pass filtering the image and then calculating the intensity or energy of the filtered image. The higher the sharpness value, the clearer the image.

[0053] The preset quality standards include the signal-to-noise ratio range and the clarity range. If the signal-to-noise ratio in the quality parameters is not within the signal-to-noise ratio range, it means that the signal-to-noise ratio does not meet the preset quality standards, and the gain device parameter should be adjusted. If the image clarity in the quality parameters is not within the clarity range, it means that the image clarity does not meet the preset quality standards, and the focus device parameter should be adjusted.

[0054] The gain difference and focal length difference can be preset. When the gain needs to be adjusted, the single adjustment method is to increase the gain, and the increased difference is the preset gain difference. When the focal length needs to be adjusted, the single adjustment method is to increase the focal length, and the increased difference is the preset focal length difference.

[0055] This embodiment uses the signal-to-noise ratio and image clarity as quality parameters and sets corresponding preset quality standards, thereby achieving precise control of the quality of infrared monitoring data.

[0056] In a possible implementation manner of the embodiment of the present application, the method further includes: Acquire multiple groups of sample data, each group of sample data includes a first sample position of a target sample collected by an infrared detection device, a second sample position of a target sample collected by a camera device, a third sample position of a target sample collected by a radar, and an actual sample position of the target sample; The first sample position, the second sample position and the third sample position are used as model inputs, and the actual sample position is used as a model output to train the model, thereby obtaining a preset fusion model after training.

[0057] In this embodiment, the collection time of each data in each group of sample data is equal, and a neural network can be selected as the model, which is not specifically limited in this embodiment.

[0058] This embodiment constructs a model to fuse data from different sensors (infrared detection, camera, radar). The model can comprehensively utilize the advantages of each sensor to more accurately determine the location of the target sample. Each sensor has its own unique detection principle and limitations. The fusion of data from multiple sensors can make up for the shortcomings of a single sensor and improve the overall positioning accuracy.

[0059] A possible implementation of the embodiment of the present application is to input the first position, the second position, and the third position into a preset fusion model, including: Determine the distance between each two positions among the first position, the second position and the third position, and compare each obtained distance with a preset distance threshold; If each distance does not exceed the preset distance threshold, the first position, the second position and the third position are input into the preset fusion model.

[0060] In this embodiment, the distance between every two locations can be determined by the Euclidean distance formula, and the preset distance threshold is artificially determined according to the equipment accuracy. If the distance does not exceed the preset distance threshold, it means that the error is within the allowable range. When the distance between the two locations exceeds the preset distance threshold, it indicates that there is an equipment error, and relevant personnel can be prompted to check.

[0061] This embodiment calculates and compares the distances between the locations collected by different sensors to preliminarily verify the consistency of these data. If there is a significant difference between the data of a certain sensor and the data of other sensors (that is, the distance exceeds a preset threshold), it may indicate that the sensor data is abnormal or has a large error. This verification helps to identify and eliminate potential data quality issues in advance, thereby improving the accuracy and reliability of subsequent fusion models.

[0062] An embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, the contents shown in the aforementioned embodiment of the target trajectory generation method based on infrared monitoring are implemented.

[0063] An electronic device is provided in an embodiment of the present application, such as Figure 2 As shown, Figure 2 The electronic device 200 shown includes: a processor 201 and a memory 203. The processor 201 and the memory 203 are connected, such as through a bus 202. Optionally, the electronic device 200 may also include a transceiver 204. It should be noted that in actual applications, the transceiver 204 is not limited to one, and the structure of the electronic device 200 does not constitute a limitation on the embodiments of the present application.

[0064] The processor 201 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor 201 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0065] The bus 202 may include a path to transmit information between the above components. The bus 202 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 202 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 Only one thick line is used in the diagram, but it does not mean that there is only one bus or only one type of bus.

[0066] The memory 203 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0067] The memory 203 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 201. The processor 201 is used to execute the application code stored in the memory 203 to implement the contents shown in the above-mentioned target trajectory generation method embodiment based on infrared monitoring.

[0068] Figure 2 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0069] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer-readable storage medium is run on a computer, the computer can execute the contents shown in the aforementioned embodiment of the method for generating target trajectories based on infrared monitoring.

[0070] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0071] The above are only some implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A target trajectory generation method based on infrared monitoring, characterized in that: include: During the inspection process of the UAV along the initial inspection route, infrared monitoring data collected by the infrared detection device, image data collected by the camera device, and radar data collected by the radar are obtained, wherein the infrared detection device, the camera device, and the radar are all arranged on the UAV; Determine a first position of the target based on the infrared monitoring data, determine a second position of the target based on the image data, and determine a third position of the target based on the radar data; Inputting the first position, the second position and the third position into a preset fusion model to obtain a target position output by the preset fusion model; A target movement trajectory is generated based on the target position at each moment, and a future movement trajectory of the target is predicted based on the target movement trajectory.

2. The target trajectory generation method based on infrared monitoring according to claim 1 is characterized in that: After predicting the future movement trajectory of the target based on the movement trajectory of the target, the method further includes: Acquire three-dimensional data of the inspection area, and dynamically determine the expected trajectory range of the drone based on the three-dimensional data and the future movement trajectory of the target; Comparing the expected trajectory with the initial inspection path to determine whether the initial inspection path exceeds the expected trajectory range; If there is an inspection section in the initial inspection path that exceeds the expected trajectory range, adjusting the inspection section based on the expected trajectory range to obtain a dynamic target inspection path; The UAV is controlled to perform inspections along the dynamic target inspection path.

3. The target trajectory generation method based on infrared monitoring according to claim 2 is characterized in that: The dynamically determining the expected trajectory range of the UAV based on the three-dimensional data and the future moving trajectory of the target includes: Based on the future moving trajectory of the target, the preset angle and the preset distance, determining the initial expected trajectory range within which the drone can monitor the target; An overlapping area between the three-dimensional data and the initial expected trajectory range is determined, and the overlapping area is removed from the initial expected trajectory range to obtain the expected trajectory range of the UAV.

4. The target trajectory generation method based on infrared monitoring according to claim 1 is characterized in that: The step of obtaining infrared monitoring data collected by the infrared detection device includes: Obtain visibility at the current time and in the current environment, and determine whether the current time is within a preset time period and whether the visibility is lower than a preset visibility threshold; If the current moment is within the preset time period and the visibility is lower than the preset visibility threshold, repeatedly performing the quality detection step until the quality parameter meets the preset quality standard; The quality detection step includes: obtaining initial infrared monitoring data collected by the infrared detection device, determining whether the quality parameters of the initial infrared monitoring data meet the preset quality standards, and if not, adjusting the equipment parameters of the infrared detection device; The initial infrared monitoring data whose quality parameters meet the preset quality standard is used as the infrared monitoring data collected by the infrared detection device.

5. The target trajectory generation method based on infrared monitoring according to claim 4 is characterized in that: The quality parameters include signal-to-noise ratio and image clarity; The determining whether the quality parameter of the initial infrared monitoring data meets the preset quality standard, and if not, adjusting the equipment parameter of the infrared detection device, comprises: Comparing the signal-to-noise ratio and the image clarity with corresponding preset quality standards respectively; If any quality parameter does not meet the corresponding preset quality standard, the device parameter corresponding to the quality parameter that does not meet the preset quality standard is adjusted according to the preset corresponding relationship. The preset corresponding relationship represents the corresponding relationship between the quality parameter and the device parameter, including the signal-to-noise ratio corresponding to the gain, and the image clarity corresponding to the focal length.

6. The target trajectory generation method based on infrared monitoring according to claim 1 is characterized in that: The method further comprises: Acquire multiple groups of sample data, each group of sample data includes a first sample position of a target sample collected by an infrared detection device, a second sample position of the target sample collected by a camera device, a third sample position of the target sample collected by a radar, and an actual sample position of the target sample; The first sample position, the second sample position and the third sample position are used as model inputs, and the actual sample position is used as a model output to train the model, thereby obtaining a preset fusion model after training.

7. The target trajectory generation method based on infrared monitoring according to claim 1 is characterized in that: The step of inputting the first position, the second position, and the third position into a preset fusion model includes: Determine the distance between each two positions among the first position, the second position and the third position, and compare each obtained distance with a preset distance threshold; If each distance does not exceed the preset distance threshold, the first position, the second position and the third position are input into a preset fusion model.

8. A computer program product, characterized in that The method comprises a computer program, which, when executed by a processor, implements the steps of the target trajectory generation method based on infrared monitoring according to any one of claims 1 to 7.

9. An electronic device, characterized in that: include: at least one processor; Memory; At least one application, wherein the at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute the target trajectory generation method based on infrared monitoring according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed in a computer, the computer is caused to execute the target trajectory generation method based on infrared monitoring according to any one of claims 1 to 7.

Images