A method and system for constructing a knowledge base for infrared target tracking and retrieval in dim environments

Abstract

This application discloses a method and system for constructing a knowledge base for infrared target tracking and retrieval in dim environments, relating to the field of infrared detection and tracking technology. The method comprises: acquiring an infrared image in a dim environment; inputting the infrared image into a target tracking model optimized based on the YOLOv8 model, and outputting target tracking information; constructing a knowledge vector library based on the infrared image and target tracking information using a retrieval enhancement generation method; performing multimodal feature fusion using the knowledge vector library, and saving the infrared data after multimodal feature fusion. This application optimizes the YOLOv8 model to construct a target tracking model, and uses the target tracking model to obtain target tracking information, thereby improving the accuracy of detection and tracking of the target tracking model; constructing and using the knowledge vector library to perform multimodal feature fusion and save infrared data, thereby achieving localized and efficient preservation of infrared data in dynamic and complex multimodal scenes.

Description

Technical Field

[0001] The present application relates to the field of infrared detection and tracking technology, and in particular to a method and system for constructing an infrared target tracking and retrieval knowledge base in a dim environment. Background Art

[0002] In complex and ever-changing dimly lit environments, accurately acquiring the real-time location, trajectory, and category information of dynamic targets is paramount for achieving intelligent and autonomous perception in unknown environments. After a natural disaster, efficient target search is crucial for accurately and effectively identifying, searching, tracking, and rescuing infrared dynamic targets in dark environments, ensuring personnel safety. To meet requirements for all-weather capabilities, good concealment, smoke penetration, and blind spot detection, infrared imaging search and tracking (IRST) technology plays a vital role in both military and civilian applications. Infrared image target detection methods can be categorized as those based on filtering local information and those based on deep learning's pixel feature representation capabilities. Deep learning-based infrared image target detection algorithms have become a major research trend. These algorithms primarily utilize convolutional neural networks (CNNs) and Transformers as foundational models for design and optimization, ensuring a balance between infrared target tracking accuracy and reasoning capabilities in dark environments.

[0003] Current infrared image target detection and tracking methods have problems such as being unable to effectively and real-time detect and track targets in dim environments, low target detection position accuracy, trajectory tracking offset, and the inability to efficiently save tracked infrared data in multimodal scenarios. Summary of the Invention

[0004] The purpose of this application is to provide a method and system for constructing an infrared target tracking retrieval knowledge base in a dim environment, which can improve the accuracy of target tracking model detection and tracking, and realize the localization and efficient preservation of infrared data in dynamic and complex multimodal scenes.

[0005] To achieve the above objectives, this application provides the following solutions.

[0006] In the first aspect, the present application provides a method for constructing an infrared target tracking retrieval knowledge base in a dim environment, the method comprising: obtaining an infrared image in a dim environment; the dim environment is an environment under conditions where visibility is lower than a preset value; inputting the infrared image into a target tracking model and outputting target tracking information; the target tracking model is optimized based on the YOLOv8 model; the target tracking information includes: the position, category and trajectory information of all detected targets; based on the infrared image and target tracking information, a knowledge vector library is constructed using a retrieval enhancement generation method; multimodal feature fusion is performed using the knowledge vector library, and the infrared data after multimodal feature fusion is saved.

[0007] In a second aspect, the present application provides a computer system comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-described methods for constructing a knowledge base for infrared target tracking and retrieval in a dim environment.

[0008] According to the specific embodiments provided in this application, this application discloses the following technical effects.

[0009] This application optimizes the YOLOv8 model to construct a target tracking model, and uses the target tracking model to obtain target tracking information, thereby improving the accuracy of target tracking model detection and tracking; constructs and uses a knowledge vector library for multimodal feature fusion, and saves infrared data, thereby achieving localized and efficient preservation of infrared data in dynamic and complex multimodal scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 A flowchart of a method for constructing an infrared target tracking and retrieval knowledge base in a dim environment provided in an embodiment of the present application.

[0012] Figure 2 This is the improved Yolov8 model provided in the embodiments of the present application.

[0013] Figure 3 A schematic diagram of the structure of the Bottleneck CSP network provided in an embodiment of the present application.

[0014] Figure 4 This is a schematic diagram of the structure of the RCCA module provided in an embodiment of the present application.

[0015] Figure 5 This is a structural diagram of the DASI dimension perception selective integration module provided in an embodiment of the present application.

[0016] Figure 6 A schematic diagram of the structure of a computer system provided for an embodiment of the application. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0018] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0019] Example 1, as Figure 1-Figure 2 As shown, this embodiment provides a method for constructing an infrared target tracking and retrieval knowledge base in a dim environment, and the method includes the following steps.

[0020] S1. Acquire an infrared image in a dim environment; a dim environment is an environment where visibility is lower than a preset value.

[0021] Furthermore, the infrared images are images captured by binocular infrared cameras (IR-Camera1 and IR-Camera2); the model of the binocular infrared camera module is InfiSense S2640-P20004S00.

[0022] Optionally, the external environment of this embodiment is: at an ambient temperature T0 = 25°C, the noise equivalent temperature difference (NETD) of the device is less than 50mK, and the operating band is [8μm, 14μm]. In this environment, the depth of the target is estimated, and the feature ID is matched and tracked.

[0023] S2. Input the infrared image into the target tracking model and output target tracking information; the target tracking model is optimized based on the YOLOv8 model; the target tracking information includes: the location, category, and trajectory information of all detected targets.

[0024] Furthermore, the target tracking model includes a backbone network, a neck network, and a head network connected in sequence; the backbone network structure is a CSPDarknet53 structure; the backbone network includes an offset learning module, a BottleneckCSP structure, and a DASI module connected in sequence; wherein, Figure 3 As shown in the figure, the BottleneckCSP structure includes a convolution operation module, an AKConv module, and an RCCA module connected in sequence; the neck network is a PANet structure; and the head network includes multiple multi-scale dynamic detection heads.

[0025] Optionally, the target tracking model can be optimized and verified through real-time inference using low-computing edge embedded devices and binocular infrared camera modules. This parameter optimization reduces computational overhead while ensuring the model's detection and tracking performance. Real-time inference verification in real-world environments ensures the accuracy and robustness of the model's detection and tracking of dynamic targets in complex, dark environments.

[0026] In practical applications, this embodiment uses CSPDarknet53 instead of Darknet53 as the backbone network structure. The Cross-Stage Partial Network (CSPNet) architecture addresses redundancy issues when fusing features at different layers. The BottleneckCSP architecture replaces the Conv layer in Bottleneck2 with the AKConv, RCCA, and DASI modules, outputting features as the weighted average of surrounding locations. Integrating the AKConv, RCCA, and DASI modules into the C2F network structure enhances the network's ability to detect and track dynamic features in complex dark environments.

[0027] Step S2 specifically includes the following steps.

[0028] S21. Utilize the offset learning module and the convolution operation module to extract the multi-scale initial feature map of the infrared image according to the offset of the convolution kernel sampling point.

[0029] In practical applications, the Offset Learning Module learns sampling point offsets from the input feature map through convolutional or fully connected layers. These offsets dynamically adjust the convolution kernel's sampling grid, allowing each kernel to sample the input feature map at different locations. This generates a tensor of offsets for each kernel sampling point. The Convolution Operation Module dynamically performs convolution using the feature map combined with the offset tensor. The offset tensor adjusts the kernel's sampling locations, enabling more flexible convolution operations based on the structure and characteristics of the input feature map.

[0030] S22. The neck network based on the PANet (Path Aggregation Network) structure uses a path aggregation mechanism to fuse features at different levels in the multi-scale initial feature map to obtain a fused multi-scale feature map.

[0031] In actual applications, the application of PANet structure enables better combination of multi-scale features at different levels and enhances the target detection effect.

[0032] S23. Based on the fused multi-scale feature map, the variable and convolutional AKConv module is used to extract geometric shape features through convolution kernels of variable shape and size to obtain a variable kernel enhanced feature map.

[0033] In practical applications, the AKConv module improves the model's feature extraction capabilities by using convolution kernels of variable shape and size. By learning the location of sampling points, the AKConv module can more flexibly adapt to the geometric characteristics of the input data, capturing more subtle and irregular shape features than standard convolution. Similar to standard convolution, AKConv also contains multiple convolution kernels, and the sampling points of these convolution kernels are not fixed but dynamically adjusted based on the characteristics of the input image and the learned offset.

[0034] S24. Figure 4 As shown in FIG, the RCCA module is used to calculate the relationship between the target feature pixel in the variable kernel enhanced feature map and all other points in the feature map, and the features of the target pixel are weighted according to the relationship to obtain the weighted variable kernel enhanced feature map.

[0035] In practical applications, a recurrent criss-cross attention module (RCCA) can be added to the convolution operation. The RCCA module consists of two cascaded CCA modules. This module calculates the correlation between the target feature pixel and all other points in the feature map and uses this correlation to weight the target pixel's features, thereby obtaining more effective target features.

[0036] S25. Figure 5 As shown in FIG, the DASI module is used to perform feature fusion on the weighted variable kernel enhanced feature map in an adaptive screening manner to obtain a dimension-aware fused feature map.

[0037] In actual application, the DASI module effectively solves the problems of feature information loss and background clutter in infrared small target detection by adaptively selecting fused features, thereby improving the saliency of small targets and ultimately improving detection performance.

[0038] S26. Use multiple multi-scale motion detection heads to extract multi-scale semantic information from the dimension-aware fusion feature map to obtain a multi-scale motion detection feature map.

[0039] Step S26 specifically includes the following steps.

[0040] S261. Use a multi-scale motion detection head to perform multi-scale semantic information processing on the dimension-aware fusion feature map to obtain a multi-scale motion detection feature map, which specifically includes the following steps.

[0041] S262. Use the five-frame difference model combined with dynamic region and depth estimation to calculate the pixel intensity difference of the dimension-aware fusion feature map of consecutive frames and generate a difference image.

[0042] Furthermore, the formula of the five-frame difference model is as follows.

[0043] .

[0044] in, The difference image after five frames of difference, For the Frame image, For the Frame image, For the Frame image.

[0045] S263. Determine the target area of the differential image through binarization to obtain a binarized image.

[0046] Optionally, the target area determination formula is as follows.

[0047] .

[0048] .

[0049] in, is a binary image, is the three-frame difference image, and τ is the coefficient value.

[0050] S264. Use a calibration method based on the camera intrinsic parameter matrix to map the pixel coordinates in the binary image to the normalized camera coordinate system.

[0051] S265. Map the binary image in the camera coordinate system to the world coordinate system to obtain a multi-scale dynamic detection feature map.

[0052] In practical applications, the addition of a multi-scale dynamic detection head improves the multi-scale semantic information processing capability of the output part, avoiding occlusions or missed detection of small objects. The multi-scale feature detection head also enhances feature detection and positioning performance.

[0053] The conversion process between the camera coordinate system and the world coordinate system is as follows.

[0054] First, to reduce the distortion that may occur in the push stream image, the pixel coordinates in the image are ) is mapped to the normalized camera coordinate system through the calibration method. The pixel coordinates are converted to normalized camera coordinates through the camera intrinsic parameter matrix , the normalized camera coordinate formula is as follows.

[0055] .

[0056] in, and is the focal length of the left and right lenses of the binocular infrared camera, The coordinate offset of the main point.

[0057] Next, the normalized camera coordinates are mapped to the complete 3D camera coordinates by combining the target’s depth value. Then, the points in the camera coordinate system are mapped to the world coordinate system through the spatial transformation using the extrinsic matrix.

[0058] The mapping formula from camera coordinates to world coordinates is as follows.

[0059] .

[0060] Where R represents the rotation matrix and T represents the translation vector.

[0061] Finally, the comprehensive transformation formula from pixel coordinates to world coordinates is as follows.

[0062] .

[0063] The mapping function of the moving target is defined by the function , associate the position of the moving target with its three-dimensional space coordinates, and define the function The formula is as follows.

[0064] .

[0065] The mapping function combines the detection results of the motion area and the depth estimation, which can achieve efficient tracking and acquisition of dynamic targets in three-dimensional space.

[0066] S27. Perform target detection and target tracking on the multi-scale dynamic detection feature map obtained at multiple scales through a dual-branch network to obtain target tracking information; the dual-branch network includes a detection branch and a tracking branch; wherein the detection branch is used for target classification and positioning, and the tracking branch is used for target trajectory tracking.

[0067] In actual application, accurate detection and continuous tracking of dynamic targets can be achieved through the collaborative cooperation of the two subtasks: detection branch and tracking branch.

[0068] S3. Based on infrared images and target tracking information, a knowledge vector library is constructed using the retrieval enhancement generation method.

[0069] Step S3 specifically includes the following steps.

[0070] S31. Based on the retrieval enhancement generation method, the infrared image is converted into an infrared feature vector using the encoding function.

[0071] S32. Construct a knowledge vector library based on the correspondence between infrared feature vectors and target tracking information.

[0072] S4. Use the knowledge vector library to perform multimodal feature fusion and save the infrared data after multimodal feature fusion.

[0073] Step S4 specifically includes the following steps.

[0074] S41. Retrieve the most relevant data features by performing similarity matching between the infrared image as a query vector and the infrared feature vectors stored in the knowledge vector library.

[0075] Furthermore, the similarity matching is cosine similarity matching.

[0076] S42. The retrieved infrared feature vector is fused with the target tracking information through the generation module of the knowledge vector library to obtain a multimodal output result.

[0077] S43. Save the multimodal output results as infrared data.

[0078] In actual application, the collected data is stored locally through the knowledge vector library retrieval-augmented generation (RAG). The infrared data R can be converted into a vector representation V using an encoding function, and the knowledge vector library is constructed by combining relevant metadata (such as time and location tags). During retrieval, the query vector q is matched with the infrared feature vector stored in the knowledge base for similarity matching (such as cosine similarity) to retrieve the most relevant data features. The generation module then fuses the retrieved infrared features with other modal information (such as text and visual features) to generate the final multimodal output. This approach provides a unified and flexible solution for the efficient storage and utilization of infrared data, particularly suitable for dynamic and complex multimodal scenarios.

[0079] The first step is infrared data encoding: convert infrared image data Convert to feature vector representation , through the encoding function The conversion formula is as follows.

[0080] .

[0081] The second step is to build a knowledge vector library :The feature vector of infrared data It is combined with its corresponding metadata m (such as timestamp, location tag, etc.) to form an entry in the knowledge vector library. The formula of the knowledge vector library is as follows.

[0082] .

[0083] The third step is the multimodal retrieval mechanism: the query vector q (which may contain text, image or infrared features) is compared with the feature vectors in the knowledge vector library to calculate the similarity (such as using cosine similarity). , to retrieve the most relevant infrared data.

[0084] .

[0085] The fourth step is to generate module integration: the retrieved infrared data features and multimodal features (other modal features) Fusion is performed and the generating function is generated Output target generation result, the formula is as follows.

[0086] .

[0087] The technical effects of this application are as follows.

[0088] This application optimizes the YOLOv8 model to construct a target tracking model and uses it to obtain target tracking information, improving the detection and tracking accuracy of the target tracking model. It also constructs and utilizes a knowledge vector library for multimodal feature fusion and preserves infrared data, achieving efficient localized preservation of infrared data in dynamic and complex multimodal scenes. Specifically, this application uses the PANet architecture to achieve effective multi-scale feature fusion. By introducing the adaptive kernel convolution AKConv module and dynamically adjusting the sampling point positions of the convolution kernel, the network can flexibly capture detailed information based on the geometric shape of the input features. This significantly improves the detection of small targets in low contrast and at long distances, enhancing the representation of key feature areas. The introduction of the DASI dimensional perception selective integration module effectively addresses the issues of feature information loss and background clutter in infrared small target detection. The introduction of the adaptive feature fusion module dynamically adjusts the feature fusion strategy based on target size, shape, motion state, and other information, maintaining high detection accuracy across a variety of target scales and shapes, enhancing infrared small target detection capabilities. The dual-branch network achieves accurate detection and stable tracking of dynamic targets, and the real-time reasoning verification of edge devices ensures the practical application performance of the model. By using a parallel convolutional network, targets can be effectively detected and tracked in real time in dark environments with complex backgrounds and unclear positive and negative sample contours and missing texture information. In dim environments, even if the target is obscured, it is impossible to continuously and accurately identify the target area of dynamic obstacles in real time, and the trajectory tracking does not deviate. By associating the feature information of the previous and next frames, a position-dependent relationship is established for the non-local domain features, and the spatial channel features are rearranged to map the similar pixel representations of deep features.

[0089] The target tracking model in this application is a target detection and tracking algorithm applied to indoor and outdoor nighttime infrared environments. In order to achieve effective detection and tracking of fuzzy contour features in dark environments, a real-time detection algorithm for infrared targets based on the spatiotemporal correlation mechanism of a deep learning network SLD-Yolov8 is proposed. The repeated cross attention module RCCA and the changeable kernel convolution AKConv module are designed to optimize the bottleneck layer Bottleneck CSP of the original Yolov8 network. This application constructs an infrared multi-target tracking and detection network configuration with a spatiotemporal correlation attention mechanism in dim environments. By associating the feature information of the previous and next frames, a position-dependent connection is established for the non-local domain features, and the spatial channel features are rearranged to map the deep feature-similar pixel representations. Based on the above configuration, the feature information of dynamic obstacles in unknown dark environments is effectively identified, and the tracking trajectory is more stable.

[0090] Example 2: This application also provides a computer system, which can be a server or a terminal, and its internal structure diagram can be as follows: Figure 6As shown. The computer system includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer system is used to provide computing and control capabilities. The memory of the computer system includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer system is used to store video tag processing data. The input / output interface of the computer system is used to exchange information between the processor and an external device. The communication interface of the computer system is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for constructing an infrared target tracking and retrieval knowledge base in a dim environment is implemented.

[0091] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer system to which the solution of the present application is applied. The specific computer system may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0092] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0093] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0094] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for constructing a knowledge base for infrared target tracking and retrieval in a dim environment, characterized in that: The method for constructing a knowledge base for infrared target tracking and retrieval in a dim environment includes: Acquire an infrared image in a dim environment; the dim environment is an environment where visibility is lower than a preset value; The infrared image is input into the target tracking model, and the target tracking information is output, specifically including: using the offset learning module and the convolution operation module to extract the multi-scale initial feature map of the infrared image according to the offset of the convolution kernel sampling point; the neck network based on the PANet structure adopts the path aggregation mechanism to fuse the features of different levels in the multi-scale initial feature map to obtain a fused multi-scale feature map; based on the fused multi-scale feature map, the variable and convolution AKConv module is used to extract the geometric shape features through the convolution kernel of variable shape and size to obtain a variable kernel enhanced feature map; the RCCA module is used to calculate the relationship between the target feature pixel point in the variable kernel enhanced feature map and all other points in the feature map, and the features of the target pixel point are weighted according to the relationship to obtain a weighted variable kernel enhanced feature map; the RCCA module can be added during the convolution operation process, and the RCCA module is composed of two CCA modules in series; by calculating the relationship between the target feature pixel point and all other points in the feature map, and using such a relationship to weight the features of the target pixel point, a more effective target feature is obtained; the DASI module is used to adaptively screen the weighted variable kernel enhanced feature map. The kernel enhanced feature map is subjected to feature fusion to obtain a dimension-aware fusion feature map; multiple multi-scale dynamic detection heads are used to extract multi-scale semantic information from the dimension-aware fusion feature map to obtain a multi-scale dynamic detection feature map; target detection and target tracking are performed on the multi-scale dynamic detection feature map obtained by a dual-branch network to obtain target tracking information; the dual-branch network includes a detection branch and a tracking branch; wherein the detection branch is used for target classification and positioning, and the tracking branch is used for target trajectory tracking; the target tracking model is obtained based on the optimization of the YOLOv8 model; the target tracking information includes: the position, category and trajectory information of all detected targets; the target tracking model includes a backbone network, a neck network and a head network connected in sequence; the structure of the backbone network is a CSPDarknet53 structure; the backbone network includes an offset learning module, a BottleneckCSP structure and a DASI module connected in sequence; wherein the BottleneckCSP structure includes a convolution operation module, an AKConv module and an RCCA module connected in sequence; the neck network is a PANet structure; the head network includes multiple multi-scale dynamic detection heads; Based on infrared images and target tracking information, a knowledge vector library is constructed using a retrieval-enhanced generation method. The collected data is locally stored through retrieval-enhanced generation of the knowledge vector library. An encoding function is used to convert the infrared data R into a vector representation V. This is combined with relevant metadata to construct the knowledge vector library, including time and location tags. The method utilizes the knowledge vector library to perform multimodal feature fusion and saves the infrared data after multimodal feature fusion, specifically including: performing similarity matching between the infrared image as the query vector and the infrared feature vector stored in the knowledge vector library to retrieve the most relevant data features; fusing the retrieved infrared feature vector with the target tracking information through the generation module of the knowledge vector library to obtain a multimodal output result; saving the multimodal output result as infrared data; wherein, during retrieval, performing similarity matching between the query vector q and the infrared feature vector stored in the knowledge library to retrieve the most relevant data features , the retrieved infrared features are fused with other modal information to generate the final multimodal output result; The specific generation process of infrared data after multimodal feature fusion is as follows: The first step is infrared data encoding: convert infrared image data Convert to feature vector representation , through the encoding function The conversion formula is as follows; ； The second step is to build a knowledge vector library :The feature vector of infrared data Combined with its corresponding metadata m, it forms an entry in the knowledge vector library. The formula of the knowledge vector library is as follows: ； The third step is the multimodal retrieval mechanism: the query vector q is used to calculate the similarity with the feature vectors in the knowledge vector library to retrieve the most relevant infrared data; ； The fourth step is to generate module integration: the retrieved infrared data features With multimodal features Fusion is performed and the generating function is generated Output target generation result, the formula is as follows; 。 2. The method for constructing a knowledge base for infrared target tracking and retrieval in a dim environment according to claim 1, characterized in that: A multi-scale motion detection head is used to process multi-scale semantic information on the dimension-aware fusion feature map to obtain a multi-scale motion detection feature map, specifically including: Using a five-frame difference model combined with dynamic region and depth estimation, the pixel intensity differences of the dimension-aware fusion feature maps of consecutive frames are calculated to generate a difference image; Determine the target area of the differential image by binarization to obtain a binary image; A calibration method is used based on the camera intrinsic parameter matrix to map the pixel coordinates in the binary image to the normalized camera coordinate system; The binary image in the camera coordinate system is mapped to the world coordinate system to obtain a multi-scale dynamic detection feature map.

3. The method for constructing a knowledge base for infrared target tracking and retrieval in a dim environment according to claim 2, characterized in that: The formula of the five-frame difference model is as follows: ； in, The difference image after five frames of difference, For the Frame image, For the Frame image, For the Frame image.

4. The method for constructing a knowledge base for infrared target tracking and retrieval in a dim environment according to claim 1, characterized in that: Based on infrared images and target tracking information, a knowledge vector library is constructed using the retrieval enhancement generation method, which includes: Based on the retrieval enhancement generation method, the infrared image is converted into an infrared feature vector using the encoding function; A knowledge vector library is constructed based on the correspondence between infrared feature vectors and target tracking information.

5. The method for constructing a knowledge base for infrared target tracking and retrieval in a dim environment according to claim 1, characterized in that: The similarity matching is cosine similarity matching.

6. The method for constructing a knowledge base for infrared target tracking and retrieval in a dim environment according to claim 1, characterized in that: The infrared image is an image captured by a binocular infrared camera; the model of the binocular infrared camera module is InfiSenseS2640-P20004S00.

7. A computer system comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for constructing a knowledge base for infrared target tracking and retrieval in a dim environment according to any one of claims 1 to 6.

Images