Image processing method, device and equipment

Abstract

The invention discloses an image processing method, device and equipment, and the method comprises the steps: firstly obtaining a to-be-processed target visible light image and a to-be-processed target infrared image, then determining the response diagrams of tracking targets corresponding to the target visible light image and the target infrared image, and finally obtaining the response diagrams of the tracking targets according to the DS evidence theory. In the embodiment of the invention, the response diagrams of the tracking targets corresponding to the target visible light image and the target infrared image are fused to obtain a fused response diagram, and then a tracking result of the tracking targets can be determined according to the fused response diagram. Therefore, the target visible light image and the target infrared image can be effectively fused to improve the tracking effect of the tracking target.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method, device and equipment. Background Art

[0002] Currently, visible light images are mostly used in target tracking problems. However, when the target is obscured, or the lighting is insufficient at night, or the distance between the camera and the target is too long, resulting in weak image contrast or low resolution, target detection and tracking will become more difficult. The change in the size of the target image caused by the change in the distance between the target and the camera, as well as the simultaneous scaling of the foreground and background of the image caused by the camera focusing, will have a certain impact on tracking and reduce the tracking effect. Currently, most related research is single-light tracking, which has great limitations.

[0003] Therefore, in order to overcome the inherent shortcomings of methods based on single-mode images, image information of different modes is usually fused to enhance scene perception capabilities and thus improve tracking effects. The commonly used fusion tracking methods currently available generally include three categories: the first is the traditional method, which uses manually designed features to represent targets, which has great limitations and may be ineffective in many cases, and has a large amount of computation, especially particle filtering; the second is the method based on sparse representation, but its efficiency is low and cannot meet real-time requirements; the third is the method based on graphs, but the modal weight is not considered when calculating the soft channel consistency, and the more reliable modality cannot be distinguished during the tracking process, and the real-time requirements cannot be met. It can be seen that the existing methods for target tracking through image fusion have not achieved good results. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to provide an image processing method, device and equipment, which can effectively improve the effect of target tracking through image fusion.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, comprising:

[0006] Acquire a visible light image and an infrared image of a target to be processed;

[0007] Determine a response map of the tracking target corresponding to each of the target visible light image and the target infrared image;

[0008] According to the DS evidence theory, the response maps obtained by the tracking targets corresponding to the target visible light image and the target infrared image are fused to obtain a fused response map;

[0009] According to the fused response map, the tracking result of the tracking target is determined.

[0010] Optionally, obtaining a target visible light image and a target infrared image to be processed includes:

[0011] The airborne optoelectronic system is equipped with a visible light and infrared dual-light camera payload to perform dual-light large and small field-of-view fusion tracking to obtain visible light images and infrared images of the target to be processed.

[0012] Optionally, determining a response map of the tracking target corresponding to each of the target visible light image and the target infrared image includes:

[0013] According to the dual-light fusion tracking method of the correlation filter, the response graphs of the tracking target corresponding to the target visible light image and the target infrared image are determined.

[0014] Optionally, the step of fusing the response graphs obtained from the tracking targets corresponding to the target visible light image and the target infrared image, respectively, according to the DS evidence theory to obtain the fused response graph includes:

[0015] According to the DS evidence theory, the target visible light image and the target infrared image are subjected to sparse processing respectively to obtain a response map matrix corresponding to the target visible light image and a response map matrix corresponding to the target infrared image;

[0016] The fused response map is calculated using the response map matrix corresponding to the target visible light image and the response map matrix corresponding to the target infrared image.

[0017] In a second aspect, an embodiment of the present application further provides an image processing device, including:

[0018] An acquisition unit, used for processing a target visible light image and a target infrared image;

[0019] A first determining unit, used to determine a response map of the tracking target corresponding to each of the target visible light image and the target infrared image;

[0020] A fusion unit is used to fuse the response images obtained by the tracking targets corresponding to the target visible light image and the target infrared image according to the DS evidence theory to obtain a fused response image;

[0021] The second determining unit is used to determine the tracking result of the tracking target according to the fused response map.

[0022] Optionally, the acquiring unit is specifically used for:

[0023] The airborne optoelectronic system is equipped with a visible light and infrared dual-light camera payload to perform dual-light large and small field-of-view fusion tracking to obtain visible light images and infrared images of the target to be processed.

[0024] Optionally, the first determining unit is specifically configured to:

[0025] According to the dual-light fusion tracking method of the correlation filter, the response graphs of the tracking target corresponding to the target visible light image and the target infrared image are determined.

[0026] Optionally, the fusion unit includes:

[0027] A processing subunit, configured to perform sparse processing on the target visible light image and the target infrared image respectively according to the DS evidence theory, so as to obtain a response map matrix corresponding to the target visible light image and a response map matrix corresponding to the target infrared image;

[0028] The calculation subunit is used to calculate a fused response map using the response map matrix corresponding to the target visible light image and the response map matrix corresponding to the target infrared image.

[0029] The embodiment of the present application also provides an image processing device, including: a processor, a memory, and a system bus;

[0030] The processor and the memory are connected via the system bus;

[0031] The memory is used to store one or more programs, and the one or more programs include instructions. When the instructions are executed by the processor, the processor executes any one of the implementations of the above-mentioned image processing method.

[0032] An embodiment of the present application further provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes any one of the implementations of the above-mentioned image processing method.

[0033] The present application provides an image processing method, device and apparatus, which first obtains a target visible light image and a target infrared image to be processed, then determines the response graphs of the tracking target corresponding to the target visible light image and the target infrared image, and then, according to the DS evidence theory, fuses the response graphs of the tracking target corresponding to the target visible light image and the target infrared image to obtain a fused response graph, and then determines the tracking result of the tracking target according to the fused response graph. Thus, the tracking effect of the tracking target can be improved by effectively fusing the target visible light image and the target infrared image. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 A flowchart of an image processing method provided in an embodiment of the present application;

[0036] Figure 2 A schematic diagram of the proposition assignment process provided in an embodiment of the present application;

[0037] Figure 3 A schematic diagram of the description of proposition uncertainty provided in an embodiment of the present application;

[0038] Figure 4 A schematic diagram of the DS fusion space interpretation provided in the embodiment of the present application;

[0039] Figure 5 DS sparse processing DS sparse processing provided in the embodiment of the present application;

[0040] Figure 6 A schematic diagram of the composition of an image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] Currently, visible light images are mostly used in target tracking problems. However, when the target is obscured, or the lighting is insufficient at night, or the distance between the camera and the target is too long, resulting in weak image contrast or low resolution, target detection and tracking will become more difficult. The change in the size of the target image caused by the change in the distance between the target and the camera, as well as the simultaneous scaling of the foreground and background of the image caused by the camera focusing, will have a certain impact on tracking and reduce the tracking effect. Currently, most related research is single-light tracking, which has great limitations.

[0042] In order to improve the tracking effect, it is considered to combine the complementary advantages of infrared and visible light images, and perform composite tracking of visible light and infrared, which can make up for the inherent shortcomings of single light tracking. In visible light images, the visible spectrum is disturbed by low illumination, high illumination and background occlusion, while infrared information can effectively overcome these problems. In infrared images, the infrared spectrum is disturbed by thermal cross and glass, while visible light information can effectively process them. These two modalities can provide strongly complementary information, so their effective fusion can significantly improve the tracking performance.

[0043] The commonly used fusion tracking methods currently include the following three categories:

[0044] (1) Traditional methods

[0045] The earliest researchers used traditional techniques such as mean shift, CamShift, Kalman filter and particle filter for fusion tracking. For example, some people studied the impact of pixel-level image fusion methods on target tracking results and compared them with tracking in single-modality videos. In this work, tracking was performed by particle filtering. Some people used the mean shift algorithm as the basis for fusion tracking, weighted the similarity of the dual-light input targets, obtained a new optimization function, and then implemented tracking through the core tracking method. However, traditional methods use hand-designed features to represent targets, which has great limitations, may be ineffective in many cases, and have a large amount of calculation, especially particle filtering.

[0046] (2) Methods based on sparse representation

[0047] Sparse representation is a possible representation of an image based on a linear combination with a sparse basis in a complete dictionary. Because of its ability to suppress noise and errors, it was first introduced into target tracking in 2009 to help create effective joint features in fusion tracking. However, due to the low efficiency of online optimization, almost all reported sparse representation function tracking algorithms cannot meet the real-time requirements. In the Bayesian filtering framework, sparse representation takes into account both intra-channel constraints and inter-channel constraints. The likelihood probability of each candidate sample is defined by the reconstruction residual and the coefficient, and the target position is represented by the candidate sample with the largest likelihood probability. The correlation between different modalities is processed by low-rank regularization, and the template is adaptively selected by sparse regularization to process appearance changes, making it more robust to modal fusion with large appearance changes. However, its efficiency is low and cannot meet the real-time requirements.

[0048] (3) Graph-based methods

[0049] Graph-based methods first cut the bounding box around the target into independent blocks, and then build the relationship between the blocks to calculate the feature representation of the target. For example, weighted sparse representation regularized graph learning, Cross-Modal, etc., these methods are also relatively slow. Weighted sparse representation regularized graph learning optimizes the affinity of the graph based on weighted sparse representation, and determines the weight of each block according to the affinity of the graph. In this work, visible light features and infrared features are cascaded by combining the weights of the foreground part. In order to suppress background information, the Cross-Modal algorithm uses the proposed cross-modal sorting algorithm to calculate the weights of image blocks. At the same time, the visible light features and infrared features are cascaded considering the weights of the foreground blocks. However, the modal weight is not considered when calculating the soft channel consistency, and the more reliable modality cannot be distinguished during the tracking process, and the real-time requirements cannot be met.

[0050] It can be seen that the above-mentioned existing methods of target tracking through image fusion have not achieved good results.

[0051] To solve the above defects, the embodiment of the present application provides an image processing method, firstly obtaining a target visible light image and a target infrared image to be processed, then determining the response graphs of the tracking target corresponding to the target visible light image and the target infrared image, and then, according to the DS evidence theory, fusing the response graphs of the tracking target corresponding to the target visible light image and the target infrared image to obtain a fused response graph, and then determining the tracking result of the tracking target according to the fused response graph. Thus, the tracking effect of the tracking target can be improved by effectively fusing the target visible light image and the target infrared image.

[0052] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] First embodiment

[0054] See also Figure 1 , is a flow chart of an image processing method provided in this embodiment, the method comprising the following steps:

[0055] S101: Acquire a target visible light image and a target infrared image to be processed.

[0056] In this embodiment, any two visible light images and infrared images of the same scene captured at the same time and subjected to fusion processing by this embodiment are defined as target visible light images and target infrared images, respectively. Moreover, this application does not limit the types of target visible light images and target infrared images for this embodiment. For example, the target visible light images and target infrared images can be color images composed of three primary colors of red (G), green (G), and blue (B), or grayscale images, etc.

[0057] In addition, the target visible light image and the target infrared image can be obtained according to actual needs through airborne optoelectronic imaging systems and other methods. For example, the visible light and infrared dual-light camera payload carried by the airborne optoelectronic system can be used to perform dual-light large and small field of view fusion tracking to obtain the target visible light image and target infrared image to be processed.

[0058] Among them, the airborne optoelectronic imaging system is an optoelectronic reconnaissance device that integrates high-precision sensor equipment. The sensors commonly carried include visible light cameras, infrared thermal imagers, GPS, inertial navigation, SAR imaging payloads, laser rangefinders, and optoelectronic encoders, etc. It has not only been widely used in the fields of aerial reconnaissance and positioning, target tracking, and border patrols, but also has huge development space and potential in the fields of geological surveys, forest fire prevention, disaster warning, traffic supervision, field search and rescue, and smart cities.

[0059] In addition, since the target visible light image has rich texture information but weak anti-interference ability, and the target infrared image has strong anti-interference ability but low resolution, considering the complementary advantages of the target visible light image and the target infrared image, in order to better track the target, the airborne optoelectronic system can be equipped with visible light and infrared dual-light camera payloads to perform dual-light large and small field of view fusion tracking to ensure that more useful information of the scene is obtained and used reasonably.

[0060] In addition, in order to solve the problem of mismatched spatial information of bifocal images caused by changes in bifocal fields of view, especially the problem of rapid changes in image scale caused by rapid zooming, the present application also establishes a model of bifocal imaging and spatial physical coordinates, projects the bifocal image into the same spatial physical coordinate system, aligns the scale in three-dimensional space, and then projects it back into a two-dimensional bifocal image to complete the normalization of the image zoom scale.

[0061] S102: Determine the response graphs of the tracking target corresponding to the target visible light image and the target infrared image respectively.

[0062] In this embodiment, after obtaining the target visible light image and target infrared image to be processed through step S101, further, in order to better fuse the information of the visible light and infrared images, the response graphs of the tracking targets corresponding to the target visible light image and the target infrared image can be determined according to the dual-light fusion tracking method of the correlation filter, so as to execute the subsequent step S103.

[0063] Among them, in the dual-light fusion tracking method based on correlation filter, decision-level fusion can be adopted to maximize the fusion efficiency. That is, fusion processing is performed on the response graphs obtained by tracking the target by visible light and infrared respectively. Although the visible light and infrared response graphs are directly added or weighted added to be fused, the tracking effect can be improved, but the advantages of both cannot be fully utilized. Therefore, this paper adopts the fusion method of the following step S103 to process the response graphs of the target visible light image and the target infrared image corresponding to the tracking target.

[0064] S103: According to the DS evidence theory, the response maps obtained by the tracking targets corresponding to the target visible light image and the target infrared image are fused to obtain a fused response map.

[0065] In this embodiment, after determining the response graphs of the tracking targets corresponding to the target visible light image and the target infrared image respectively in step S102, the response graphs of the tracking targets corresponding to the target visible light image and the target infrared image respectively can be fused according to the DS evidence theory to obtain a fused response graph for executing the subsequent step S104.

[0066] Specifically, an optional implementation method is to first perform sparse processing on the target visible light image and the target infrared image respectively according to the DS (Dempster-Shafer) evidence theory to obtain the response map matrix corresponding to the target visible light image and the response map matrix corresponding to the target infrared image, and then use the response map matrix corresponding to the target visible light image and the response map matrix corresponding to the target infrared image to calculate the fused response map.

[0067] In this implementation, when tracking based on visible light and infrared composite, each update of the filter will generate independent response maps, and it is very important to fuse the two information to obtain a more accurate target position. Since camera imaging is affected by many environmental factors, the imaging quality is uncertain, and it is also uncertain which tracking is better. Camera zoom will also bring more uncertainty factors. Therefore, this application selects the DS evidence theory, a data fusion method that is good at handling uncertainty problems, to fuse the visible and infrared information (i.e., the response map obtained by the tracking target corresponding to the target visible light image and the target infrared image).

[0068] Among them, DS evidence theory is a fuzzy set theory obtained by generalizing traditional Bayesian reasoning. Since Bayesian reasoning requires prior information about data samples, it has great limitations in practical applications. The generalized DS evidence theory does not rely on prior information, and its theoretical form is simple, which can be better applied to the analysis of problems dealing with uncertainty. It takes the credibility of the proposition as its construction basis. As a basic concept of probability, this probability is the assignment of the proposition by people through the analysis of evidence. The relationship between decision makers, evidence and propositions is expressed by Figure 2 Express more clearly.

[0069] Next, several basic concepts of DS evidence theory are introduced:

[0070] (1) Identification framework

[0071] The discriminant framework is a non-empty set of sample spaces in DS theory, which consists of possible situations of basic propositions and describes the problem to be analyzed more comprehensively. It is denoted as Ω = {A1, A2, ... A n}. The elements in the set are the primitives of the proposition and they are mutually exclusive.

[0072] (2) Basic probability distribution function

[0073] Basic probability is used to express the support of evidence for a proposition. The probability range is between [0,1], and for all possible situations of a proposition, the sum of its corresponding basic probabilities in the entire domain is 1. For any proposition A, Ω is the set of all its possible situations. Its basic probability distribution function m is defined to represent the probability that the evidence is credible, satisfying the following formula:

[0074]

[0075] in, Represents the empty set.

[0076] If for If m(A)>0, then A is also called the focal element. One difference from probability theory is that

[0077] (3) Trust Function

[0078] The trust function is used to describe the sum of the basic probabilities of all possibilities of proposition A, that is, the degree of trust in A, also known as the lower limit function, and the formula is defined as follows:

[0079]

[0080] Among them, B is a subset of A, and the trust function is similar to the basic probability distribution function, which is This shows that the degree of trust in an issue is not black and white, and there is room for doubt. In addition, the trust function also has the following properties:

[0081]

[0082] Bel(Ω)=1

[0083]

[0084] (4) Likelihood function

[0085] The likelihood function is used to describe the sum of the basic probabilities of all possibilities contained in proposition A, that is, the degree of non-rejection of question A. The formula is defined as follows:

[0086]

[0087] The likelihood function is defined as the degree to which the non-A proposition is not supported, that is, The sum of the basic probabilities. It can be expressed as:

[0088]

[0089] Similarly, the likelihood function has similar properties to the belief function:

[0090]

[0091] PL(Ω)=1

[0092]

[0093] Regarding the degree of confirmation of Proposition A, the description of DS theory is as follows Figure 3 shown.

[0094] Furthermore, the DS fusion rules are introduced in detail:

[0095] DS theory is mostly used to fuse one-dimensional signal data from multiple sensors, process the data, and obtain quantitative data results to help decision makers make decisions.

[0096] For two pieces of evidence, suppose m1 and m2 are evidence bodies A and B respectively. j , B i The probability distribution function of , the DS fusion rule is expressed as follows:

[0097]

[0098] The fusion process of these two bodies of evidence is as follows Figure 4 shown.

[0099] The vertical axis represents the probability distribution function m1 distributed among A1, A2, ..., A k The horizontal axis represents the probability distribution function m2 distributed on B1, B2, ..., B k The shaded part indicates the probability of being assigned to A at the same time. j , B i probability.

[0100] On this basis, the DS theory is extended to two dimensions. The response map obtained by tracking is a two-dimensional matrix, and each point represents the probability of the target appearing. Through sparse processing, the response map matrix is ​​normalized, and the points under a given threshold α are set to 0, which can reduce the amount of calculation. The sparse formula is as follows:

[0101]

[0102] Where F represents the response map matrix, and (x, y) represents the matrix coordinates. When the threshold α is selected as 0, the amount of calculation on the two-dimensional layer can be reduced through sparse processing, and the fusion rate can be increased, such as 2.5fps. The response map before and after sparse processing is compared. Figure 5 As shown. Assume that the normalized visible response map matrix is ​​F v , the infrared response map matrix is ​​F i, then the response map obtained by DS fusion of the target visible light image and the target infrared image corresponding to the tracking target is obtained, and the calculation formula of the fused response map is as follows:

[0103]

[0104] Among them, G(x,y) represents the fused response map.

[0105] S104: Determine the tracking result of the tracking target according to the fused response map.

[0106] It should be noted that since camera focusing will cause image defocus and blur, resulting in target loss, blur detection is introduced in fusion to improve the fusion method. In the method of evaluating image clarity and blur, the energy gradient function EOG takes into account the gradients in both horizontal and vertical directions, and the calculation is relatively simple, so this application selects the EOG function as a means of judging the clarity and blur of the image. In order to reduce the amount of calculation, the local blur detection method is adopted, and the image part in the tracking frame is selected as the area for calculating the blur, and then the visible light energy gradient E is obtained respectively. v and infrared energy gradient E i Normalization is performed, and the specific calculation formula is as follows:

[0107]

[0108] Thus, the visible light blur e can be obtained v and infrared ambiguity e i .

[0109] In summary, the image processing method provided in this embodiment first obtains the target visible light image and target infrared image to be processed, then determines the response graphs of the tracking target corresponding to the target visible light image and the target infrared image, and then, according to the DS evidence theory, fuses the response graphs of the tracking target corresponding to the target visible light image and the target infrared image to obtain the fused response graph, and then determines the tracking result of the tracking target according to the fused response graph. Therefore, the tracking effect of the tracking target can be improved by effectively fusing the target visible light image and the target infrared image.

[0110] Second embodiment

[0111] This embodiment will introduce an image processing device. For related content, please refer to the above method embodiment.

[0112] See also Figure 6 , is a schematic diagram of the composition of an image processing device provided in this embodiment, the device comprising:

[0113] An acquisition unit 601 is used for obtaining a target visible light image and a target infrared image to be processed;

[0114] A first determining unit 602 is used to determine a response map of the tracking target corresponding to each of the target visible light image and the target infrared image;

[0115] A fusion unit 603 is used to fuse the response images obtained by the tracking targets corresponding to the target visible light image and the target infrared image according to the DS evidence theory to obtain a fused response image;

[0116] The second determining unit 604 is used to determine the tracking result of the tracking target according to the fused response map.

[0117] In an implementation of this embodiment, the acquiring unit 601 is specifically configured to:

[0118] The airborne optoelectronic system is equipped with a visible light and infrared dual-light camera payload to perform dual-light large and small field-of-view fusion tracking to obtain visible light images and infrared images of the target to be processed.

[0119] In an implementation of this embodiment, the first determining unit 602 is specifically configured to:

[0120] According to the dual-light fusion tracking method of the correlation filter, the response graphs of the tracking target corresponding to the target visible light image and the target infrared image are determined.

[0121] In an implementation of this embodiment, the fusion unit 603 includes:

[0122] A processing subunit, configured to perform sparse processing on the target visible light image and the target infrared image respectively according to the DS evidence theory, so as to obtain a response map matrix corresponding to the target visible light image and a response map matrix corresponding to the target infrared image;

[0123] The calculation subunit is used to calculate a fused response map using the response map matrix corresponding to the target visible light image and the response map matrix corresponding to the target infrared image.

[0124] In summary, the image processing device provided in this embodiment first obtains the target visible light image and target infrared image to be processed, then determines the response graphs of the tracking target corresponding to the target visible light image and the target infrared image, and then, according to the DS evidence theory, fuses the response graphs of the tracking target corresponding to the target visible light image and the target infrared image to obtain the fused response graph, and then determines the tracking result of the tracking target according to the fused response graph. Therefore, the tracking effect of the tracking target can be improved by effectively fusing the target visible light image and the target infrared image.

[0125] Furthermore, an embodiment of the present application also provides an image processing device, including: a processor, a memory, and a system bus;

[0126] The processor and the memory are connected via the system bus;

[0127] The memory is used to store one or more programs, and the one or more programs include instructions. When the instructions are executed by the processor, the processor executes any one of the implementation methods of the above-mentioned image processing method.

[0128] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes any one of the implementation methods of the above-mentioned image processing method.

[0129] It can be known from the description of the above implementation mode that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0130] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0131] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0132] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, characterized in that: include: Acquire a visible light image and an infrared image of a target to be processed; Determine a response map of the tracking target corresponding to each of the target visible light image and the target infrared image; According to the DS evidence theory, the response maps obtained by the tracking targets corresponding to the target visible light image and the target infrared image are fused to obtain a fused response map; According to the fused response map, the tracking result of the tracking target is determined.

2. The method according to claim 1, characterized in that The step of obtaining a target visible light image and a target infrared image to be processed includes: The airborne optoelectronic system is equipped with a visible light and infrared dual-light camera payload to perform dual-light large and small field-of-view fusion tracking to obtain visible light images and infrared images of the target to be processed.

3. The method according to claim 1, characterized in that The step of determining a response map of the tracking target corresponding to each of the target visible light image and the target infrared image comprises: According to the dual-light fusion tracking method of the correlation filter, the response graphs of the tracking target corresponding to the target visible light image and the target infrared image are determined.

4. The method according to claim 1, characterized in that: According to the DS evidence theory, the response graphs obtained by the tracking targets corresponding to the target visible light image and the target infrared image are fused to obtain the fused response graph, including: According to the DS evidence theory, the target visible light image and the target infrared image are subjected to sparse processing respectively to obtain a response map matrix corresponding to the target visible light image and a response map matrix corresponding to the target infrared image; The fused response map is calculated using the response map matrix corresponding to the target visible light image and the response map matrix corresponding to the target infrared image.

5. An image processing device, characterized in that: include: An acquisition unit, used for processing a target visible light image and a target infrared image; A first determining unit, used to determine a response map of the tracking target corresponding to each of the target visible light image and the target infrared image; A fusion unit is used to fuse the response images obtained by the tracking targets corresponding to the target visible light image and the target infrared image according to the DS evidence theory to obtain a fused response image; The second determining unit is used to determine the tracking result of the tracking target according to the fused response map.

6. The device according to claim 5, characterized in that The acquisition unit is specifically used for: The airborne optoelectronic system is equipped with a visible light and infrared dual-light camera payload to perform dual-light large and small field-of-view fusion tracking to obtain visible light images and infrared images of the target to be processed.

7. The device according to claim 5, characterized in that The first determining unit is specifically configured to: According to the dual-light fusion tracking method of the correlation filter, the response graphs of the tracking target corresponding to the target visible light image and the target infrared image are determined.

8. The device according to claim 5, characterized in that The fusion unit comprises: A processing subunit, configured to perform sparse processing on the target visible light image and the target infrared image respectively according to the DS evidence theory, so as to obtain a response map matrix corresponding to the target visible light image and a response map matrix corresponding to the target infrared image; The calculation subunit is used to calculate a fused response map using the response map matrix corresponding to the target visible light image and the response map matrix corresponding to the target infrared image.

9. An image processing device, characterized in that: include: Processor, memory, system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by the processor, the processor executes the method according to any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the method according to any one of claims 1 to 4.

Images