A target search method, system and device based on infrared image

Abstract

The present invention relates to the field of image processing technology, and in particular to a method, system and device for target search based on infrared images, wherein the method comprises the following steps: acquiring an infrared image and extracting feature points; judging whether the feature points meet the image offset condition, if not, identifying them as background objects, and if they meet the condition, identifying them as target objects; calculating the radiation influence coefficient of the background object; calculating the motion influence coefficient of the background object based on the radiation influence coefficient; calculating the environmental influence coefficient of the target object based on the radiation influence coefficient and the motion influence coefficient; performing contrast enhancement processing on the third infrared image of the next frame based on the environmental influence coefficient to obtain a third enhanced image; and searching for the target object based on the third enhanced image. The present invention realizes rapid search and accurate identification of target objects in complex environments while ensuring that the image quality is not damaged, thereby improving the dynamic range of infrared images and the identification capability of target objects.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a target search method, system and device based on infrared images. Background Art

[0002] Infrared images are images obtained by capturing infrared radiation emitted by target objects through infrared sensors. Infrared imaging technology has been widely used in military reconnaissance, security monitoring, industrial inspection, medical diagnosis and other fields because of its ability to penetrate darkness and certain obstacles and capture the thermal radiation characteristics of objects.

[0003] However, in practical applications, the acquisition of infrared images is often affected by environmental factors. For example, in a high-temperature environment, high-thermal radiation objects in the background may affect the dynamic range of the image, making the target objects with low thermal radiation less prominent in the image and difficult to be effectively identified and searched. This phenomenon is particularly obvious in infrared images acquired remotely by satellites and other means.

[0004] To overcome this problem, traditional solutions usually rely on image enhancement technology, which aims to improve the visibility of target objects by adjusting image contrast, brightness and other parameters. However, existing technologies still have limitations when processing high dynamic range infrared images, especially when distinguishing objects with similar temperatures, making it difficult to achieve high-accuracy target recognition. Summary of the invention

[0005] In order to solve the technical problem of enhancing infrared images, the present invention provides a target search method based on infrared images, which optimizes the infrared images and realizes rapid search and accurate recognition of target objects in complex environments while ensuring that the image quality is not damaged, thereby improving the dynamic range of infrared images and the recognition ability of target objects.

[0006] In a first aspect, the present invention provides a target search method based on infrared images, comprising the steps of: acquiring a first infrared image and extracting a first feature point; acquiring a second infrared image of the next frame of the first infrared image and extracting a second feature point; judging whether the second feature point satisfies an image offset condition based on the first feature point, if not, judging the second feature point to be a static feature point, and if so, judging the second feature point to be a moving feature point; identifying background objects based on the static feature points and identifying target objects based on the moving feature points; calculating the radiation influence coefficient of the background object; calculating the motion influence coefficient of the background object based on the radiation influence coefficient; calculating the environmental influence coefficient of the target object based on the radiation influence coefficient and the motion influence coefficient; performing contrast enhancement processing on a third infrared image of the next frame based on the environmental influence coefficient to obtain a third enhanced image; and searching for the target object based on the third enhanced image.

[0007] As a preferred solution, the first infrared image, the second infrared image and the third infrared image are grayscale infrared images, and the grayscale infrared images are obtained by grayscale processing the collected infrared images.

[0008] As a preferred solution, judging whether an image feature point satisfies an image offset condition includes the following steps: calculating an optical flow vector of a first feature point and an optical flow vector of a second feature point; obtaining the optical flow vectors with the largest number of the same in the first infrared image to obtain a first optical flow vector; obtaining the optical flow vectors with the largest number of the same in the second infrared image to obtain a second optical flow vector; comparing the difference between the second optical flow vector and the first optical flow vector to determine an offset vector of the second infrared image; judging whether the optical flow vector corresponding to the second feature point is the same as the offset vector; if they are the same, the image offset condition is not satisfied; if they are not the same, the image offset condition is satisfied.

[0009] As a preferred solution, calculating the radiation influence coefficient of the target object includes the steps of: determining the radiation influence domain of the target object according to the target object, the range of the radiation influence domain of the target object being the minimum circumscribed circle of the target object; judging whether there is a background object in the radiation influence domain, and if so, determining the radiation influence coefficient by the distance between the background object and the target object and the difference in the average grayscale value between the background object and the target object; wherein, the smaller the distance between the background object and the target object, the greater the motion influence coefficient; the greater the difference in the average grayscale value between the background object and the target object, the greater the motion influence coefficient.

[0010] As a preferred solution, determining the difference in average grayscale values ​​between a background object and a target object includes the following steps: marking both the target object and the background object as objects to be identified; determining the influence domain of the object to be identified with the center of mass of the object to be identified as the radiation center; obtaining all direction change lines passing through the radiation center within the influence domain, and determining characteristic direction change lines therefrom; obtaining the average value of the grayscale change values ​​on the characteristic direction change lines, which is the average grayscale value of the object to be identified; and determining the difference in average grayscale values ​​between the background object and the target object based on the average grayscale value of the object to be identified.

[0011] As a preferred solution, calculating the motion influence coefficient of the background object includes the steps of: comparing the displacement information between the first feature point and the second feature point, and calculating the moving direction vector; determining the motion influence domain of the target object according to the moving direction vector, the range of the motion influence domain of the target object is a range determined by taking the center of mass of the target object in the first infrared image as the center of the circle and the modulus of the moving direction vector as the radius; judging whether there is a background object in the motion influence domain, and if so, determining the motion influence coefficient by the motion angle between the background object and the target object, and the difference in average grayscale values ​​between the background object and the target object; wherein, the smaller the motion angle between the background object and the target object, the larger the motion influence coefficient; the larger the difference in average grayscale values ​​between the background object and the target object, the larger the motion influence coefficient.

[0012] As a preferred solution, when calculating the environmental impact coefficient of the target object, the larger the radiation impact coefficient, the larger the environmental impact coefficient; the larger the motion impact coefficient, the larger the environmental impact coefficient.

[0013] As a preferred solution, contrast enhancement processing includes the steps of: obtaining all gradient value sets of each target object; mapping the gradient value set to a new intensity value in an incremental manner according to the environmental impact coefficient, thereby achieving contrast enhancement processing of the third infrared image.

[0014] In a second aspect, a target search system based on infrared images is provided, the system comprising:

[0015] A feature extraction module, used to extract a first feature point from the acquired first infrared image, and to extract a second feature point from a second infrared image of a next frame of the acquired first infrared image;

[0016] A motion judgment module, used for judging whether the second feature point satisfies the image shift condition based on the first feature point, and if not, judging the second feature point as a stationary feature point; if so, judging the second feature point as a motion feature point;

[0017] An object recognition module, used to recognize background objects based on static feature points and to recognize target objects based on moving feature points;

[0018] The first calculation module is used to calculate the radiation influence coefficient of the background object;

[0019] A second calculation module is used to calculate the motion influence coefficient of the background object based on the radiation influence coefficient;

[0020] A third calculation module, used to calculate the environmental impact coefficient of the target object based on the radiation impact coefficient and the motion impact coefficient;

[0021] An image enhancement processing module, used for performing contrast enhancement processing on the third infrared image of the next frame based on the environmental impact coefficient to obtain a third enhanced image;

[0022] The target search module is used to search for the target object based on the third enhanced image.

[0023] In a third aspect, a target search device is provided, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the device executes the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0024] Compared with the prior art, the above technical solution conceived by the present invention has the following beneficial effects:

[0025] 1. In order to search for the target, the present invention can ensure that the target object is not affected by the radiation of the surrounding heat source through contrast enhancement processing, thereby avoiding the problems of feature blur and target tracking interruption caused by background objects. Specifically, the radiation influence coefficient of the background object is calculated by using adjacent frames and combining the optical flow method to obtain the motion vector therein, and then correcting it based on the motion factor. This is an important feature of the present invention to achieve high-accuracy target recognition, and the environmental influence coefficient determined based on these two factors is used for contrast enhancement processing, thereby reducing and eliminating the interference caused by motion and radiation, and providing better image support for subsequent target search.

[0026] 2. The scheme adopted by the present invention does not need to be analyzed and obtained from the color space to search for the target object. Therefore, the collected infrared image can be grayed to obtain an infrared grayed image, which has the following advantages: a. The infrared grayed image only contains brightness information but not color information, which reduces the amount of data to be processed, thereby reducing the computational complexity and the required computing resources; b. Due to the reduction in the amount of data, the processing speed is usually improved, which is very important for real-time or near real-time applications; c. In grayscale images, feature points are easier to detect and track because they are not affected by color changes, which helps to improve the stability and accuracy of the optical flow method; d. After grayscale processing, the influence of illumination changes on feature point detection and tracking can be further reduced; e. Grayscale can reduce the influence of color noise on optical flow calculation because noise may be more obvious in color space. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 A flowchart of an implementation of a target search method based on infrared images provided by an embodiment of the present invention;

[0029] Figure 2 A flowchart for determining whether an image offset condition is met is provided by an embodiment of the present invention;

[0030] Figure 3 A flow chart for calculating the radiation influence coefficient provided by an embodiment of the present invention;

[0031] Figure 4 A flowchart for determining the difference in average grayscale values ​​between a background object and a target object provided by an embodiment of the present invention;

[0032] Figure 5 A flow chart for calculating a motion influence coefficient according to an embodiment of the present invention;

[0033] Figure 6 A flowchart for implementing contrast enhancement processing provided by an embodiment of the present invention;

[0034] Figure 7 A schematic diagram of the structure of a target search system based on infrared images provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the target search method based on infrared images proposed by the present invention, its specific implementation, structure, features and effects, in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0036] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0037] A specific solution of a target search method based on infrared images provided by the present invention is described in detail below with reference to the accompanying drawings.

[0038] In order to efficiently search for target objects based on infrared images, it is necessary to use high-resolution and high-sensitivity infrared sensors carried by satellites to capture infrared radiation signals emitted by ground objects. These infrared radiation signals are generated by the heat of ground objects. The infrared radiation signals are converted into electrical signals and transmitted to the ground station through the satellite's communication system after digital processing. Finally, several continuous infrared images of the surface are obtained, and the target objects are searched based on the continuous infrared images obtained.

[0039] When using infrared images to search for target objects, the contrast between the target object and the background in the infrared image is low due to factors such as the environment and weather. Therefore, the image needs to be enhanced according to image features.

[0040] Reference Figure 1 In order to solve this problem, the present invention proposes a target search method based on infrared images, including steps S100-S900.

[0041] S100, acquiring a first infrared image and extracting a first feature point.

[0042] By processing the first infrared image, an initial reference frame is established, which contains the state of the target area or object at a specific point in time.

[0043] Feature points are points extracted from images with unique properties, such as edges, corners, or significant texture changes. These points are relatively stable in image sequences and can be used as a basis for tracking and comparison. The extraction of the first feature points is crucial for subsequent image analysis, and they can serve as a starting point for tracking the movement or changes of the target object.

[0044] S200, acquiring a second infrared image of a frame next to the first infrared image, and extracting second feature points.

[0045] Specifically, in this embodiment, extracting the first feature point or the second feature point includes the steps of:

[0046] Use edge detection algorithms to extract edge information of images;

[0047] A feature point detection algorithm is applied to the edge image to extract the first feature point or the second feature point.

[0048] In a preferred embodiment, the edge detection algorithm is the Canny algorithm, and the feature point detection algorithm is the FAST algorithm.

[0049] By acquiring infrared images of consecutive frames, the movement changes of the target object over time can be observed. The second infrared image is the next infrared image frame of the first infrared image, which ensures the continuity of the time series. After the second feature point is matched and compared with the first feature point, the movement changes of the object between the two frames can be determined.

[0050] The solution adopted by the present invention does not need to be analyzed and obtained from the color space when searching for the target object. Therefore, the collected infrared image can be grayed to obtain an infrared grayed image, which has the following advantages: 1. The infrared grayed image only contains brightness information but not color information, which reduces the amount of data to be processed, thereby reducing the computational complexity and the required computing resources; 2. Since the amount of data is reduced, the processing speed is usually improved, which is very important for real-time or near real-time applications; 3. In grayscale images, feature points are easier to detect and track because they are not affected by color changes, which helps to improve the stability and accuracy of the optical flow method; 4. After grayscale processing, the influence of illumination changes on feature point detection and tracking can be further reduced; 5. Grayscale can reduce the influence of color noise on optical flow calculation because noise may be more obvious in color space.

[0051] Therefore, as a preferred solution, in the present invention, the first infrared image, the second infrared image and the third infrared image are grayscale infrared images, wherein the second infrared image is the next frame of infrared image of the first infrared image, and the third infrared image is the next frame of infrared image of the second infrared image. The grayscale infrared image is obtained by graying the collected infrared image.

[0052] S300, judging whether the second feature point satisfies an image shift condition based on the first feature point, if not, judging the second feature point to be a stationary feature point, if yes, judging the second feature point to be a moving feature point.

[0053] By comparing the optical flow fields of the previous infrared image and the current infrared image, since only some objects in the infrared image are in motion, most of the optical flow vectors in the infrared image are the same. Therefore, the offset direction of the current infrared image can be obtained based on the largest number of optical flow vectors, thereby distinguishing the target object from the background object.

[0054] Reference Figure 2 , the step of determining whether the image offset condition is met includes steps S301-S304:

[0055] S301, calculating the optical flow vector of the first feature point and the optical flow vector of the second feature point;

[0056] In this embodiment, the calculation of the optical flow vector is implemented using an optical flow estimation algorithm, which can be implemented through a variety of methods, such as the Lucas-Kanade algorithm, the Licas-Kanade improved algorithm, the Dis Flow algorithm, the Horn–Schunck algorithm, and the like.

[0057] S302, obtaining the optical flow vectors with the largest number of the same in the first infrared image to obtain the first optical flow vector; obtaining the optical flow vectors with the largest number of the same in the second infrared image to obtain the second optical flow vector;

[0058] S303, comparing the difference between the second optical flow vector and the first optical flow vector to determine an offset vector of the second infrared image;

[0059] In this embodiment, comparing the difference between the second optical flow vector and the first optical flow vector to determine the offset vector of the second infrared image includes steps S3031-S3035:

[0060] S3031, converting the first optical flow vector into a first optical flow amplitude and a first optical flow angle;

[0061] S3032, converting the second optical flow vector into a second optical flow amplitude and a second optical flow angle;

[0062] S3033, comparing the second optical flow amplitude with the first optical flow amplitude to obtain a first amplitude difference value;

[0063] S3034, comparing the second optical flow angle with the first optical flow angle to obtain a first angle difference value;

[0064] S3035: Based on the first amplitude difference value and the first angle difference value, convert and obtain the determined offset vector of the second infrared image, that is, determine the direction of the offset vector according to the first angle difference value, and determine the modulus of the offset vector according to the first amplitude difference value.

[0065] S304, determining whether the optical flow vector corresponding to the second feature point is the same as the offset vector. If they are the same, the image offset condition is not satisfied; if they are not the same, the image offset condition is satisfied.

[0066] Furthermore, based on the moving feature points, the target object can be determined, and based on the static feature points, the background object can be determined. After distinguishing the target object from the background object, the basic conditions for image enhancement and target search are laid.

[0067] S400, identifying background objects based on static feature points, and identifying target objects based on moving feature points.

[0068] In infrared images, stationary feature points are background objects, that is, background objects, which may be heat sources such as various mechanical equipment, electrical appliances, buildings, etc.

[0069] In infrared images, motion feature points are moving objects, namely target objects, which may be heat sources such as people, vehicles, and animals.

[0070] A heat source can exert a radiative influence on a target object.

[0071] In practical applications, the acquisition of infrared images is often affected by environmental factors. For example, in a high-temperature environment, high-thermal radiation objects in the background may affect the dynamic range of the infrared image, making the target object with low thermal radiation not prominent enough in the image and difficult to be effectively identified and searched. Therefore, the present invention enhances the infrared image according to the infrared image features around the target object in the image, thereby facilitating the search for the target object.

[0072] To enhance the infrared image, the influence of the radiation factor of the background object must first be clarified and calculated in step S500.

[0073] S500, calculating the radiation influence coefficient of the background object.

[0074] Reference Figure 3 Furthermore, different background objects have different radiation influence coefficients on the target object. In order to improve the capture of background objects with large radiation influence coefficients, the present invention proposes a preferred solution, including steps S501-S502:

[0075] S501, determining a radiation influence domain of the target object according to the target object, wherein the range of the radiation influence domain of the target object is the minimum circumscribed circle of the target object;

[0076] S502, determining whether there is a background object in the radiation influence domain, and if so, calculating the radiation influence coefficient of the background object.

[0077] Specifically, the radiation influence coefficient of the background object is calculated by determining the radiation influence coefficient through the distance between the background object and the target object and the difference value of the average gray value between the background object and the target object.

[0078] Among them, the smaller the distance between the background object and the target object, the greater the motion influence coefficient; the greater the difference in the average gray value between the background object and the target object, the greater the motion influence coefficient.

[0079] Reference Figure 4 , determining the difference in average grayscale values ​​between the background object and the target object, including steps S511-S515:

[0080] S511, marking both the target object and the background object as objects to be identified;

[0081] S512, determining the influence domain of the object to be identified with the centroid of the object to be identified as the radiation center;

[0082] In the present invention, the radiation influence domain of the background object can be expressed as an n×n neighborhood; as a preferred solution, the value of n can be any integer from 2 to 10; in a preferred embodiment, the value of n is 5.

[0083] S513, obtaining all direction change lines passing through the radiation center within the influence domain, and determining characteristic direction change lines therefrom;

[0084] In this embodiment, starting from the radiation center, rays of all possible directions passing through the influence domain are drawn as direction change lines, and the difference value of the grayscale value between the radiation center and the boundary of the influence domain is calculated; the difference values ​​of all direction change lines are sorted, and the direction change line with the largest difference value change is determined as the characteristic direction change line;

[0085] S514, obtaining the average value of the grayscale change value on the characteristic direction change line, which is the average grayscale value of the object to be identified.

[0086] In this embodiment, the grayscale is discrete 0-255, from black to white; along the feature direction change line, the grayscale sampling is performed on the pixel points on the feature direction change line, the grayscale change value between adjacent pixel points is calculated, all the grayscale change values ​​are summed up and then divided by the number of grayscale change values, and the average value of the grayscale change values ​​on the feature direction change line is obtained, which is the average grayscale value of the object to be identified.

[0087] S515, determining a difference value of the average grayscale value between the background object and the target object based on the average grayscale value of the object to be identified.

[0088] In a preferred embodiment, the difference in average grayscale values ​​between the background object and the target object = the average grayscale value of the target object - the average grayscale value of the background object within the influence domain of the target object, wherein the influence domain of the target object is the influence domain of the aforementioned object to be identified.

[0089] In the infrared image, the gray value of the object to be identified decreases from the inside to the outside. The gray value outside the object to be identified changes faster, and the gray value inside the object to be identified changes slower. If the target object is within the infrared radiation range of the background object, the target object will definitely be affected by the heat source. Therefore, although the background object of the present invention defines an n×n neighborhood, the influence it exerts on the target object can be reflected by the calculated radiation influence coefficient.

[0090] It should be noted that there may be multiple background objects in the radiation influence domain. When there are multiple background objects in the radiation influence domain, it is necessary to calculate the radiation influence coefficients of different background objects on the target object respectively.

[0091] As a preferred solution, the present invention uses the following formula to calculate the radiation influence coefficient:

[0092]

[0093] Where:

[0094] It is The radiation impact coefficient of each target object;

[0095] is a hyperbolic function;

[0096] is any target object;

[0097] It is any background object within the influence domain of the target object;

[0098] is the number of background objects within the influence domain of the target object;

[0099] It is The average gray value of the target object;

[0100] It is The target object has the following influence area: The average gray value of background objects;

[0101] It is The target object and its influence area The distance between background objects.

[0102] For the target object in the image obtained from the previous frame to the current frame, since the thermal radiation generated by the background objects around the target object will affect the infrared image of the target object, it is necessary to calculate the difference between the average grayscale value of the pixel point of the target object and the average grayscale value of the background object Δ , the value Δ The larger the value, the higher the average gray value of the background object is compared with the average gray value of the target object, which further indicates that the background object has a stronger radiation effect on the target object, so its thermal radiation influence coefficient is higher, which is similar to There is a positive correlation.

[0103] For the distance between the target object and the background object , the value The smaller the distance between the target object and the background object, the closer the distance between the target object and the background object. Therefore, the radiation influence coefficient of the background object on the target object is higher. There is a negative correlation.

[0104] Radiation influence coefficient The larger it is, the more obvious the impact of the background object on the target object is.

[0105] In addition, in addition to the influence of thermal radiation, the influence of motion factors on the target object also exists, which is an important feature of the present invention to achieve high-accuracy target recognition.

[0106] In the optical flow field, the optical flow different from the image offset direction vector may be caused by motion or radiation interference, so the motion factor and the radiation factor need to be considered separately. The present invention calculates the influence of the motion factor on the target object through step S600.

[0107] S600: Calculate the motion influence coefficient of the background object based on the radiation influence coefficient.

[0108] The calculation of the radiation influence coefficient obtained by analyzing the heat sources around the target object in the image is based only on the distribution of heat sources in the current frame image. However, when the target object moves toward a background object with a higher radiation intensity, the influence of the background object on the target object will change accordingly due to the radiation effect of the background object. Therefore, during the movement of the target object, the relative position between the target object and the surrounding background objects changes, which may cause errors in the assessment of the heat influence on the target object. In order to more accurately evaluate this influence, it is necessary to calculate and consider the motion influence coefficient of each background object.

[0109] In addition, for the method of using optical flow vector to determine the motion vector of the target object, since the target object will be affected by the radiation of the background object when it moves, this may cause its infrared image characteristics to change, thereby affecting the accuracy of optical flow detection. This deviation may lead to inaccurate motion vector analysis results based on optical flow vector.

[0110] Reference Figure 5 In order to obtain a more accurate target object motion influence coefficient, we first need to determine the object's motion direction and speed by analyzing the feature points in adjacent frame images. Therefore, the present invention proposes a preferred solution, including steps S601-S603:

[0111] S601, comparing the displacement information between the first feature point and the second feature point, and calculating the moving direction vector ;

[0112] The specific calculation method can adopt the existing technical solution. This embodiment exemplifies a calculation step of comparing the displacement information between the first feature point and the second feature point, including steps S6011-S6017:

[0113] S6011, calculating the optical flow vector of the first feature point and the optical flow vector of the second feature point;

[0114] S6012, obtaining a third optical flow vector corresponding to the first feature point; obtaining a fourth optical flow vector corresponding to the first feature point;

[0115] S6013, converting the third optical flow vector into a third optical flow amplitude and a third optical flow angle;

[0116] S6014, converting the fourth optical flow vector into a fourth optical flow amplitude and a fourth optical flow angle;

[0117] S6015, comparing the second amplitude difference value between the third optical flow amplitude and the fourth optical flow amplitude;

[0118] S6016, comparing a second angle difference value of the third optical flow angle and a second angle difference value of the fourth optical flow angle;

[0119] S6017, based on the second amplitude difference value and the second angle difference value, convert to obtain a moving direction vector That is, the direction of the moving direction vector is obtained according to the second angle difference value, and the modulus of the moving direction vector is obtained according to the second amplitude difference value.

[0120] Moving direction vector The ⌥ itself indicates the direction of movement.

[0121] S602, according to the moving direction vector Determine the motion influence domain of the target object, where the range of the motion influence domain of the target object is a range determined by taking the mass center of the target object in the first infrared image as the center of the circle and the modulus length of the moving direction vector as the radius;

[0122] S603, determining whether there is a background object in the motion influence domain, and if so, calculating the motion influence coefficient of the background object.

[0123] Specifically, the motion influence coefficient is calculated by determining the motion influence coefficient through the motion angle between the background object and the target object and the difference in average grayscale values ​​between the background object and the target object, wherein the smaller the motion angle between the background object and the target object is, the larger the motion influence coefficient is; the larger the difference in average grayscale values ​​between the background object and the target object is, the larger the motion influence coefficient is.

[0124] To calculate the motion influence coefficient, the following steps can be used:

[0125] The motion influence coefficient is calculated using the following formula:

[0126]

[0127] in:

[0128] It is The motion influence coefficient of each target object;

[0129] is a hyperbolic function;

[0130] It is any background object within the affected range of the target object;

[0131] is the number of affected background objects within the affected range of the target object;

[0132] It is The average gray value of the target object;

[0133] It is The target object is within the affected area. The average gray value of background objects;

[0134] It is The target object and its affected area The angle between the background objects.

[0135] For the background objects around the target object, since the movement of the target object will be affected by the relative movement of the background objects and the target object, it is necessary to calculate the difference between the average gray value of the current target object and the average gray value of the background objects within its affected range, Δ , the Δ The larger the value, the higher the gray value of the background objects within the affected range of the target object is compared to the current target object, which means that the target object is more strongly affected by the surrounding background objects. Positive correlation; the motion angle between the background object and the target object within the affected range of the target object ,Should The smaller the value, the closer the moving direction of the current target object is to the position of the moving object of the background object. Therefore, it indicates that the current target object is approaching the background object, which indicates that its influence on the target object is greater. There is a negative correlation.

[0136] S700: Calculate the environmental impact coefficient of the target object according to the radiation impact coefficient and the motion impact coefficient.

[0137] According to the above operation, the thermal radiation influence coefficient and the motion influence coefficient of different background objects on the moving object are obtained, and the environmental influence coefficient of each target object is calculated.

[0138] The environmental impact coefficient is calculated using the following formula:

[0139]

[0140] in:

[0141] Representative The environmental impact coefficient of each target object;

[0142] Representative target object;

[0143] The greater the radiation impact coefficient, the greater the environmental impact coefficient; the greater the motion impact coefficient, the greater the environmental impact coefficient.

[0144] The larger the environmental impact coefficient is, the more likely it is that the target object will have its image features weakened by environmental heat sources during its movement, so it is necessary to perform adaptive contrast enhancement on the object in subsequent frames.

[0145] For the environmental impact coefficient of the target object in the current frame, the contrast of the target object in the next frame is enhanced, specifically:

[0146] S800, performing contrast enhancement processing on the third infrared image of the next frame according to the environmental impact coefficient to obtain a third enhanced image, including the steps of:

[0147] Get all the gradient value sets for each target object;

[0148] According to the environmental impact coefficient, the gradient value set is mapped to a new intensity value in an incremental manner to achieve contrast enhancement processing of the third infrared image.

[0149] Reference Figure 6 In a preferred embodiment, the third infrared image of the next frame is subjected to contrast enhancement, including steps S801-S803:

[0150] S801, obtaining a set of all gradient values ​​of the target object in the current frame;

[0151] S802, obtaining a contrast enhancement processing value according to the environmental impact coefficient;

[0152] The contrast enhancement processing value is calculated using the following formula: :

[0153]

[0154] S803, the gradient value set {G} and the contrast enhancement processing value A multiplication operation is performed to realize contrast enhancement processing on the third infrared image.

[0155] In the present invention, due to the contrast enhancement processing value The value of must be greater than 1, so the gradient value set {G} is mapped to a higher new intensity value set {G }, thereby achieving contrast enhancement processing of the third infrared image.

[0156] Through contrast enhancement processing, the target object can be guaranteed not to be affected by the radiation of surrounding heat sources, thereby avoiding problems such as feature blur and target tracking interruption caused by background objects.

[0157] S900: Search for a target object based on the third enhanced image.

[0158] By cyclically using the method of the present invention, the effects of continuous recognition, positioning and tracking of a target object can be achieved based on a plurality of continuous infrared images.

[0159] It should be noted that the specific method for performing target search is not the focus of the present invention, and the relevant target search method can refer to the prior art. Commonly used target search methods include methods based on deep learning, traditional image processing and machine learning methods, etc., such as object detectors based on convolutional neural networks, including single-stage YOLO object detection algorithms, two-stage Faster R-CNN algorithms, etc., as well as Haar cascade classifiers, HOG features and support vector machines, etc.

[0160] For the acquired enhanced image, the target object is searched by combining the texture information in the infrared image with the characteristics of the target object.

[0161] refer to Figure 7 The present invention also provides a target search system based on infrared images, comprising:

[0162] The feature extraction module S1001 is used to extract a first feature point from a first infrared image acquired, and to extract a second feature point from a second infrared image of a next frame of the first infrared image acquired;

[0163] A motion judgment module S1002 is used to judge whether the second feature point meets the image shift condition based on the first feature point, and if not, the second feature point is judged to be a stationary feature point; if so, the second feature point is judged to be a motion feature point;

[0164] The object recognition module S1003 is used to recognize background objects based on static feature points and to recognize target objects based on moving feature points;

[0165] The first calculation module S1004 is used to calculate the radiation influence coefficient of the background object;

[0166] The second calculation module S1005 is used to calculate the motion influence coefficient of the background object based on the radiation influence coefficient;

[0167] A third calculation module S1006, used to calculate the environmental impact coefficient of the target object based on the radiation impact coefficient and the motion impact coefficient;

[0168] The image enhancement processing module S1007 is used to perform contrast enhancement processing on the third infrared image of the next frame based on the environmental impact coefficient to obtain a third enhanced image;

[0169] The target search module S1008 is used to search for a target object based on the third enhanced image.

[0170] In a preferred embodiment, the motion determination module S1002 is further used for:

[0171] Calculate the optical flow vector of the first feature point and the optical flow vector of the second feature point;

[0172] Obtain the optical flow vectors with the largest number of the same in the first infrared image to obtain a first optical flow vector; obtain the optical flow vectors with the largest number of the same in the second infrared image to obtain a second optical flow vector;

[0173] The difference value between the second optical flow vector and the first optical flow vector is compared to determine the offset vector of the second infrared image.

[0174] It is determined whether the optical flow vector corresponding to the second feature point is the same as the offset vector. If they are the same, the image offset condition is not satisfied; if they are not the same, the image offset condition is satisfied.

[0175] In a preferred embodiment, the first calculation module S1004 is also used to: determine the radiation influence domain of the target object based on the target object, the range of the radiation influence domain of the target object is the minimum circumscribed circle of the target object; determine whether there is a background object in the radiation influence domain, and if so, determine the radiation influence coefficient through the distance between the background object and the target object and the difference in the average grayscale value between the background object and the target object.

[0176] In another preferred embodiment, the first calculation module S1004 is also used to: mark both the target object and the background object as objects to be identified; determine the influence domain of the object to be identified with the center of mass of the object to be identified as the radiation center; obtain all direction change lines passing through the radiation center within the influence domain, and determine the characteristic direction change line therefrom; obtain the average value of the grayscale change value on the characteristic direction change line, which is the average grayscale value of the object to be identified; and determine the difference in the average grayscale value between the background object and the target object based on the average grayscale value of the object to be identified.

[0177] In a preferred embodiment, the second calculation module S1005 is also used to: compare the displacement information between the first feature point and the second feature point to calculate the moving direction vector; determine the motion influence domain of the target object based on the moving direction vector, the range of the motion influence domain of the target object is a range determined by taking the center of mass of the target object in the first infrared image as the center of the circle and the modulus of the moving direction vector as the radius; determine whether there is a background object in the motion influence domain, and if so, determine the motion influence coefficient through the motion angle between the background object and the target object and the difference in the average grayscale value between the background object and the target object.

[0178] In a preferred embodiment, the image enhancement processing module S1007 is further used to: obtain all gradient value sets of each target object; and map the gradient value set to a new intensity value in an incremental manner according to the environmental impact coefficient to achieve contrast enhancement processing of the third infrared image.

[0179] It should be noted that: the system provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the mobile power performance automation test system provided in the above embodiment and the mobile power performance automation test method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0180] In a third aspect, a target search device is provided, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the device executes the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0181] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0182] It should also be noted that in the method and system of the present invention, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0183] The above embodiments are merely examples for the purpose of clear explanation and do not constitute a limitation on the protection scope of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description, and it is not necessary and impossible to list all the implementation methods here. All designs that are the same or similar to the present invention belong to the protection scope of the present invention.

Claims

1. A target search method based on infrared images, characterized in that: The method comprises: Acquire a first infrared image and extract a first feature point; Acquire a second infrared image of a frame next to the first infrared image, and extract a second feature point; Based on the first feature point, determine whether the second feature point meets the image shift condition. If not, determine that the second feature point is a stationary feature point. If so, determine that the second feature point is a moving feature point. Identify background objects based on static feature points and identify target objects based on moving feature points; Calculate the radiation influence coefficient of background objects; Calculate the motion influence coefficient of the background object based on the radiation influence coefficient; Calculate the environmental impact coefficient of the target object based on the radiation impact coefficient and the motion impact coefficient; Based on the environmental impact coefficient, contrast enhancement processing is performed on the third infrared image of the next frame to obtain a third enhanced image; searching for a target object based on the third enhanced image; Calculating the radiation impact coefficient of the target object includes the following steps: Determine the radiation influence domain of the target object according to the target object, wherein the range of the radiation influence domain of the target object is the minimum circumscribed circle of the target object; Determine whether there is a background object in the radiation influence domain. If so, determine the radiation influence coefficient by the distance between the background object and the target object and the difference in the average grayscale value between the background object and the target object. Among them, the smaller the distance between the background object and the target object, the greater the motion influence coefficient; The greater the difference in the average gray value between the background object and the target object, the greater the motion influence coefficient; Determining the difference between the average grayscale value of the background object and the target object includes the steps of: Mark both the target object and the background object as objects to be identified; Taking the centroid of the object to be identified as the radiation center, determine the influence domain of the object to be identified; Obtain all direction change lines passing through the radiation center within the influence domain; Sort all the direction change lines, and determine the direction change line with the largest gray value change as the characteristic direction change line; Get the average value of the grayscale change value on the feature direction change line, which is the average grayscale value of the object to be identified; Based on the average gray value of the object to be identified, determining the difference between the average gray value of the background object and the target object; Calculating the motion influence coefficient of the background object includes the following steps: Compare the displacement information between the first feature point and the second feature point, and calculate the moving direction vector; Determine the motion influence domain of the target object according to the moving direction vector, wherein the range of the motion influence domain of the target object is a range determined by taking the mass center of the target object in the first infrared image as the circle center and the modulus length of the moving direction vector as the radius; Determine whether there is a background object in the motion influence domain. If so, determine the motion influence coefficient by the motion angle between the background object and the target object and the difference between the average grayscale values ​​of the background object and the target object. Among them, the smaller the motion angle between the background object and the target object, the greater the motion influence coefficient; The greater the difference in the average gray value between the background object and the target object, the greater the motion influence coefficient; When calculating the environmental impact coefficient of the target object, the larger the radiation impact coefficient, the larger the environmental impact coefficient; the larger the motion impact coefficient, the larger the environmental impact coefficient; The contrast enhancement process includes the following steps: Get all the gradient value sets for each target object; According to the environmental impact coefficient, the gradient value set is mapped to a new intensity value in an incremental manner to achieve contrast enhancement processing of the third infrared image.

2. The target search method based on infrared images according to claim 1, characterized in that: The first infrared image, the second infrared image and the third infrared image are grayscale infrared images, and the grayscale infrared images are obtained by performing grayscale processing on the collected infrared images.

3. The target search method based on infrared images according to claim 1, characterized in that: Determining whether the image feature points meet the image offset condition includes the following steps: Calculate the optical flow vector of the first feature point and the optical flow vector of the second feature point; Obtain the optical flow vectors with the largest number of the same in the first infrared image to obtain a first optical flow vector; obtain the optical flow vectors with the largest number of the same in the second infrared image to obtain a second optical flow vector; Comparing the difference between the second optical flow vector and the first optical flow vector to determine the offset vector of the second infrared image; It is determined whether the optical flow vector corresponding to the second feature point is the same as the offset vector. If they are the same, the image offset condition is not satisfied; if they are not the same, the image offset condition is satisfied.

4. A target search system based on infrared images, characterized in that: include: A feature extraction module, used to extract a first feature point from the acquired first infrared image, and to extract a second feature point from a second infrared image of a next frame of the acquired first infrared image; A motion judgment module, used for judging whether the second feature point satisfies the image shift condition based on the first feature point, and if not, judging the second feature point as a stationary feature point; if so, judging the second feature point as a motion feature point; An object recognition module, used to recognize background objects based on static feature points and to recognize target objects based on moving feature points; The first calculation module is used to calculate the radiation influence coefficient of the background object; A second calculation module is used to calculate the motion influence coefficient of the background object based on the radiation influence coefficient; A third calculation module, used to calculate the environmental impact coefficient of the target object based on the radiation impact coefficient and the motion impact coefficient; An image enhancement processing module, used for performing contrast enhancement processing on the third infrared image of the next frame based on the environmental impact coefficient to obtain a third enhanced image; A target search module, used for searching a target object based on the third enhanced image; Calculating the radiation impact coefficient of the target object includes the following steps: Determine the radiation influence domain of the target object according to the target object, wherein the range of the radiation influence domain of the target object is the minimum circumscribed circle of the target object; Determine whether there is a background object in the radiation influence domain. If so, determine the radiation influence coefficient by the distance between the background object and the target object and the difference in the average grayscale value between the background object and the target object. Among them, the smaller the distance between the background object and the target object, the greater the motion influence coefficient; The greater the difference in the average gray value between the background object and the target object, the greater the motion influence coefficient; Determining the difference between the average grayscale value of the background object and the target object includes the steps of: Mark both the target object and the background object as objects to be identified; Taking the centroid of the object to be identified as the radiation center, determine the influence domain of the object to be identified; Obtain all direction change lines passing through the radiation center within the influence domain; Sort all the direction change lines, and determine the direction change line with the largest gray value change as the characteristic direction change line; Get the average value of the grayscale change value on the feature direction change line, which is the average grayscale value of the object to be identified; Based on the average gray value of the object to be identified, determining the difference between the average gray value of the background object and the target object; Calculating the motion influence coefficient of the background object includes the following steps: Compare the displacement information between the first feature point and the second feature point, and calculate the moving direction vector; Determine the motion influence domain of the target object according to the moving direction vector, wherein the range of the motion influence domain of the target object is a range determined by taking the mass center of the target object in the first infrared image as the circle center and the modulus length of the moving direction vector as the radius; Determine whether there is a background object in the motion influence domain. If so, determine the motion influence coefficient by the motion angle between the background object and the target object and the difference between the average grayscale values ​​of the background object and the target object. Among them, the smaller the motion angle between the background object and the target object, the greater the motion influence coefficient; The greater the difference in the average gray value between the background object and the target object, the greater the motion influence coefficient; When calculating the environmental impact coefficient of the target object, the larger the radiation impact coefficient, the larger the environmental impact coefficient; the larger the motion impact coefficient, the larger the environmental impact coefficient; The contrast enhancement process includes the following steps: Get all the gradient value sets for each target object; According to the environmental impact coefficient, the gradient value set is mapped to a new intensity value in an incremental manner to achieve contrast enhancement processing of the third infrared image.

5. A target search device based on infrared images, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 3 are implemented.

Images