Light ray tracing method for tiny target in complex scene

Abstract

The invention discloses a light ray tracing method for tiny targets in a complex scene, relates to the field of image analysis, and is used for ensuring that all tiny targets make contributions to infrared radiation characteristics of pixels projected on a focal plane of the tiny targets. According to the method, all tiny targets are initialized from all targets of a current scene; sub-pixel-level division is carried out on all unshielded tiny targets based on focal plane pixels where the tiny targets are located; determining the tracking light of the focal plane pixel based on the reference point of each sub-pixel region in the focal plane pixel; and calculating the infrared radiation brightness of each focal plane pixel based on the tracking light of the focal plane pixel. According to the method, it is ensured that tracking light corresponding to part of sub-pixels can certainly track a tiny target, and then the infrared radiation contribution of the tiny target can be brought into calculation.

Description

Technical Field

[0001] The present invention relates to the field of image analysis, and particularly to a ray tracing method for tiny targets in complex scenes. Background Art

[0002] As Figure 1 shown is a schematic diagram of the pixels of the detector focal plane array. In a conventional ray tracing-based simulation algorithm, when calculating the infrared radiation characteristics of each pixel, the pixel is subsampled to obtain a number of representative rays (i.e., traced rays), as shown by the "×" in Figure 1 . Then, the infrared radiation characteristics calculated based on these rays are accumulated to obtain the infrared radiation luminance of the pixel.

[0003] For example, in the integrated modeling method for the infrared radiation characteristics of complex targets proposed in Chinese Patent Publication No. CN105243289A, the ray tracing process involves sampling the detector focal plane pixels to generate a ray bundle, calculating the intersection of each ray with the complex targets in the scene; calculating the radiation reflection value for each intersection, sampling and reconstructing the radiation luminance obtained from all the rays of the pixel to obtain the radiation luminance in the spectral form at the pixel of the detector; and finally integrating over the required infrared spectral band to obtain the target infrared radiation characteristic value received by the detector.

[0004] The infrared simulation technology based on the combination of the inverse Monte Carlo method and the ray tracing algorithm can accurately simulate the calculation of infrared radiation characteristics in complex scenes, avoid the unsolvable large-scale linear equations, and can flexibly handle more types of scene targets in addition to typical walls. However, in the traditional ray tracing algorithm, if there are tiny targets in terms of geometric scale in the scene, due to the limitation of the number of rays selected by the ray tracing algorithm and the influence of randomness, as well as the too small spatial occupancy ratio of the tiny targets, there is a high probability that the tiny targets will not be traced by any ray, resulting in the inability to image the tiny targets and causing the problem of computational "defocus". Referring to Figure 1 , when there are tiny targets within the detector's field of view, the triangle represents the tiny target, and its projected area is less than one pixel. If the subsampled rays fail to hit the tiny target, the tiny target cannot contribute to the infrared radiation luminance of the pixel. Summary of the Invention

[0005] The object of the present invention is to provide a ray tracing method for tiny targets in complex scenes to ensure that all tiny targets contribute to the infrared radiation characteristics of the pixels projected on their focal planes, aiming at all or part of the above problems.

[0006] The technical solution adopted by the present invention is as follows:

[0007] A ray tracing method for tiny targets in complex scenarios, comprising:

[0008] Initialize all tiny targets from all targets in the current scene;

[0009] Perform sub-pixel level division on all unoccluded tiny targets respectively based on the pixels in the focal plane where they are located;

[0010] Determine the tracking rays of each pixel in the focal plane based on the reference points of each sub-pixel region in the focal plane pixel, where the tracking rays include the tracking rays for tiny targets and the tracking rays for non-tiny targets;

[0011] Calculate the infrared radiation luminance of each pixel in the focal plane respectively based on the tracking rays of each pixel in the focal plane.

[0012] Furthermore, the step of initializing all tiny targets from all targets in the current scene includes:

[0013] Take the targets with pixel ranges not exceeding a single pixel in the current scene as tiny targets.

[0014] Furthermore, the step of taking the targets with pixel ranges not exceeding a single pixel in the current scene as tiny targets includes:

[0015] Take the targets with angles subtended at the detector lower than the angle subtended by a single pixel in the current scene as tiny targets.

[0016] Furthermore, the step of performing sub-pixel level division on all unoccluded tiny targets respectively based on the pixels in the focal plane where they are located includes:

[0017] For each unoccluded tiny target, divide the region in each pixel of the focal plane into sub-pixels respectively based on the number of sub-pixels set for the pixel in the focal plane.

[0018] Furthermore, each pixel in the focal plane sets the number of sub-pixels according to the area size of the tiny targets in the pixel in the focal plane.

[0019] Furthermore, on the basis of initializing the bounding box for the unoccluded tiny targets, project them onto the focal plane, and use the minimum bounding rectangle in the focal plane as the region of the unoccluded tiny target in the focal plane.

[0020] Furthermore, the method for determining the tracking rays for tiny targets includes:

[0021] Use the line connecting the focus of the detector to the reference point of each sub-pixel region as the tracking ray for the tiny target.

[0022] Furthermore, the number of tracking rays for each pixel in the focal plane is the same.

[0023] Further, the method for determining the tracking rays for non - minute targets includes:

[0024] Randomly sample a corresponding number of tracking rays in the non - minute target area of the focal - plane pixels as the tracking rays for non - minute targets.

[0025] Further, calculating the infrared radiation luminance of each focal - plane pixel based on the tracking rays of each focal - plane pixel includes:

[0026] For each focal - plane pixel, there is:

[0027] Calculate the infrared radiation luminance of the minute - target area based on the tracking rays for minute targets, and calculate the infrared radiation luminance of the non - minute - target area based on the tracking rays for non - minute targets;

[0028] Perform weighted fusion on the infrared radiation luminance of the minute - target area and the infrared radiation luminance of the non - minute - target area to obtain the infrared radiation luminance of this focal - plane pixel.

[0029] In summary, due to the adoption of the above - mentioned technical solution, the beneficial effects of the present invention are as follows:

[0030] Compared with the sub - pixel sampling scheme in the prior art, the present invention can adaptively adjust the sub - pixel division scheme of the corresponding focal - plane pixels for each specific minute target, so as to ensure that the tracking rays corresponding to some sub - pixels can surely track the minute target, and further can incorporate the infrared radiation contribution of the minute target into the calculation. Although the present invention separately designs the step of determining the tracking rays for minute targets, the calculation efficiency is reduced compared with the prior art, but the present invention only performs sub - pixel adjustment on the minute - target area, and the proportion of pixels containing minute targets in all pixels is usually extremely low. Therefore, the reduction in the calculation efficiency of this solution is negligible, while the improvement in the calculation performance of the infrared radiation characteristics of the minute targets concerned by users is very significant. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The present invention will be described by way of examples with reference to the accompanying drawings, where:

[0032] Figure 1 is a schematic diagram of ray tracing of the detector focal - plane pixels in the prior art.

[0033] Figure 2 is a flowchart of the ray - tracing method for minute targets in complex scenes in one embodiment.

[0034] Figure 3 is a flowchart of the ray - tracing method for minute targets in complex scenes in another embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0035] All features disclosed in this specification, or steps in all methods or processes disclosed, except for mutually exclusive features and / or steps, can be combined in any way.

[0036] Any feature disclosed in this specification (including any additional claims, abstract) can be replaced by other equivalent or alternative features with a similar purpose, unless specifically stated. That is, unless specifically stated, each feature is only an example of a series of equivalent or similar features.

[0037] Embodiment 1

[0038] The ray tracing method for tiny targets in complex scenarios in this embodiment, as Figure 2 shown, includes the following processes:

[0039] S1. Initialize all tiny targets from all targets in the current scene.

[0040] The essence of initializing tiny targets is to screen out all visible tiny targets from all targets in the current scene. It includes:

[0041] Initialize all targets in the current scene;

[0042] Traverse all targets and determine whether the currently traversed target is a visible tiny target.

[0043] In some embodiments, initializing all tiny targets from all targets in the current scene includes: regarding targets with a pixel range not exceeding a single pixel in the current scene as tiny targets.

[0044] Regarding the pixel range of a target, in some embodiments, it is judged by its angular extent relative to the detector. Specifically, there is:

[0045] Regard targets with an angular extent relative to the detector in the current scene lower than the angular extent corresponding to a single pixel of the detector as tiny targets.

[0046] Since targets are usually irregularly shaped, in some embodiments, this process includes:

[0047] 1a) Initialize the bounding boxes of all targets in the scene. The so-called bounding box is the smallest cuboid that just encloses the target in a three-dimensional scene. The three side lengths of this bounding box are a, b, and c, where a ≥ b ≥ c, and then calculate the distance l from the center point of the bounding box to the detector.

[0048] 1b) Estimate the angular extent of each target relative to the detector. If this angular extent is not greater than the horizontal / vertical field of view angle corresponding to 1 pixel of the detector, regard this target as a tiny target. The specific calculation idea is as follows:

[0049] Let the estimated value of the angular extent of the target relative to the detector be θ t , then the horizontal field of view angle corresponding to a single pixel where FOV x is the field of view angle of the detector in the horizontal direction, and n x is the number of pixels for imaging in the horizontal direction of the detector. Then θ t ≤ θ x When this is the case, then the target is a small target. The same applies to the vertical field of view angle.

[0050] In addition, in some embodiments, the process of initializing small targets further includes a screening process for non-occluded small targets. This includes:

[0051] 1c) For all the screened small targets, perform target visibility analysis to determine whether there is terrain or other target occlusion between the target and the detector. Screen all the non-occluded small targets, and then enter the calculation process of the infrared radiation brightness of small targets. Otherwise (i.e., the occluded small targets), calculate the infrared radiation brightness normally.

[0052] S2. Sub-pixel level division is performed for all unoccluded small targets respectively based on the pixels in the focal plane where they are located.

[0053] The sub-pixel level division is performed on the focal plane. Therefore, it is necessary to project the three-dimensional (unoccluded) small target onto the focal plane.

[0054] 2a) Determine the projection area of the small target on the focal plane.

[0055] As mentioned in the previous embodiments, small targets are marked in the three-dimensional scene by a cuboid bounding box. Therefore, for this embodiment, when the small target is projected onto the focal plane, the bounding box is also projected together. And when the bounding box (including projecting the entire bounding box onto the focal plane, or projecting at most three faces of the bounding box facing the detector onto the focal plane) is projected onto the focal plane, generally speaking, the projection of the bounding box will form a hexagon. In some embodiments, find the smallest bounding rectangle in the focal plane that can completely cover this hexagon, and then this rectangle is the area representation in the focal plane without considering the specific shape and attitude of the small target.

[0056] In some embodiments, for each unoccluded small target, the area in each focal plane pixel is respectively sub-pixel level divided based on the number of sub-pixels set for this focal plane pixel.

[0057] Based on this embodiment, first, division is performed for each small target in different focal plane pixels. In some embodiments, the division of the small target area includes:

[0058] 2b) Confirm how many focal plane pixels the rectangular frame of the tiny target belongs to. Since the pixel range of the tiny target does not exceed 1 pixel, the rectangular frame of the tiny target on the focal plane belongs to at least 1 focal plane pixel and at most 4 focal plane pixels. And the rectangular frame is divided into sub-rectangular frames according to the focal plane pixels it is in. If the rectangular frame belongs to only 1 focal plane pixel, it is not divided.

[0059] Then, each sub-rectangular frame of the tiny target (including the case where it belongs to only 1 focal plane pixel) is sub-pixel divided based on the number of sub-pixels set for the focal plane pixel it belongs to. Therefore, the number of sub-pixels is also set for these focal plane pixels.

[0060] In some embodiments, the number of sub-pixels for each focal plane pixel is set according to the area size of the tiny target in that focal plane pixel. That is, the number of sub-pixels for each focal plane pixel is set according to the area size of all the sub-rectangular frames in that focal plane pixel. Then there is:

[0061] 2c) Combine the total area of the sub-rectangular frames in each focal plane pixel, set the number of sub-pixels to be divided, and then divide all the sub-rectangular frames in each focal plane pixel to obtain the sub-pixel division of the sub-rectangular frames.

[0062] That is: the larger the total area of the sub-rectangular frame, the more sub-pixels. In some specific embodiments, the method for setting the number of sub-pixels for the focal plane pixel is: based on the total number of tracking rays set for the focal plane pixel, according to the proportion of the total area of the tiny target (i.e., the total area of all sub-rectangular frames) in that focal plane pixel, calculate the number of tracking rays for the tiny target, and this number is the number of sub-pixels.

[0063] According to the embodiments of the present application, targeted designs of tracking rays are carried out for each tiny target. Therefore, in some embodiments, there is:

[0064] 2d) For each sub-pixel region after division, determine its reference point as the tracking ray. In some embodiments, this reference point can be the center point of each sub-pixel region.

[0065] S3. Determine the tracking ray of the focal plane pixel based on the reference point of each sub-pixel region in the focal plane pixel. This tracking ray includes the tracking ray for the tiny target and the tracking ray for the non-tiny target.

[0066] In the previous embodiments, for each sub-pixel region divided in each focal plane pixel, its reference point is determined. Based on this embodiment, its tracking ray is determined according to the reference point of each sub-pixel region. Specifically, the line connecting the focus of the detector to the reference point of each sub-pixel region is used as the tracking ray for the tiny target in the corresponding focal plane pixel.

[0067] In some embodiments, step S3 includes:

[0068] 3a) Statistically calculate the total area of the sub-rectangular frames of the small targets on each focal plane pixel. Taking any focal plane pixel as an example, denote the area of all sub-rectangular frames thereon as S t , and in addition, the area of each focal plane pixel on the focal plane is the same, all denoted as S 0 , then there is 0 ≤ S t ≤ S 0 .

[0069] 3b) Determine the specific tracking rays that need to be ray-traced for the small target areas on each focal plane pixel. The specific method includes: combining the aforementioned sub-pixel level division of each sub-rectangular frame, connecting the focus of the detector and the reference points of each sub-pixel area as the tracking rays for each sub-pixel area. Denote that there are p such tracking rays on each focal plane pixel. Obviously, the order of steps 3b) and 3a) has no precedence in execution.

[0070] 3c) Determine the specific tracking rays that need to be ray-traced for the non-small target areas on each focal plane pixel.

[0071] Assume that the number of sampled tracking rays set for each focal plane pixel is n, that is, the number of tracking rays for each focal plane pixel is the same. According to the method for calculating the number of sub-pixels in the previous embodiments, there is:

[0072]

[0073] And for the method of tracking rays for non-small targets, it is to randomly sample the corresponding number of tracking rays in the non-small target areas of the focal plane pixels as the tracking rays for non-small targets. Assume that the number of tracking rays for non-small targets is q, then there is In addition, based on the method for calculating the number of sub-pixels in the previous embodiments, there is q = n - p. According to the calculated number of tracking rays q, randomly sample q tracking rays for non-small targets in the non-small target areas of the focal plane pixels.

[0074] In some embodiments, the method for randomly sampling q tracking rays for non-small targets includes:

[0075] Randomly sample tracking rays within the focal plane pixel area, count the tracking rays not within the S t range (i.e., the small target area), discard and re-sample the tracking rays within the S t range until q tracking rays are obtained.

[0076] S4. Calculate the infrared radiation luminance of each focal plane pixel based on the tracking rays of each focal plane pixel respectively.

[0077] According to the embodiments of the present application, the tiny target areas in the individual focal plane pixels are located and segmented, and the contributions of the tiny targets to the infrared radiation luminance of the focal plane pixels are considered separately according to the segmented sub-pixel areas. Therefore, in some embodiments, the contributions of the infrared radiation luminance sampled by the above-mentioned p tracking rays and the infrared radiation luminance sampled by the q tracking rays to the infrared radiation luminance of the focal plane pixel are considered respectively, and the infrared radiation luminance sampled by the p tracking rays and the infrared radiation luminance sampled by the q tracking rays are weighted and fused.

[0078] This step for each focal plane pixel includes:

[0079] Calculating the infrared radiation luminance of the tiny target area based on the (p) tracking rays for the tiny target, and calculating the infrared radiation luminance of the non-tiny target area based on the (q) tracking rays for the non-tiny target;

[0080] Weightedly fusing the infrared radiation luminance of the tiny target area and the infrared radiation luminance of the non-tiny target area to obtain the infrared radiation luminance of this focal plane pixel.

[0081] Specifically, this step may include:

[0082] 4a) For each focal plane pixel, using the ray tracing-based infrared simulation technology, calculate the infrared radiation luminance of the p tracking rays pointing to the tiny target area, denoted as L i (i = 1,..., p).

[0083] 4b) For each focal plane pixel, using the ray tracing-based infrared simulation technology, calculate the infrared radiation luminance of the q tracking rays pointing to the non-tiny target area, denoted as L i (i = p + 1,..., p + q).

[0084] 4c) Weight the infrared radiation luminance calculated in the above two sub-steps to obtain the infrared radiation luminance L of this focal plane pixel including the contribution of the tiny target, and L can be calculated by the following formula:

[0085]

[0086] Embodiment 2

[0087] The ray tracing method for tiny targets in complex scenes in this embodiment, as Figure 3 shown, includes the following processes:

[0088] S1. Initialize all tiny targets from all the targets in the current scene.

[0089] For the specific implementation of this step, reference can be made to step S1 in Embodiment 1, which will not be elaborated here.

[0090] S2. Respectively determine the projection regions of each minute target on the focal plane.

[0091] The minute targets are marked in the three-dimensional scene through a cuboid bounding box. The minute targets are projected onto the focal plane, and the bounding box is projected together. Generally speaking, when the bounding box is projected onto the focal plane, a hexagon will be formed. In some embodiments, find the smallest enclosing rectangle in the focal plane that can completely cover this hexagon, and this rectangle is the regional representation in the focal plane without considering the specific shape and posture of the minute targets.

[0092] S3. Respectively calculate the infrared radiation luminance of each pixel on the focal plane.

[0093] In some embodiments, this step includes:

[0094] 3a) Determine whether the current pixel on the focal plane contains a minute target. If so, jump to sub-step 3b); otherwise, execute sub-step 3c).

[0095] 3b) Respectively sample and calculate the infrared radiation luminance of the minute target region and the non-minute target region in the current pixel on the focal plane, and then perform weighted fusion. Among them, sub-pixel sampling is performed on the minute target region, and random sampling is performed on the non-minute target region.

[0096] In this sub-step 3b), the implementation method can refer to steps S3 - S4 in Embodiment 1 above, as well as the relevant parts in step S2.

[0097] 3c) Perform random sampling on the pixel region of the focal plane, and calculate the average value of the infrared radiation luminance sampled by each tracking ray, which is the infrared radiation luminance of this pixel on the focal plane.

[0098] The present invention is not limited to the foregoing specific implementation manners. The present invention extends to any new feature or any new combination disclosed in this specification, as well as any new method or process step or any new combination disclosed.

Claims

1. A ray tracing method for small targets in complex scenes, characterized in that: include: Initialize all tiny targets from all targets in the current scene; All unobstructed tiny targets are divided at sub-pixel level based on the pixels in the focal plane; Determining a tracking ray of the focal plane pixel based on a reference point of each sub-pixel region in the focal plane pixel, wherein the tracking ray includes a tracking ray for a tiny target and a tracking ray for a non-tiny target; The infrared radiation brightness of each focal plane pixel is calculated based on the traced rays of the focal plane pixel.

2. The ray tracing method for small targets in complex scenes as claimed in claim 1, characterized in that: Initializing all tiny targets from all targets in the current scene includes: Targets whose pixel range does not exceed a single pixel in the current scene are considered tiny targets.

3. The ray tracing method for small targets in complex scenes as claimed in claim 2, characterized in that: The targets whose pixel range does not exceed a single pixel in the current scene are regarded as tiny targets, including: An object whose opening angle relative to the detector in the current scene is less than a single pixel is regarded as a tiny object.

4. The ray tracing method for small targets in complex scenes as claimed in claim 1, characterized in that: The sub-pixel division of all unobstructed tiny targets based on the focal plane pixels thereof comprises: For each unobstructed tiny target, the area in each focal plane pixel is divided at the sub-pixel level based on the number of sub-pixels set for the focal plane pixel.

5. The ray tracing method for small targets in complex scenes as claimed in claim 4, characterized in that: The number of sub-pixels of each focal plane pixel is set according to the area size of the tiny target in the focal plane pixel.

6. The ray tracing method for small targets in complex scenes as claimed in any one of claims 1 to 5, characterized in that: The unobstructed tiny target is projected onto the focal plane based on the initialization of the bounding box, and the minimum bounding rectangular frame in the focal plane is used as the area of ​​the unobstructed tiny target in the focal plane.

7. The ray tracing method for small targets in complex scenes as claimed in claim 6, characterized in that: The method for determining the tracing light for the tiny target includes: The line connecting the focus of the detector to the reference point of each sub-pixel area is used as the tracking light for the tiny target.

8. The ray tracing method for small targets in complex scenes as claimed in claim 7, characterized in that: The same number of traced rays are used for each focal plane pixel.

9. The ray tracing method for small targets in complex scenes as claimed in claim 8, characterized in that: The method for determining the tracing ray for the non-micro target includes: A corresponding number of tracing rays are randomly sampled in the non-micro target area of ​​the focal plane pixel as tracing rays for the non-micro target.

10. The ray tracing method for small targets in complex scenes as claimed in claim 1, characterized in that: The step of calculating the infrared radiation brightness of each focal plane pixel based on the tracing light of the focal plane pixel comprises: For each focal plane pixel, we have: The infrared radiation brightness of the tiny target area is calculated based on the tracking light for the tiny target, and the infrared radiation brightness of the non-tiny target area is calculated based on the tracking light for the non-tiny target; The infrared radiation brightness of the tiny target area and the infrared radiation brightness of the non-tiny target area are weightedly fused to obtain the infrared radiation brightness of the focal plane pixel.

Images