A ray tracing method for small objects in complex scenes
By performing sub-pixel division of the focus plane pixels and designing targeted tracing rays, the problem of defocusing calculations of small targets in the ray tracing algorithm is solved, and effective calculation of the infrared radiation characteristics of small targets is achieved.
Patent Information
- Application Number
- CN202510138519.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-08
- Publication Date
- 2025-09-30
- Estimated Expiration
- 2045-02-08
AI Technical Summary
Existing ray tracing algorithms have difficulty effectively tracking tiny targets in complex scenes, resulting in computational defocus and inability to image tiny targets.
The focus plane pixels are divided into sub-pixel levels, tracking rays are designed for small targets and non-small targets respectively, the infrared radiation brightness is calculated, and weighted fusion is performed.
It ensures that the infrared radiation characteristics of tiny targets are contributed to the pixels in the focal plane, and improves the calculation performance of the infrared radiation characteristics of tiny targets. Although the calculation efficiency is reduced, the effect is significant.
Smart Images
Figure CN120070709B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image analysis, and in particular to a ray tracing method for small targets in complex scenes. Background Art
[0002] like Figure 1 The figure shows a schematic diagram of the detector focal plane array pixels. Conventional ray tracing-based simulation algorithms subsample each pixel when calculating the infrared radiation characteristics of each pixel to obtain several representative rays (i.e., tracing rays). Figure 1 As shown by “×” in the figure, the infrared radiation characteristics calculated based on these rays are then accumulated to obtain the infrared radiation brightness of the pixel.
[0003] For example, the integrated modeling method for the infrared radiation characteristics of complex targets proposed in China's publication number CN105243289A uses a ray tracing process to sample the pixels in the detector focal plane to generate a ray beam, and calculates the intersection of each ray with the complex target in the scene; calculates the radiation reflection value for each intersection point, samples and reconstructs the radiation brightness obtained by all rays of the pixel, and obtains the radiation brightness in the form of a spectrum at the pixel of the detector; finally, integrates the required infrared spectrum segment to obtain the target infrared radiation characteristic value received by the detector.
[0004] Infrared simulation technology based on the combination of the inverse Monte Carlo method and the ray tracing algorithm can simulate the infrared radiation characteristic calculation problems in complex scenes with high precision, avoid the large-scale linear equations that are difficult to solve, and flexibly handle more types of scene targets besides typical walls. However, in traditional ray tracing algorithms, if there are small targets on a geometric scale in the scene, due to the number of light rays selected by the ray tracing algorithm and the influence of randomness, as well as the small space occupied by the small target, there is a high probability that the small target will not be traced by any ray, which will make it impossible to image the small target, resulting in a calculation "out of focus" problem. Compared to Figure 1 ,When there is a tiny target in the detector field of view, the triangle represents the tiny ,target, and its projected area is less than one pixel. If the sub-sampled light ,failure to hit the tiny target, the tiny target cannot contribute to the infrared ,radiation brightness of the pixel. Summary of the Invention
[0005] The object of the present invention is to provide a ray tracing method for small targets in complex scenes to address all or part of the above-mentioned problems, so as to ensure that all small targets contribute to the infrared radiation characteristics of the pixels projected on their focal plane.
[0006] The technical solution adopted in the present invention is as follows:
[0007] A ray tracing method for small objects in complex scenes, comprising:
[0008] Initialize all tiny targets from all targets in the current scene;
[0009] All unobstructed tiny targets are divided into sub-pixel levels based on the focal plane pixels where they are located;
[0010] Determining a tracking ray for a focal plane pixel based on a reference point of each sub-pixel region in the focal plane pixel, wherein the tracking ray includes a tracking ray for a tiny target and a tracking ray for a non-tiny target;
[0011] The infrared radiation brightness of each focal plane pixel is calculated based on the traced rays of the focal plane pixel.
[0012] Furthermore, the initialization of all tiny targets from all targets in the current scene includes:
[0013] Targets whose pixel range does not exceed a single pixel in the current scene are considered tiny targets.
[0014] Furthermore, the process of treating targets whose pixel range does not exceed a single pixel in the current scene as tiny targets includes:
[0015] Targets whose angle relative to the detector in the current scene is less than a single pixel are considered as tiny targets.
[0016] Furthermore, the sub-pixel division of all unobstructed tiny targets based on the focal plane pixels includes:
[0017] For each unobstructed tiny target, the area in each focal plane pixel is divided at the sub-pixel level based on the sub-pixel number set for the focal plane pixel.
[0018] Furthermore, the number of sub-pixels in each focal plane pixel is set according to the area of the tiny object in the focal plane pixel.
[0019] Furthermore, the unobstructed tiny target is projected onto the focal plane based on the initial bounding box, and the minimum bounding rectangle in the focal plane is used as the area of the unobstructed tiny target in the focal plane.
[0020] Furthermore, the method for determining the tracing ray for the small target includes:
[0021] The line connecting the focus of the detector to the reference point of each sub-pixel area is used as the tracking ray for the tiny target.
[0022] Furthermore, the number of traced rays for each focal plane pixel is the same.
[0023] Furthermore, the method for determining the tracing ray for the non-tiny target includes:
[0024] A corresponding number of tracing rays are randomly sampled from the non-tiny target area of the focal plane pixel as tracing rays for the non-tiny target.
[0025] Furthermore, the calculating the infrared radiation brightness of each focal plane pixel based on the tracing ray of the focal plane pixel includes:
[0026] For each focal plane pixel, we have:
[0027] Calculate the infrared radiation brightness of the tiny target area based on the tracking light for the tiny target, and calculate the infrared radiation brightness of the non-tiny target area based on the tracking light for the non-tiny target;
[0028] The infrared radiation brightness of the tiny target area and the infrared radiation brightness of the non-tiny target area are weightedly fused to obtain the infrared radiation brightness of the focal plane pixel.
[0029] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:
[0030] Compared to the sub-pixel sampling scheme in the prior art, the present invention can adaptively adjust the sub-pixel division scheme of the corresponding focal plane pixels for each specific tiny target, thereby ensuring that the tracking rays corresponding to some sub-pixels can track the tiny target, and thus the infrared radiation contribution of the tiny target can be included in the calculation. Although the present invention has designed a separate step for determining the tracking rays for tiny targets, and the computational efficiency is lower than that of the prior art, the present invention only performs sub-pixel adjustments on the tiny target area, and the proportion of pixels containing tiny targets in the total pixels is usually very low. Therefore, the reduction in computational efficiency of this scheme is negligible, but the improvement in computational performance of the infrared radiation characteristics of tiny targets that users care about is very significant. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] The present invention will now be described by way of example with reference to the accompanying drawings, in which:
[0032] Figure 1 It is a schematic diagram of ray tracing of pixels in the focal plane of the detector using the existing technology.
[0033] Figure 2 The present invention is a flowchart of an embodiment of a ray tracing method for small objects in complex scenes.
[0034] Figure 3 This is a flowchart of another embodiment of a ray tracing method for small objects in complex scenes. DETAILED DESCRIPTION
[0035] All features disclosed in this specification, or all steps in the disclosed methods or processes, except mutually exclusive features and / or steps, can be combined in any manner.
[0036] Any feature disclosed in this specification (including any appended claims and abstract), unless otherwise stated, may be replaced by other equivalent or similar features. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.
[0037] Implementation Method 1
[0038] The ray tracing method for small targets in complex scenes in this embodiment is as follows: Figure 2 As shown, it includes the following processes:
[0039] S1. Initialize all tiny targets from all targets in the current scene.
[0040] The essence of initializing small targets is to filter out all visible small targets from all targets in the current scene, including:
[0041] Initialize all targets in the current scene;
[0042] Traverse all targets and determine whether the currently traversed target is a visible tiny target.
[0043] In some embodiments, all tiny targets are initialized from all targets in the current scene, including: taking targets whose pixel range does not exceed a single pixel in the current scene as tiny targets.
[0044] In some embodiments, the pixel range of the target is determined by its angle relative to the detector. Specifically, there are:
[0045] Targets whose angle relative to the detector in the current scene is less than a single pixel are considered as tiny targets.
[0046] Since targets are often irregularly shaped, in some embodiments, the process includes:
[0047] 1a) Initialize the bounding boxes of all objects in the scene. The so-called bounding box is the smallest cuboid that just encloses the object in the 3D scene. The three sides of the bounding box are a, b, and c, where a ≥ b ≥ c. Then calculate the distance l from the center point of the bounding box to the detector.
[0048] 1b) Estimate the angle of each target relative to the detector. If the angle is no greater than the horizontal / vertical field of view corresponding to one pixel of the detector, the target is considered a small target. The specific calculation method is as follows:
[0049] Assume that the estimated angle of the target relative to the detector is θ t ,but Horizontal field of view corresponding to a single pixel FOV x is the detector’s horizontal field of view, n x is the number of pixels of the detector imaging in the horizontal direction, then θ t ≤θ x The same applies to the vertical field of view.
[0050] In addition, in some embodiments, the process of initializing small targets also includes a screening process for non-occluded small targets, including:
[0051] 1c) Perform target visibility analysis on all screened tiny targets to determine whether there is terrain or other targets blocking the target and the detector. Screen all unobstructed tiny targets and then enter the tiny target infrared radiation brightness calculation process. Otherwise (i.e., obstructed tiny targets) the infrared radiation brightness is calculated normally.
[0052] S2. Perform sub-pixel segmentation on all unobstructed tiny targets based on the focal plane pixels.
[0053] Sub-pixel segmentation is on the focal plane, so the three-dimensional (unobstructed) tiny target needs to be projected onto the focal plane.
[0054] 2a) Determination of the focal plane projection area of the micro-target.
[0055] As mentioned in the previous embodiment, the small target is marked in the three-dimensional scene by a rectangular bounding box. Therefore, for this embodiment, the small target is projected onto the focal plane together with the bounding box. When the bounding box (including projecting the entire bounding box onto the focal plane, or projecting up to three faces of the bounding box face detector onto the focal plane) is projected onto the focal plane, generally speaking, the bounding box projection will form a hexagon. In some embodiments, the smallest bounding rectangle that can completely cover the hexagon is found in the focal plane. The rectangular box is the area representation in the focal plane without considering the specific shape and posture of the small target.
[0056] In some embodiments, for each unobstructed tiny target, the area in each focal plane pixel is divided at the sub-pixel level based on the number of sub-pixels set for the focal plane pixel.
[0057] Based on this embodiment, each small target is first divided into different focal plane pixels. In some embodiments, the division of the small target area includes:
[0058] 2b) Determine how many focal plane pixels the small target's rectangular frame belongs to. Since the pixel range of a small target does not exceed 1 pixel, the small target's rectangular frame on the focal plane belongs to at least 1 focal plane pixel and at most 4 focal plane pixels. The rectangular frame is then split into sub-rectangular frames based on the focal plane pixels. If the rectangular frame only belongs to 1 focal plane pixel, no segmentation is performed.
[0059] Then, each sub-rectangular frame of the small object (including the case where it belongs to only one focal plane pixel) is divided into sub-pixel levels based on the sub-pixel number set for the focal plane pixel to which it belongs. Therefore, the sub-pixel number is also set for these focal plane pixels.
[0060] In some embodiments, the number of sub-pixels in each focal plane pixel is set based on the area of the small target in the focal plane pixel. That is, the number of sub-pixels in each focal plane pixel is set based on the area of all sub-rectangular boxes in the focal plane pixel. Then:
[0061] 2c) Based on the total area of the sub-rectangular frames in each focal plane pixel, the number of sub-pixels to be segmented is set, and then all sub-rectangular frames in each focal plane pixel are divided to obtain sub-pixel segmentation of the sub-rectangular frames.
[0062] That is, the larger the total area of the sub-rectangular frames, the greater the number of sub-pixels. In some specific embodiments, the method for setting the number of sub-pixels for a focal plane pixel is as follows: based on the total number of tracking rays set for the focal plane pixel, the number of tracking rays for the small target is calculated based on the proportion of the total area of the small target in the focal plane pixel (i.e., the total area of all sub-rectangular frames) in the focal plane pixel. This number is the sub-pixel number.
[0063] According to the embodiments of the present application, a targeted design of tracing rays is performed for each small target. Therefore, in some embodiments, there are:
[0064] 2d) For each segmented sub-pixel region, determine a reference point for tracing the ray. In some embodiments, the reference point may be the center point of each sub-pixel region.
[0065] S3. Determine a tracking ray for the focal plane pixel based on a reference point of each sub-pixel region in the focal plane pixel, where the tracking ray includes a tracking ray for a tiny target and a tracking ray for a non-tiny target.
[0066] In the previous embodiment, a reference point was determined for each sub-pixel region segmented within each focal plane pixel. Based on this embodiment, a tracking ray is determined based on the reference point of each sub-pixel region. Specifically, the line connecting the detector's focus to the reference point of each sub-pixel region serves as the tracking ray for the small target within the corresponding focal plane pixel.
[0067] In some embodiments, step S3 includes:
[0068] 3a) Count the total area of the sub-rectangular boxes of the tiny target on each focal plane pixel. Take any focal plane pixel as an example and record the area of all the sub-rectangular boxes on it as S t In addition, the area of each focal plane pixel on the focal plane is the same, which is recorded as S0, so 0≤S t ≤S0.
[0069] 3b) Determine the specific tracing rays to be traced for the small target area on each focal plane pixel. The specific method includes: combining the aforementioned sub-pixel division of each sub-rectangular frame, connecting the focus of the detector with the reference point of each sub-pixel area, and using these as the tracing rays for each sub-pixel area. Note that there are p such tracing rays for each focal plane pixel. Obviously, steps 3b) and 3a) can be performed in any order.
[0070] 3c) Determine the specific tracing rays that need to be traced for the non-micro target area on each focal plane pixel.
[0071] Assume that the number of sampled tracing rays set for each focal plane pixel is n, that is, the number of tracing rays for each focal plane pixel is the same. According to the calculation method of the number of sub-pixels in the above embodiment, we have:
[0072]
[0073] The method of tracing rays for non-micro targets is to randomly sample a corresponding number of tracing rays in the non-micro target area of the focal plane pixel as tracing rays for non-micro targets. Assuming that the number of tracing rays for non-micro targets is q, then In addition, based on the method for calculating the number of sub-pixels in the previous embodiment, q = np. According to the calculated number of tracking rays q, q tracking rays targeting non-small targets are randomly sampled in the non-small target area of the focal plane pixel.
[0074] In some embodiments, a method for randomly sampling q tracing rays for a non-tiny target includes:
[0075] Random sampling and tracing of rays within the focal plane pixel area will not be done in S t The tracing rays within the range (i.e. small target area) are counted. t Tracing rays within the range are discarded and resampled until q tracing rays are obtained.
[0076] S4. Calculate the infrared radiation brightness of each focal plane pixel based on the traced light of the focal plane pixel.
[0077] According to the embodiments of the present application, the small target area in the focal plane pixel is separately located and segmented, and the contribution of the small target to the infrared radiation brightness of the focal plane pixel is separately considered based on the segmented sub-pixel area. Therefore, in some embodiments, the contribution of the infrared radiation brightness of the above-mentioned p tracking ray samples and the infrared radiation brightness of the q tracking ray samples to the infrared radiation brightness of the focal plane pixel is considered separately, and the infrared radiation brightness of the p tracking ray samples and the infrared radiation brightness of the q tracking ray samples are weightedly fused.
[0078] This step, for each focal plane pixel, includes:
[0079] The infrared radiation brightness of the tiny target area is calculated based on the (p) tracking rays for the tiny target, and the infrared radiation brightness of the non-tiny target area is calculated based on the (q) tracking rays for the non-tiny target;
[0080] The infrared radiation brightness of the tiny target area and the infrared radiation brightness of the non-tiny target area are weightedly fused to obtain the infrared radiation brightness of the focal plane pixel.
[0081] Specifically, this step may include:
[0082] 4a) For each focal plane pixel, use infrared simulation technology based on ray tracing to calculate the infrared radiation brightness of p tracking rays pointing to the small target area, which are denoted as L i (i=1,…,p).
[0083] 4b) For each focal plane pixel, use infrared simulation technology based on ray tracing to calculate the infrared radiation brightness of q tracking rays pointing to the non-micro target area, which are denoted as L i (i=p+1,…,p+q).
[0084] 4c) The infrared radiation brightness calculated in the above two sub-steps is weighted to obtain the infrared radiation brightness L of the focal plane pixel including the contribution of the small target. L can be calculated by the following formula:
[0085]
[0086] Implementation Method 2
[0087] The ray tracing method for small targets in complex scenes in this embodiment is as follows: Figure 3 As shown, it includes the following processes:
[0088] S1. Initialize all tiny targets from all targets in the current scene.
[0089] The specific implementation of this step can refer to step S1 in the first implementation mode, which will not be described in detail here.
[0090] S2. Determine the projection area of each tiny target on the focal plane.
[0091] Small objects are marked in the 3D scene using a cuboid bounding box. The small object is projected onto the focal plane, along with the bounding box. Generally speaking, the bounding box projection forms a hexagon. In some embodiments, a minimum bounding rectangle that completely covers this hexagon is found within the focal plane. This rectangle represents the area within the focal plane, regardless of the specific shape and pose of the small object.
[0092] S3. Calculate the infrared radiation brightness of each focal plane pixel respectively.
[0093] In some embodiments, this step includes:
[0094] 3a) Determine whether the current focal plane pixel contains a small target. If so, skip to sub-step 3b), otherwise, execute sub-step 3c).
[0095] 3b) Sample the tiny target area and non-tiny target area in the current focal plane pixel, calculate the infrared radiation brightness, and then perform weighted fusion. Sub-pixel sampling is performed on the tiny target area, while random sampling is performed on the non-tiny target area.
[0096] In this sub-step 3b), the implementation method can refer to steps S3 to S4 of the above-mentioned embodiment 1, as well as the relevant part of step S2.
[0097] 3c) Randomly sample the focal plane pixel area and calculate the average infrared radiation brightness of each tracking ray sample, which is the infrared radiation brightness of the focal plane pixel.
[0098] The present invention is not limited to the aforementioned specific embodiments, but extends to any new features or any new combination disclosed in this specification, as well as any new method or process steps or any new combination disclosed.
Claims
1. A ray tracing method for small targets in complex scenes, characterized by: include: Initializing all tiny targets from all targets in the current scene, including: treating targets whose pixel range in the current scene does not exceed a single pixel as tiny targets, specifically including: treating targets whose angle relative to the detector in the current scene is less than a single pixel as tiny targets; All unobstructed tiny targets are divided into sub-pixel levels based on the focal plane pixels where they are located; Determining tracking rays for a focal plane pixel based on a reference point in each sub-pixel region of the focal plane pixel, the tracking rays including tracking rays for a tiny target and tracking rays for a non-tiny target; determining the tracking rays for the tiny target by using a line connecting the focus of the detector to the reference point in each sub-pixel region as the tracking rays for the tiny target; determining the tracking rays for the non-tiny target by randomly sampling a corresponding number of tracking rays in the non-tiny target region of the focal plane pixel as the tracking rays for the non-tiny target; The infrared radiation brightness of each focal plane pixel is calculated based on the traced rays of the focal plane pixel, including: For each focal plane pixel, we have: Calculate the infrared radiation brightness of the tiny target area based on the tracking light for the tiny target, and calculate the infrared radiation brightness of the non-tiny target area based on the tracking light for the non-tiny target; The infrared radiation brightness of the tiny target area and the infrared radiation brightness of the non-tiny target area are weightedly fused to obtain the infrared radiation brightness of the focal plane pixel.
2. The ray tracing method for small targets in complex scenes according to claim 1, characterized in that: The sub-pixel division of all unobstructed tiny targets based on the focal plane pixels thereof includes: For each unobstructed tiny target, the area in each focal plane pixel is divided at the sub-pixel level based on the sub-pixel number set for the focal plane pixel.
3. The ray tracing method for small targets in complex scenes according to claim 2, characterized in that: The number of sub-pixels in each focal plane pixel is set according to the area of the tiny object in the focal plane pixel.
4. The ray tracing method for small targets in complex scenes according to any one of claims 1 to 3, characterized in that: The unobstructed tiny target is projected onto the focal plane based on the initial bounding box, and the minimum bounding rectangle in the focal plane is used as the area of the unobstructed tiny target in the focal plane.
5. The ray tracing method for small targets in complex scenes according to claim 1, characterized in that: The same number of rays are traced for each focal plane pixel.