Corrected quadtree back-projection for large target near-field imaging

Abstract

To provide a method of near-field radar image reconstruction from radar data.SOLUTION: The method includes: receiving radar data corresponding to an image; and segmenting the image into multiple sub-images. Each sub-image includes a section of the image. The method further includes, for each sub-image, generating shifted data, by multiplying the radar data by a shifting term. The shifting term accounts for a distance between a center of the image and a center of the sub-image. The center of the sub-image is corrected for a curvature of a wavefront of a radar pulse. The method further includes: generating reduced data by filtering and downsampling the shifted data, such that a total amount of the shifted data is reduced; and reconstructing the sub-image, by back-projecting the reduced data. Performing the shifting by using the shifting term reduces distortion in the reconstructed sub-image.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] SUMMARY OF THE INVENTION The embodiments described herein relate to systems and methods for image reconstruction using hierarchical decomposition. [Background technology]

[0002] Radar imaging is widely used for military and civilian applications, such as mapping and surveying areas of interest from the air and controlling maritime borders. Radar imaging techniques can be grouped into two main categories: (i) real-aperture radar imaging, which is typically achieved using an array of antennas with multiple inputs and multiple outputs (MIMO), and (ii) synthetic aperture radar imaging, which is typically achieved using a single antenna and is based on knowledge of the relative motion between objects. The latter can be divided into two techniques. When the radar and object are moving in a controlled manner, synthetic aperture radar (SAR) algorithms can extract cross-range information and create a range-cross-range 2D image from multiple 1D range profiles. When imaging moving objects, inverse SAR (ISAR) methods are applied to first estimate and then compensate for their motion. Later, image formation algorithms coherently process the scans to obtain a radar image.

[0003] Doppler-beam sharpening (DBS) is one of the most commonly used image formation algorithms for SAR / ISAR. Its advantage lies in its thorough use of fast Fourier transforms (FFTs) in both dimensions, which can significantly minimize processing load. However, DBS algorithms rely on geometric approximations that break down in various areas. First, the large coherence times and large integration angles required for high-resolution imaging result in range and Doppler shifts and, therefore, unfocused images. Second, large targets relative to their distance to the radar can result in images with distorted dimensions. Third, wavefront curvatures evident in near-field imaging are not apparent to DBS, adversely affecting image quality. Various algorithms have been proposed in the literature to address each of these challenges. However, the resulting computations required for each of these corrections are so costly that the original DBS approximation is no longer practical.

[0004] Back projection (BP) is a more accurate image formation algorithm that uses fewer approximations at the expense of substantially more processing time. Considering the history of relative motion between the target and the radar, the value of an image pixel from the BP algorithm is the contribution (i.e., the sum) from all radar range profiles. Thus, to reconstruct an image of size N × N using BP with P = 0(N) radar range profiles, the total number of back projections required is PN 2 and the calculation is 0(N 3 ) using BP, P = 0(N 2 To reconstruct a 3D volumetric image of size N × N × N using radar range profiles, the total number of backprojections required is PN 3 and the calculation is 0(N 5 ) degree.

[0005] Quadtree backprojection is an accelerated version of standard BP reconstruction using hierarchical decomposition. Under the same circumstances, the quadtree BP algorithm reduces the computational cost to 0(N 2 logN) and 0(N 3 logN), respectively.

[0006] The central idea of ​​this fast algorithm is rooted in the angular bandlimited property, or so-called bowtie approximation, of the Radon transform of an image, i.e., an image that is half the original image size can be reconstructed from half the number of projections. This theorem can be extended to the 3D Radon transform in azimuth and elevation. By dividing the original image into four (or eight, for 3D) sub-images, we reduce the computation by a factor of two (or a factor of four for 3D imaging). If this decomposition algorithm is then recursively applied to each of the sub-images, followed by backprojections using half the number of projections (or a quarter, for 3D) for the smaller sub-images, then 0(N 2 logN) (or, for 3D imaging, 0(N 3 logN) computational cost can be achieved.

[0007] However, the quadtree BP has limitations when imaging large targets in the near field, specifically:

[0008]

number

[0009] When r s is the radius of the target, r0 is the distance from the center of the target to the radar transceiver, and Δ r is the radar range resolution, and ε controls the amount of error tolerance. In a typical mmWave security scanner setup, significant distortion occurs at r sObserved when >0.2 m. Quadtree BP does not account for the wavefront curvature, which impairs data shifting and backprojection to form subimages, resulting in distortion.

[0010] Embodiments will now be described, by way of example, with reference to the accompanying drawings, in which: [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 shows a flowchart illustrating a conventional method for implementing a quadtree BP. [Figure 2] FIG. 2 shows a flowchart illustrating a method for implementing a corrected quadtree BP according to one embodiment. [Figure 3] FIG. 3 shows the location of the center of a 7.5×7.5 cm 2 sub-image at the third depth of the quadtree decomposition for an image of size 60×60 cm 2 . [Figure 4A] FIG. 4A shows the location of the center of a 7.5×7.5 cm 2 sub-image at the third depth of the quadtree decomposition for an image of size 60×60 cm 2 in the far-field at tm=0 in the range-Doppler domain. [Figure 4B] FIG. 4B shows the location of the partial image center from FIG. 4A as the target rotates 360° about its center. [Figure 5A] FIG. 5A shows the location of the center of a 7.5×7.5 cm 2 sub-image at the third depth of the quadtree decomposition for an image of size 60×60 cm 2 at a distance of 1 meter to the radar at tm=0 in the range-Doppler domain. [Figure 5B] FIG. 5B shows the location of the partial image center from FIG. 5A as the target rotates 360° about its center. [Figure 6] Figure 6 shows an image of a mesh target of size 60x60 cm2 with 9x9 point scatterers. [Figure 7A]FIG. 7A shows the 2D Fast Fourier Transform magnitude in dB of the radar data of a mesh target at a distance of 18 meters from the radar. [Figure 7B] FIG. 7B shows the 2D Fast Fourier Transform magnitude in dB of the radar data of the mesh target at a distance of 1 meter from the radar. [Figure 8A] FIG. 8A shows the reconstruction of a radar image of a mesh target from a distance of 1 meter from the radar by the conventional method of quadtree BP. [Figure 8B] FIG. 8B shows the reconstruction of a radar image of a mesh target from a distance of 1 meter from the radar by a method for implementing a corrected quadtree BP according to an embodiment. [Figure 9] FIG. 9 shows a flowchart illustrating a conventional method for implementing decimation-in-image BP. [Figure 10] FIG. 10 shows a flowchart illustrating a method for performing compensated intra-image decimation BP according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] According to a first embodiment, there is provided a method for near-field radar image reconstruction from radar data, comprising the steps of: receiving radar data corresponding to the image; segmenting the image into a plurality of sub-images, each sub-image comprising a section of the image, and for each sub-image: generating shifted data by multiplying the radar data by a shifting term, wherein the shifting term accounts for the distance between the center of the image and the center of the sub-image, wherein the sub-image center is corrected for the curvature of the radar pulse wavefront; filtering and downsampling the shifted data to generate reduced data, thereby reducing the total amount of shifted data; reconstructing the partial image by backprojecting the reduced data; Equipped with wherein performing the shift by using a shift term reduces distortion in the reconstructed sub-image. A method is provided.

[0013] The method may further comprise shifting the reduced data before reconstructing by multiplying the reduced data by an inverse shift term, wherein the inverse shift term shifts the reduced data to its original location.

[0014] The steps including segmenting the image, generating shifted data, generating reduced data, and shifting the reduced data may be performed iteratively, where at least one of the partial images at the end of one iteration becomes the image to be segmented in the next iteration.

[0015] The steps may be performed iteratively until a maximum decomposition level for all of the sub-images is reached. The maximum decomposition level for a sub-image may be determined to be reached when the sub-image comprises a single pixel or voxel of the image.

[0016] The multiple steps may be performed iteratively until a maximum decomposition level for all multiple sub-images is reached, where the maximum decomposition level for a sub-image is determined to be reached when the sub-image does not contain data representative of the target object.

[0017] The shifting of the reduced data at the end of one iteration and the segmentation of the image and generation of the shifted data for the subsequent iteration may be performed as one multiplication.

[0018] The subimage centers are corrected for wavefront curvature using the following approximation:

[0019]

number

[0020] During the ceremony,

[0021]

number

[0022] is the corrected center of the subimage, and (x s ,y s ) is the original center of the subimage, and r0 is the distance of the image center to the radar.

[0023] The image may be a two-dimensional image, and segmenting the image into a plurality of sub-images may include segmenting the image into four sub-images, where each sub-image is a quarter of the image.

[0024] The image may be a three-dimensional image, and segmenting the image into a plurality of sub-images may include segmenting the image into eight sub-images, where each sub-image is one-eighth of the image.

[0025] The data may be received by an mmWave radar system.

[0026] The target object may be of a size approximately equivalent to the distance between the target object and the mmWave radar system.

[0027] According to another embodiment, there is provided a computer system configured to perform any of the methods described above.

[0028] According to a further embodiment, there is provided a non-transitory computer-readable storage medium comprising a plurality of computer-executable instructions that, when executed by a computer, cause the computer to perform any of the methods described above.

[0029] According to another embodiment, there is provided a method for near-field radar image reconstruction from radar data, comprising the steps of: receiving radar data corresponding to the image; Segmenting the radar data into a plurality of data sections; and for each data section: backprojecting the data section to form an intermediate image; shifting the intermediate image by multiplying each pixel of the intermediate image by a shift term, where the shift term accounts for the distance between the center of the data segment and a corrected image grid, where the corrected image grid accounts for the curvature of the radar pulse wavefront; upsampling the intermediate images; aggregating the intermediate image with a plurality of intermediate images formed from a plurality of other data sections of the plurality of data sections; Equipped with A method is provided.

[0030] Intermediate images may be aggregated with intermediate images formed from adjacent sections of data.

[0031] The method further comprises shifting the intermediate image after upsampling by multiplying each pixel of the intermediate image by an inverse shift term before aggregating, wherein the inverse shift term shifts the intermediate image to its original location.

[0032] The radar data may be repetitively segmented to produce multiple smaller and smaller sections of data.

[0033] The radar data segmentation may be repeated until a maximum resolution level is reached for each data section.

[0034] The maximum level of resolution may be determined to be reached when a data section comprises a single element of data.

[0035] Multiple intermediate images may be iteratively aggregated until a single image is formed.

[0036] The corrected image grid can reveal the wavefront curvature using the following approximate formula:

[0037]

number

[0038] During the ceremony,

[0039]

number

[0040] is the corrected pixel location in the image, (x,y) is the original pixel location in the image, and r0 is the distance between the full image center and the radar sensor.

[0041] The data may be received by an mmWave radar system.

[0042] The target object may be of a size approximately equivalent to the distance between the target object and the mmWave radar system.

[0043] According to another embodiment, there is provided a computer system configured to perform any of the methods described above.

[0044] According to a further embodiment, there is provided a non-transitory computer-readable storage medium comprising a plurality of computer-executable instructions that, when executed by a computer, cause the computer to perform any of the methods described above.

[0045] 1 shows a flowchart illustrating a method 100 for performing quadtree backprojection (BP) according to a conventional approach. Quadtree BP is a method of image reconstruction that can be used to reconstruct images based on data received in a radar system. Radar systems often alternate between emitting regular pulses of electromagnetic waves and listening for reflected signals incident on the radar system from target objects.

[0046] The conventional quadtree BP process 100 begins at step S101, where radar data corresponding to an image is received from a radar system. The image may contain one or more target objects. In some embodiments, the radar data comprises signals received and demodulated at the radar system from a set of P emitted pulses. The radar data may be stored two-dimensionally along fast and slow time axes. In some embodiments, the full image may be a two-dimensional image of size N×N, and the data may comprise data received from P=0(N) pulses. In alternative embodiments, the full image may be a three-dimensional image of size N×N×N, and the data may comprise data received from P=0(N 2 ) pulses.

[0047] The hierarchical decomposition process 110 begins at step S102, where an image is segmented into a plurality of sub-images. The segmentation may be performed such that the image is segmented into a plurality of sub-images, where each sub-image comprises a section of the image, and where the combination of all sub-images results in the image. In some embodiments, the image may be segmented into a plurality of equally sized sub-images.

[0048] In embodiments where the image is a two-dimensional image, segmenting the image may consist of segmenting into four sub-images, where each sub-image comprises one-quarter of the image. The segmentation of the image into four may be performed by dividing the image at the midpoint of the axis corresponding to each dimension.

[0049] In embodiments where the image is a three-dimensional image, segmenting the image may consist of segmenting into eight sub-images, where each sub-image comprises one-eighth of the image. The segmentation of the image into eight may be performed by dividing the image at the midpoint of the axis corresponding to each dimension.

[0050] During the first iteration of the hierarchical decomposition process, the entire image may be considered the image for the hierarchical decomposition. If the decomposition is performed multiple times, there may be multiple parent images for the hierarchical decomposition formed from previous segmentation processes, each of which may be segmented into further sub-images, in further iterations of the decomposition process.

[0051] Steps S103 to S107 may be performed independently and in parallel for each subimage. In FIG. 1, S103-1 is performed for the first subimage and therefore refers to step S103, and S103-M is performed for the Mth subimage and therefore refers to step S103. Although not explicitly shown in FIG. 1, steps S103 to S107 may be performed for all of the subimages. The number of subimages present will necessarily increase with each iteration of the hierarchical decomposition.

[0052] In step S103 of the conventional method, the data may be shifted so that the center of the associated sub-image coincides with the center of its parent image after the shift. This operation may be performed separately for each sub-image. In general, the data is shifted to the center of the sub-image (x s ,y s ), the data is shifted relative to

[0053]

number

[0054] where f c is the chirp carrier frequency of the emitted pulse signal, μ is the chirp rate, t^ is the fast time value, and t m is the pulse time, c is the speed of light, and S(x s ,y s ,t m ) is t m The center of the subimage for the radar at (x s ,y s ) and the range of the center of its parent image.

[0055] In a conventional approach to quadtree BP, this range difference can be obtained through:

[0056]

number

[0057] where Ω is the radar rotation rate in SAR imaging or the target rotation rate in ISAR imaging. Because the sinusoidal components need only be calculated once per target imaging, this formula allows for rapid calculation of range differences with only two multiplications and additions per pulse.

[0058] In step S104 of the method, the shifted data may be filtered and decimated to form reduced data. As discussed above, the band-limited property of the Radon transform of an image provides the ability to reconstruct an image from half the number of projections required for an image that is twice as large in each dimension. Thus, for example, the P projections required to reconstruct an N×N object may be reduced to P / 2 projections for an N / 2×N / 2 object. This may occur by applying an angular decimation operator to the data by angularly convolving the data with a low-pass filter before downsampling it. In some embodiments, the shifted data may be downsampled by a factor of two, halving the amount of data present so that the reduced data is data obtained from half the original number of projections.

[0059] In step S105, it may be determined whether the subimage should be further segmented. The decomposition may be performed any number of times. In some embodiments, the decomposition is performed iteratively for each subimage until the maximum depth of the hierarchical decomposition is reached. In some embodiments, the maximum depth of the hierarchical decomposition may be determined to be reached when the size of the subimage is equal to one pixel (for two-dimensional subimages) or one voxel (for three-dimensional subimages). If the method determines that the maximum depth of the hierarchical decomposition has not been reached, the method may designate the current set of subimages as a new set of parent images and begin another iteration of the hierarchical decomposition for each of these parent images.

[0060] In some embodiments, the maximum depth of hierarchical decomposition for a subimage may be determined to be reached when the data corresponding to that subimage does not contain data representative of one or more target objects. In these embodiments, the subimage may be designated as a blank image and not further decomposed, while additional iterations of the hierarchical decomposition process may continue to be performed on other subimages. In this way, computational expense may be avoided in decomposing subimages that do not contain data relevant to image reconstruction. These computational savings are valuable when implementing security scanners that output 3D real-time video streams. This may also prove useful in automatic target recognition applications, for example, to allow prescreening detectors to be applied at an intermediate (or low-resolution) stage, resulting in high-resolution imaging only in areas likely to contain targets.

[0061] If it is determined in step S106 that no further segmentation will be performed, the reduced data may be backprojected to form a reconstructed sub-image. In some embodiments, hierarchical decomposition splits a two-dimensional N×N image into N 2 In these embodiments, the backprojection operation may be decomposed into N single pixel images. 2 The method may consist of performing P / N backprojections, where each backprojection consists of backprojections of P / N projections onto a single pixel. 2 The back projections are N 2 Alternatively, in some embodiments, the hierarchical decomposition may be used to decompose a three-dimensional N×N×N image into N 3 In these embodiments, the backprojection operation may be performed using N projections of P / N projections onto a single voxel. 3 The method may consist of performing backprojections.

[0062] Finally, in step S106, the set of reconstructed sub-images may be aggregated to form a reconstructed full image. In some embodiments, each individual reconstructed sub-image may consist of a single pixel or a single voxel of the reconstructed full image. In some embodiments, a maximum depth is set so that each reconstructed sub-image may consist of an area of ​​the reconstructed full image that is larger than a single pixel or a single voxel. In some embodiments, if a sub-image is designated as blank and not further decomposed, the corresponding area of ​​the full image may consist of more than a single pixel or a single voxel and may be set to zero without backprojection.

[0063] The traditional method of quadtree BP improves the processing time of image reconstruction over BP. However, the high resolution and high integration of mmWave radars have enabled them to become small and inexpensive, making them increasingly attractive for use in security scanners for imaging people. Generally, in mmWave scanner imaging, the distance from the center of the target to the radar transceiver is r ≈ 1 m, and the radius of the target is r s >0.2 m, the standard far-field approximation of a plane wavefront begins to break down. The wavefront curvature is not taken into account in conventional quadtree BP, resulting in suboptimal image quality in the reconstructed image.

[0064] FIG. 2 shows a method 200 for implementing quadtree BP in which wavefront curvature is taken into account, according to one embodiment. The corrected quadtree BP process 200 begins at step S201, where radar data corresponding to an image is received. The image may contain one or more target objects. In some embodiments, the data comprises signals received and demodulated at the radar system from a set of P emitted pulses. In some embodiments, the image may be a two-dimensional image of size N×N, and the data may comprise data received from P=0(N) pulses. In an alternative embodiment, the image may be a three-dimensional image of size N×N×N, and the data may comprise data received from P=0(N 2) pulses.

[0065] The hierarchical decomposition process 210 begins at step S202, where an image is segmented into a plurality of sub-images. The segmentation may be performed such that the image is segmented into a plurality of sub-images, where each sub-image comprises a section of the image, and where the combination of all sub-images results in the image. In some embodiments, the image may be segmented into a plurality of equally sized sub-images.

[0066] In embodiments where the image is a two-dimensional image, segmenting the image may consist of segmenting into four sub-images, where each sub-image comprises one-quarter of the image. The segmentation of the image into four may be performed by dividing the image at the midpoint of the axis corresponding to each dimension.

[0067] In embodiments where the image is a three-dimensional image, segmenting the image may consist of segmenting into eight sub-images, where each sub-image comprises one-eighth of the image. The segmentation of the image into eight may be performed by dividing the image at the midpoint of the axis corresponding to each dimension.

[0068] During the first iteration of the hierarchical decomposition process, the entire image may be considered the image for the hierarchical decomposition. If the decomposition is performed multiple times, there may be multiple parent images for the hierarchical decomposition formed from previous segmentation processes, each of which may be segmented into further sub-images, in further iterations of the decomposition process.

[0069] Steps S203 to S207 may be performed independently and in parallel for each subimage. In Figure 2, S203-1 is performed for the first subimage and therefore refers to step S203, and S203-M is performed for the Mth subimage and therefore refers to step S203. Although not explicitly shown in Figure 2, steps S203 to S207 may be performed for all of the subimages. The number of subimages present will necessarily increase with each iteration of the hierarchical decomposition.

[0070] In step S203, the corrected quadtree BP method calculates the center of the partial image (x s ,y s When calculating the range difference between (r0,0) and the center of the full image (r0,0), a correction is implemented in the shift step to account for the wavefront curvature. The data is shifted using the same formula as before, but this time the range difference is corrected to account for the wavefront curvature. Equation (2) below represents the exact formula for this range difference:

[0071]

number

[0072] where r0 is the distance of the full image center to the radar. Using equation (2) to calculate the range difference will account for the wavefront curvature. However, as discussed above, one of the advantages of equation (1) is its relative computational simplicity, which allows for faster calculations. Equation (2) adds significant computational expense to the process, thus resulting in a less computationally efficient image reconstruction method.

[0073] To facilitate the use of equation (1) while still accounting for the wavefront curvature, the correction is instead made by dividing the center of each subimage (x s ,y s ) can be introduced into the ratio

[0074]

number

[0075] If , the subimage center location is

[0076]

number

[0077] may be approximated by, where:

[0078]

number

[0079] represents the corrected partial image center. By using the corrected partial image center, the image reconstruction method can take into account the wavefront curvature, thus reducing image distortion of large targets in the near field while maintaining low computational cost. This operation assumes that the integration angle is less than 30°. Furthermore, the approximations in equations (3) and (4) depend only on the size and location of the full image, which means that they can be pre-computed prior to imaging.

[0080] At S204 of the method, the shifted data may be filtered and decimated to form reduced data. As discussed above, the band-limited property of the Radon transform of an image provides the ability to reconstruct an image from half the number of projections required for an image twice as large in each dimension. Thus, the P projections required to reconstruct an N x N object may be reduced to P / 2 projections for an N / 2 x N / 2 object. This may occur by applying an angular decimation operator to the data by angularly convolving the data with a low-pass filter before downsampling it. In some embodiments, the shifted data may be downsampled by a factor of two, halving the amount of data present so that the reduced data is data obtained from half the original number of projections.

[0081] In step S204A-1, the reduced data may be shifted back to its original location, which after decimation is

[0082]

number

[0083] This may be done by multiplying the data by:

[0084]

number

[0085] is the range difference calculated using the corrected center of the subimage.

[0086] When this step is performed, when the backprojection operation is performed, it is performed on the subimages in their original locations, rather than shifted to the center of the full image. This ensures that the effects of the approximations in equations (3) and (4) do not adversely affect the reconstruction of the full image. At close ranges, shifting the data leads to defocusing of image pixels due to the inherent projection path. Shifting the data back to its original location prevents this problem.

[0087] The data shifting in steps S203 and S204A may be combined into one multiplication instead of two multiplications as the hierarchy depth increases, when the chart flows from S204A to S205 to S202 to S203, in order to save computational cost. These modifications of the quadtree BP method can reveal the curvature of the wavefront, and thus the quality of the reconstructed image of a large target object in the near field can be improved.

[0088] In step S205, it may be determined whether the subimage should be further segmented. The decomposition may be performed any number of times. In some embodiments, the decomposition is performed iteratively for each subimage until the maximum depth of the hierarchical decomposition is reached. In some embodiments, the maximum depth of the hierarchical decomposition may be determined to be reached when the size of the subimage is equal to one pixel (for two-dimensional subimages) or one voxel (for three-dimensional subimages). If the method determines that the maximum depth of the hierarchical decomposition has not been reached, the method may designate the current set of subimages as a new set of parent images and begin another iteration of the hierarchical decomposition on these parent images.

[0089] In some embodiments, the maximum depth of hierarchical decomposition for a subimage may be determined to be reached when the data corresponding to that subimage does not contain data representative of a target object. In these embodiments, the subimage may be designated as a blank image and not further decomposed, while additional iterations of the hierarchical decomposition process may continue to be performed on other subimages. In this way, the computational expense of decomposing subimages that do not contain data relevant to image reconstruction may be avoided. These computational savings are valuable when implementing security scanners that output 3D real-time video streams. This may also prove useful in automatic target recognition applications, for example, allowing prescreening detectors to be applied at an intermediate (or low-resolution) stage, resulting in high-resolution imaging only in areas likely to contain the target.

[0090] If it is determined in step S206 that no further segmentation is to be performed, the reduced data may be backprojected to form a reconstructed sub-image. In some embodiments, hierarchical decomposition splits a two-dimensional N×N image into N 2 In these embodiments, the backprojection operation may be decomposed into N single pixel images. 2 The method may consist of performing P / N backprojections, where each backprojection consists of backprojections of P / N projections onto a single pixel. 2 The back projections are N 2 Alternatively, in some embodiments, the hierarchical decomposition may be used to decompose a three-dimensional N×N×N image into N 3 In these embodiments, the backprojection operation may be performed using N projections of P / N projections onto a single voxel. 3 The method may consist of performing backprojections.

[0091] Finally, in step S207, the set of reconstructed sub-images may be aggregated to form a reconstructed full image. In some embodiments, each individual reconstructed sub-image may consist of a single pixel or a single voxel of the reconstructed full image. In some embodiments, if a sub-image is designated as blank and not further decomposed, the corresponding reconstructed sub-image may consist of an area of ​​the reconstructed full image that is larger than a single pixel or a single voxel.

[0092] A key difference between the conventional method of quadtree BP and the corrected method disclosed herein is that the corrected method accounts for wavefront curvature throughout the algorithm. Importantly, the embodiments disclosed herein achieve this goal without any additional computational cost due to the approximations introduced by equations (3) and (4). These factors help facilitate imaging of large targets in the near field, for example, in mmWave security scanners, as well as other projection imaging applications.

[0093] Figure 3 is 60 x 60 cm in size. 2 7.5x7.5cm at the third depth of quadtree decomposition for the image 2 At the third depth of the quadtree decomposition, the center location of the subimage of 60x60cm is shown. 2 It can be seen that the image of 60x60 cm can be decomposed into 64 sub-images. If it is determined that a sub-image can be further decomposed by not being at the maximum depth of decomposition, each of the sub-images present in Figure 3 can be designated as a parent image for the next iteration of decomposition. In some embodiments, the fourth depth of the quadtree decomposition is therefore 60x60 cm. 2The image may be decomposed into 256 sub-images. In some embodiments, not all sub-images may be designated as parent images and therefore may not be further decomposed in subsequent iterations of the hierarchical decomposition. For example, if a sub-image is determined to not contain data representative of a target object, such as reflections of an emitted pulse from the target object, the sub-image may be designated as blank and not further decomposed. In this manner, computational power may be efficiently focused on decomposing sub-images that contain relevant information for image reconstruction, preventing wasted computational resources.

[0094] Figure 3 illustrates a third depth of quadtree decomposition for a two-dimensional object. While Figure 3 and the examples described below are specifically two-dimensional, it will be appreciated that all of the embodiments practiced in this disclosure are equally suitable for reconstructing three-dimensional images.

[0095] Figure 4A shows the t m Dimensions 60x60cm from Figure 3 in the far field at =0 2 7.5x7.5cm at the third depth of quadtree decomposition for the image 2 Figure 1 shows the locations of the centers of the sub-images of . Since in the far field the centers of these images were in the Cartesian domain, it can be seen that they are located at approximately the same location in the range-Doppler domain. This is due to the far-field approximation of a plane wavefront, as discussed above.

[0096] 4B shows the location of the partial image center as the target rotates about its center at (0,0). In SAR imaging, this rotation can be caused by the emitter rotating about the target object, while in ISAR imaging, this rotation can be a rotation of the object. It can be seen that as the target rotates about its center at (0,0), the partial image centers form a complete circle. Thus, the range difference for the partial image can be determined by equation (1) above.

[0097] Figure 5A shows the t mDimensions 60x60cm from Figure 3 in the near field at =0 2 7.5x7.5cm at the third depth of quadtree decomposition for the image 2 Figure 1 shows the location of the centers of the sub-images of the radar. When imaging objects with sizes comparable to their distance from the radar, it can be seen that they are not distributed in the same way in the range-Doppler domain because the sub-image centers were in the Cartesian domain, but are now distributed in a circular shape. This is due to the curvature of the wavefront, which is more pronounced in the near field.

[0098] Figure 5B shows the location of the sub-image center as the target rotates about its center at (0,0). Due to the curvature of the wavefront, the sub-image center can now be seen to follow an elliptical path about the center. This means that equation (1) no longer provides acceptable results in the near field relative to the sub-image center.

[0099] However, when the integration angle is less than 30°, the arc traced by the sub-image center about the ellipse can be approximated as an arc from a circle. With this approximation, equation (1) may still be used with the corrected sub-image center to determine the range difference, allowing for accommodation of the wavefront curvature without increasing computational cost.

[0100] Furthermore, the re-shifting step S204A shifts the data back to its original location by the same slightly approximated range difference, so that the effects of the approximation do not propagate through to the final image formation.

[0101] The need for approximations for imaging near-field targets can be seen from an inspection of Figures 6 and 7. Figure 6 shows a two-dimensional 60x60 cm 2 1 shows a mesh target in a Cartesian domain with 9×9 point scatterers that may be shown in an image of FIG.

[0102] Figures 7A and 7B show the 2D FFT energy output of radar data received from the mesh target of Figure 6 for far-field and near-field scenarios, respectively. This directly relates to the filtering of the quadtree BP, which is performed using an FFT for computational efficiency. Because it is a far-field scenario, it can be seen why the energy in the near-field scenario in Figure 7B is no longer distributed in a square shape. Instead, the energy is distributed throughout in a circular torus, highlighting the importance of the correction in step S203.

[0103] 8A and 8B show a 60×60 cm 2 scatterer with 9×9 point scatterers centered 1 meter from the radar, created using the conventional method of quadtree BP and the corrected method of quadtree BP described herein, respectively. 2 8A and 8B show reconstructed images of a mesh target of FIG. The image sharpness metric was calculated using the image gradient magnitude (i.e., the sum of all gradient norms / number of pixels), and a comparison of FIG. 8A and FIG. 8B showed an improvement in image sharpness of over 25% according to the embodiments proposed herein.

[0104] In another embodiment, the introduced corrections in (3) and (4) can also be applied to the dual of quadtree BP, a method known as intra-image decimation BP. According to this method, to reduce the computation required to reconstruct an image, compared to the original method, decimation is applied to the image rather than the radar data, and quadtree segmentation is applied to the radar data rather than the image. Thus, intra-image decimation BP does not require the complete radar data to be subsampled to form a radar image; instead, segments of the radar data can be processed to create low-resolution intermediate images that can be combined to create a final image. This feature can provide additional advantages in real-time radar imaging.

[0105] However, intra-image decimation BP can also introduce significant distortion when imaging large targets in the near field. The introduced corrections in (3) and (4) can be applied to pixel locations in the original image to overcome this problem with negligible additional computation. This embodiment is described in further detail below with reference to the accompanying figures. In some embodiments, the corrections are applied to pixel locations in the image grid during a setup phase before data is collected and processed.

[0106] 9 shows a flowchart illustrating a method 900 for performing intra-image decimation BP according to a conventional approach. Intra-image decimation BP is again another method of image reconstruction that can be used to reconstruct images based on data received in a radar system.

[0107] The conventional intra-image decimation BP process 900 begins in step S901 where radar data corresponding to an image is received from a radar system. The image may contain one or more target objects. In some embodiments, the radar data comprises signals received and demodulated at the radar system from a set of P emitted pulses. The radar data may be stored two-dimensionally along a fast time axis and a slow time axis. In some embodiments, the full image may be a two-dimensional image of size N×N, and the data may comprise data received from P=0(N) pulses. In an alternative embodiment, the full image may be a three-dimensional image of size N×N×N, and the data may comprise data received from P=0(N 2 ) pulses.

[0108] In step S902, instead of segmenting the image into multiple sub-images, the radar data is segmented into sections and the image grid is decimated accordingly. The segmentation may be performed such that the radar data is segmented into multiple sections, where combining all the sections results in the complete radar data. In some embodiments, the radar data may be segmented into equally sized sections. The segmentation of the data may be performed by dividing the data at the midpoint of the axis corresponding to each dimension. Accordingly, the image may be decimated by a factor of two in each axis corresponding to each dimension.

[0109] In step S903, for each section of radar data, it may be determined whether the section should be further segmented. In some embodiments, the radar data may be iteratively segmented. In some embodiments, the radar data may be iteratively segmented until a maximum level of decomposition is reached. In some embodiments, the maximum depth of hierarchical decomposition may be determined to be reached when the section contains only one element of data. If the method determines that the maximum depth of hierarchical decomposition has not been reached, the method may begin another iteration of hierarchical decomposition for each section of radar data and further segment them.

[0110] Steps S903 to S906 may be performed independently and in parallel for each section. In Fig. 9, S903-1 is performed for the first section and therefore refers to step S903, and S903-M is performed for the Mth section and therefore refers to step S903. Although not explicitly shown in Fig. 9, steps S903 to S906 may be performed for all of the partial images.

[0111] In step S904, each section of radar data may be backprojected. The backprojection of each section of radar data may form an intermediate image within the decimated image grid.

[0112] In step S905, the image may be shifted so that the center of the associated data segment coincides with the center of the complete data after the shift. This operation may be performed separately for each partial image. In general, to shift an intermediate image relative to the center of a data segment, the image pixel at (x,y) is

[0113]

number

[0114] may be multiplied by, where

[0115]

number

[0116] is the fast time at the center of the data segment, and t p is the pulse time at the center of the segment, and A(x,y,t p ) is the difference between the range of the pixel location (x,y) relative to the radar at tp and the range of the image center. In traditional approaches to intra-image decimation BP, the range difference for far-field targets is

[0117]

number

[0118] where Ω is the radar rotation rate in SAR imaging or the target rotation rate in ISAR imaging. Because the sinusoidal components need only be calculated once per target imaging, this formula allows for fast calculation of the range difference with only two multiplications and additions per pixel and segment. Equation (6) below represents the exact formula for the range difference:

[0119]

number

[0120] where r0 is the distance of the full image center to the radar. Using equation (6) to calculate the range difference will account for the wavefront curvature. However, one of the advantages of equation (5) is its relative computational simplicity, which allows for faster calculations. Equation (6) adds significant computational expense to the process, thus resulting in a less computationally efficient image reconstruction method.

[0121] Once shifted, the intermediate image may be upsampled by a factor of two in each direction. An upsampled intermediate image may look like a low-resolution approximation of the final image. Upsampling an intermediate image allows that image to be combined with other intermediate images in the next step to approach the complete image at full resolution. Once upsampled, the image is

[0122]

number

[0123] The intermediate images may be shifted back to their original locations by multiplying by . In step S906, the intermediate images may be combined with other images by pixel-by-pixel addition to form a higher resolution intermediate image. The intermediate images may be combined in groups of four at a time. Each group of four intermediate images may consist of four images formed from four neighboring sections of radar data. The higher resolution intermediate image may represent a new larger data segment comprising four neighboring sections of radar data. In step S907, the method may determine whether all of the intermediate images have been combined. If all intermediate images have been combined, the image reconstruction is determined to be complete. If multiple intermediate images still exist, the method may repeat steps S905 and S906 until all of the intermediate images have been upsampled and combined. The shift in the repeated step S905 is relative to the center of the new larger data segment and the higher resolution intermediate image grid.

[0124] FIG. 10 shows a flowchart illustrating a method 1000 for performing corrected intra-image decimation BP to account for wavefront curvature without additional computational complexity, according to an embodiment.

[0125] The intra-image decimation BP process 1000 begins in step S1001, where radar data corresponding to an image is received from a radar system. The image may contain one or more target objects. In some embodiments, the radar data comprises signals received and demodulated at the radar system from a set of P emitted pulses. In some embodiments, the full image may be a two-dimensional image of size N×N, and the data may comprise data received from P=0(N) pulses. In alternative embodiments, the full image may be a three-dimensional image of size N×N×N, and the data may comprise data received from P=0(N 2 ) pulses.

[0126] In step S1002, the radar data is segmented into sections, and the image grid is decimated accordingly. The segmentation may be performed such that the radar data is segmented into multiple sections, where combining all the sections results in the complete radar data. In some embodiments, the radar data may be segmented into equally sized sections. The segmentation of the data may be performed by dividing the data at the midpoint of the axis corresponding to each dimension. Accordingly, the image may be decimated by a factor of two in each axis corresponding to each dimension.

[0127] In step S1003, for each section of radar data, it may be determined whether the section should be further segmented. In some embodiments, the radar data may be segmented iteratively. In some embodiments, the radar data may be segmented iteratively until a maximum level of decomposition is reached. In some embodiments, the maximum depth of hierarchical decomposition may be determined to be reached when a section contains only one element of data.

[0128] Steps S1003 to S1006 may be performed independently and in parallel for each segment. In Figure 10, S1003-1 is performed for the first section and therefore refers to step S1003, and S1003-M is performed for the Mth section and therefore refers to step S1003. Although not explicitly shown in Figure 10, steps S1003 to S1006 may be performed for all of the subimages.

[0129] In step S1004, each section of radar data may be backprojected. The backprojection of each section of radar data may form an intermediate image within the decimated image grid.

[0130] In step S1005, each intermediate image may be shifted by a factor of two in each direction and upsampled. The upsampled intermediate images may appear as low-resolution approximations of the final image. A corrected intra-image decimation BP method, according to an embodiment, implements a correction in the shifting step to account for wavefront curvature when calculating range differences. Intermediate images are shifted using the same formula as before, but a correction may be introduced to the image pixel locations (x,y) to facilitate the use of equation (5) while still accounting for wavefront curvature. Ratio

[0131]

number

[0132] If , the pixel location is

[0133]

number

[0134] may be approximated by, where:

[0135]

number

[0136] represents the corrected pixel location of the image. By using a corrected image grid, the image reconstruction method can take into account the curvature of the wavefront, thus reducing image distortion of large targets in the near field while maintaining a low computational cost. This operation assumes that the integration angle is less than 30°. Furthermore, the approximations in equations (7) and (8) depend only on the size and location of the full image, which means that they can be pre-computed prior to imaging. When upsampled, the image is such that each image pixel is

[0137]

number

[0138] It may be shifted back to its original location again by multiplying it by .

[0139] In step S1006, the intermediate images may be combined with other intermediate images by pixel-by-pixel addition to form a higher resolution image. The intermediate images may be combined in groups of four at a time. Each group of four intermediate images may consist of four images formed from four neighboring sections of radar data. The higher resolution intermediate image may represent a new, larger data segment comprising the four neighboring sections of radar data.

[0140] In step S1007, the method may determine whether all of the intermediate images have been combined. If all of the intermediate images have been combined, the image reconstruction is determined to be complete. If multiple intermediate images still exist, the method may repeat steps S1005 and S1006 until all of the intermediate images have been upsampled and combined. The shift in the repeated step S1005 is relative to the center of the new larger data segment and higher resolution intermediate image grid.

[0141] Implementations of the subject matter and operations described herein may be realized in digital electronic circuitry, or in computer software, computer firmware, or computer hardware, including the structures disclosed herein and their structural equivalents, or in a combination of one or more of them. Implementations of the subject matter described herein may be realized using one or more computer programs, i.e., one or more modules of computer program instructions, encoded on a computer storage medium for execution by or to control the operation of a data processing apparatus. Alternatively, or additionally, the program instructions may be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal generated to encode information for transmission to a receiver apparatus suitable for execution by a data processing apparatus. The computer storage medium may be, or may be included in, a computer-readable storage device, a computer-readable storage substrate, a random-access or serial-access memory array or device, or a combination of one or more of them. Moreover, although a computer storage medium is not a propagating signal, a computer storage medium may be a source or destination of computer program instructions encoded in an artificially generated propagating signal. A computer storage medium may be, or may be contained within, one or more separate physical components or physical media (e.g., multiple CDs, disks, or other storage devices).

[0142] While several embodiments have been described, these embodiments are presented by way of example only and are not intended to limit the scope of the invention. Indeed, the novel methods, devices, and systems described herein may be embodied in a variety of forms, and further, various omissions, substitutions, and changes may be made in the form of the methods and systems described herein without departing from the spirit of the invention. The accompanying claims and their equivalents are intended to cover all such forms or modifications that fall within the scope and spirit of the invention.

Claims

1. 1. A method for near-field radar image reconstruction from radar data, comprising: receiving radar data corresponding to the image; segmenting the image into a plurality of sub-images, each sub-image comprising a section of the image, and for each sub-image: generating shifted data by multiplying the radar data by a shift term, wherein the shift term accounts for the distance between the center of the image and the center of the sub-image, wherein the center of the sub-image is corrected for the curvature of a radar pulse wavefront; filtering and downsampling the shifted data to generate reduced data, thereby reducing the total amount of the shifted data; reconstructing the subimage by backprojecting the reduced data; and Equipped with wherein performing the shift by using the shift term reduces distortion in the reconstructed sub-image. method.

2. 2. The method of claim 1, further comprising shifting the reduced data by multiplying the reduced data by an inverse shift term before reconstructing, wherein the inverse shift term shifts the reduced data to its original location.

3. 3. The method of claim 2, wherein the steps of segmenting the image, generating shifted data, generating reduced data, and shifting the reduced data are performed iteratively, and wherein at least one of the plurality of partial images at the end of one iteration becomes the image to be segmented in a next iteration.

4. 4. The method of claim 3, wherein the steps are performed iteratively until a maximum decomposition level for all of a plurality of sub-images is reached, wherein the maximum decomposition level for a sub-image is determined to be reached when the sub-image comprises a single pixel or a single voxel of the image.

5. 4. The method of claim 3, wherein the steps are performed iteratively until a maximum decomposition level for all of the plurality of sub-images is reached, and wherein the maximum decomposition level for a sub-image is determined to be reached when the sub-image does not contain data representative of the target object.

6. 4. The method of claim 3, wherein the shifting of the reduced data at the end of one iteration and the segmentation of the image and generation of shifted data for subsequent iterations are performed as one multiplication.

7. The centers of the sub-images are corrected for the curvature of the wavefront using the approximation: [Equation 1] where: [Equation 2] is the corrected center of the subimage, and (x s , y s ) is the original center of the subimage, and r 0 The method of claim 1 , wherein: is the distance of the center of the image to the radar.

8. 2. The method of claim 1, wherein the image is a two-dimensional image, and wherein segmenting the image into a plurality of sub-images comprises segmenting the image into four sub-images, wherein each sub-image is a quarter of the image.

9. 2. The method of claim 1, wherein the image is a three-dimensional image, and wherein segmenting the image into a plurality of sub-images comprises segmenting the image into eight sub-images, wherein each sub-image is one-eighth of the image.

10. The method of claim 1 , wherein the radar data is received by an mmWave radar system.

11. The method of claim 10 , wherein a target object is approximately the same size as the distance between the target object and the mmWave radar system.

12. 1. A method for near-field radar image reconstruction from radar data, comprising: receiving radar data corresponding to the image; segmenting the radar data into a plurality of data sections; and for each data section: backprojecting the data sections to form an intermediate image; shifting the intermediate image by multiplying each pixel of the intermediate image by a shift term, wherein the shift term accounts for the distance between a center of the data segment and a corrected image grid, wherein the corrected image grid accounts for the curvature of a radar pulse wavefront; upsampling the intermediate image; aggregating the intermediate image with a plurality of intermediate images formed from a plurality of other data sections of the plurality of data sections; A method comprising:

13. 13. The method of claim 12, further comprising shifting the intermediate images after upsampling by multiplying each pixel of the intermediate images by an inverse shift term before aggregating, wherein the inverse shift term shifts the intermediate images to their original locations.

14. The method of claim 12 , wherein the radar data is recursively segmented to generate a plurality of smaller and smaller data sections.

15. 15. The method of claim 14, wherein segmenting the radar data is repeated until a maximum resolution level is reached for each data section.

16. 16. The method of claim 15, wherein the maximum level of decomposition is determined to be reached when the data section comprises a single element of data.

17. The method of claim 14 , wherein the multiple intermediate images are iteratively aggregated until a single image is formed.

18. The corrected image grid accounts for the curvature of the wavefront using the approximation: [Equation 3] where: [Equation 4] is the corrected pixel location in the image, (x, y) is the original pixel location in the image, and r 0 The method of claim 12 , wherein ∑ i = ...

19. 13. The method of claim 12, wherein the radar data is received by a mmWave radar system and / or the target object is approximately the same size as the distance between the target object and the mmWave radar system.

20. 1. A non-transitory computer-readable storage medium comprising a plurality of computer-executable instructions that, when executed by a computer, cause the computer to perform, upon receiving radar data corresponding to an image, the plurality of computer-executable instructions: segmenting the image into a plurality of sub-images, each sub-image comprising a section of the image, and for each sub-image: generating shifted data by multiplying the radar data by a shift term, wherein the shift term accounts for the distance between the center of the image and the center of the sub-image, wherein the center of the sub-image is corrected for the curvature of a radar pulse wavefront; filtering and downsampling the shifted data to generate reduced data, thereby reducing the total amount of the shifted data; reconstructing the subimage by backprojecting the reduced data; and Let them do this, wherein performing the shift by using the shift term reduces distortion in the reconstructed sub-image. A non-transitory computer-readable storage medium.

Images