Imaging apparatus, imaging system, and imaging method

Abstract

This enables the imaging of objects that may exist in areas beyond the line of sight. [Solution] The system includes: a geometry information acquisition unit that acquires geometry information representing the environment of an imaging target area which is a non-line-of-sight region; a propagation characteristics calculation unit that calculates radio wave propagation characteristics in the environment of the imaging target area using the geometry information and the observation conditions of an observation device that performs observations using radio waves; an observation result acquisition unit that acquires observation results from an observation device; and an imaging processing unit that performs imaging processing to generate an image of the non-line-of-sight region using the radio wave propagation characteristics calculated by the propagation characteristics calculation unit and the observation results from the observation device.

Description

Technical Field

[0001] The disclosed technology relates to imaging technology for imaging an object using radio waves.

Background Art

[0002] In recent years, technologies such as autonomous driving, autonomous mobile robots, and advanced driver-assistance systems (ADAS) have been developed, and radar technology is used therein. In technologies such as autonomous driving, autonomous mobile robots, or advanced driver-assistance systems, it is particularly effective to capture the shape of a target as an image for the recognition and identification of the target such as an obstacle, and it is imaged using radar. However, conventional radar technology assumes an environment with free space, ground, or sea surface, and presupposes that the target, which is the object to be recognized and identified, is within the line of sight of the radar. By the way, Patent Document 1 discloses a method for specifying the position of a target that may exist out of line of sight (NLOS). Specifically, Patent Document 1 shows a method for obtaining the true position of a target by comparing the radar image when the target is placed in free space with the radar image considering road layout information such as the coordinates of buildings constituting, for example, an "L"-shaped road, for a target (object) existing out of line of sight. In the method of Patent Document 1, the shape of the target is not imaged.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventional radar technology assumes that the target object is within the radar's line of sight, and therefore has the limitation of being unable to image objects that may be located outside the line of sight. The method described in Patent Document 1 is a method for determining the true position of a target, but it cannot image the shape of the object in areas beyond the line of sight, and therefore cannot solve the above problem.

[0005] This disclosure aims to solve the above-mentioned problems and to enable imaging of objects that may exist in areas outside of line of sight. [Means for solving the problem]

[0006] One example of the configuration of the imaging apparatus disclosed herein is: A geometry information acquisition unit acquires geometry information representing the environment of the imaging target area, which is outside the line of sight. A propagation characteristics calculation unit calculates the radio wave propagation characteristics in the environment of the imaging target region using the aforementioned geometry information and the observation conditions of an observation device that performs observations using radio waves. An observation result acquisition unit that acquires observation results from the aforementioned observation device, An imaging processing unit that performs imaging processing to generate an image of the beyond-line-of-sight region using the radio wave propagation characteristics calculated by the propagation characteristics calculation unit and the observation results from the observation device, It is something that is provided. [Effects of the Invention]

[0007] According to this disclosure, the effect is that it becomes possible to image objects that may be located in areas outside the line of sight. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a diagram showing an example of the configuration of an imaging apparatus according to Embodiment 1 of this disclosure. [Figure 2] Figure 2 shows an example of the configuration of an imaging system including an imaging apparatus according to Embodiment 1 of this disclosure. [Figure 3] Figure 3 is a diagram illustrating an example of a non-line-of-sight environment and imaging of a non-line-of-sight area in this disclosure. [Figure 4] Figure 4 is a flowchart showing an example of processing in an imaging system including an imaging apparatus according to Embodiment 1 of this disclosure. [Figure 5] Figure 5 is a flowchart showing an example of the propagation characteristics calculation process in the imaging apparatus of this disclosure. [Figure 6] Figure 6 shows an example of the configuration of an imaging system including an imaging apparatus according to Embodiment 2 of this disclosure. [Figure 7] Figure 7 is a flowchart showing an example of processing in an imaging system including an imaging apparatus according to Embodiment 2 of this disclosure. [Figure 8] Figure 8 is a flowchart showing an example of the observation condition output processing in the imaging apparatus of this disclosure. [Figure 9] Figure 9 shows a first example of a simulation of processing in an imaging apparatus according to Embodiment 2 of the present disclosure. [Figure 10] Figure 10 shows a second example of a simulation of processing in an imaging apparatus according to Embodiment 2 of the present disclosure. [Figure 11] Figure 11 shows a first example of a hardware configuration for realizing the functions described herein. [Figure 12] Figure 12 shows a second example of a hardware configuration for realizing the functions described herein. [Modes for carrying out the invention]

[0009] To further illustrate this disclosure, embodiments of this disclosure will be described below with reference to the accompanying drawings.

[0010] First, we will explain the difficulty of imaging target objects that are likely to be located in non-line-of-sight (NLOS) regions, using these regions as the imaging target area. Typical non-line-of-sight environments include T-junctions and L-shaped corners, where objects beyond the corner or T-junction are beyond the radar's line of sight. Even if no radio waves directly reach the object in such environments, diffraction due to the corner and multiple reflection components generated between the wall and the object can reach the radar. We focused on the possibility of detecting and imaging objects even in non-line-of-sight environments using these multipath components such as diffracted waves and multiple reflections. However, while imaging using multipath waves is possible in non-line-of-sight environments, the propagation characteristics are complex and pose many difficulties. For example, considering SAR (Synthetic Aperture Radar), a typical radar imaging method, complex radar wave propagation occurs depending on the environment in which the radar and object exist. Conventional radar systems that assume propagation in free space face many difficulties, such as the generation of numerous virtual images due to multipath waves. Furthermore, while conventional radars assuming free space have a resolution corresponding to the sampling of observation data, such as frequency bandwidth and synthetic aperture length, in non-line-of-sight environments, the same argument becomes invalid due to the irregularly occurring multipath. Furthermore, when radar acquires images while moving, the multipath component tends to change significantly due to the movement, making it difficult to obtain stable imaging performance. In contrast, the configuration described below enables imaging of areas outside the line of sight.

[0011] Embodiment 1. Embodiment 1 describes an example of the basic configuration of the imaging apparatus and the imaging system including the imaging apparatus of this disclosure.

[0012] An example of the configuration of an imaging apparatus according to Embodiment 1 of this disclosure will be described. Figure 1 is a diagram showing an example of the configuration of an imaging apparatus according to Embodiment 1 of this disclosure. FIG. 2 is a diagram showing an example of the configuration of an imaging system including an imaging device according to Embodiment 1 of the present disclosure. FIG. 3 is a diagram for explaining an example of an out-of-sight environment and imaging of an out-of-sight area in the present disclosure. FIG. 3 shows a state where the observation device is a radar and the out-of-sight environment, which is the environment of the out-of-sight area, is a T-junction. The imaging device 100 generates an image of an imaging target area using the observation results by radio waves. ` The imaging device 100 has a function of generating an image of an out-of-sight area. The imaging device 100 performs imaging processing using the geometry information indicating the environment of the imaging target area and the observation results for each observation condition with respect to the periphery of the imaging target area, generates an image of the out-of-sight area, and outputs it to the output destination device 400. First, the imaging principle of the present disclosure will be described. The imaging device 100 has a computing function of calculating a channel matrix representing propagation characteristics based on geometry information. The calculation is performed by a linearized inverse scattering method based on the physical optics method shown below. The scattered electric field “E s ” in the signal received by the radar, the normal vector field distribution “n S (x)” of the scatterer boundary representing the shape distribution of the target, and the incident magnetic field “H inc ” on the area to be imaged by the radar transmission wave are related by the following equation (1). E s (f,x O )=-2jωμ∫ D g(f,x O ,x)n S (x)×H inc (f,x)dx. …(1) f: Frequency of incident wave ω: Angular frequency μ: Permeability g: Dyadic Green's function x O : Coordinates of observation point x: Coordinate of the region where the target (distribution of scatterers) is analyzed (hereinafter referred to as inverse analysis) using the inverse scattering method. D: Imaging region (inverse analysis region) Here, “f” is the frequency of the incident wave, “ω” is the angular frequency, “μ” is the permeability, and “g” is the diadic Green's function. O " represents the coordinates of the observation point, and "x" represents the coordinates of the region where the target (distribution of scatterers) is analyzed (hereinafter referred to as inverse analysis) by the inverse scattering method. "D" represents the region to be imaged (inverse analysis region). Discretizing the inverse analysis region for equation (1) results in the matrix form of equation (2) below. E si =Σ j G ij n j …(2) i: Label of observational data j: A label that distinguishes the components of the discretized spatial grid and normal vector in the inverse analysis domain. E s ={E si},n={n j},G={G ij} Here, “i” is the label for the observation data, distinguishing the radar's transmit and receive channels and frequencies. “j” is the label that distinguishes the components of the discretized spatial grid and normal vector in the inverse analysis domain. Now, regarding equation (2), E s ={E si},n={n j},G={G ij When written as a matrix, it becomes the following equation (3). E s =Gn …(3) This matrix G will be called the channel matrix G. It shows the response characteristics of each channel of the observed signal and each point in the imaging region. When obtaining a 2D image, the number of grid points in space is n, and the components of the normal vector are n. x (x j ), n y (y j ) and n j=(n x1 ,···,n xn ,n y1 ,···,n yn Let ) be a 2n-component vector. The observed data is also a 2n-component vector, with the label i obtained through the observation channel and frequency. The channel matrix G is 2n × 2n. When vertical polarization is used for radar transmission and reception, the specific form of the channel matrix G is given by equation (4). TIFF2026076465000002.tif43166 Here, g zz This can be expressed by the following equation (5). g zz =G((f,x o ) i ,x j ) = E z (f i ,x j ) / E0((f,x o ) i ) …(5) In equation (5), E0((f,x o ) i ) is each observation point “x oi This is the amplitude of the incident signal when the wave source is set to ". Based on these, the components of the channel matrix G can be analytically given by the Green's function in free space, or by assuming plane wave or spherical wave incidence. However, analytical treatment is not possible beyond line of sight, so the geometry is modeled based on the acquired geometric information, and the matrix components are calculated by numerical simulation (ray tracing, method of moments, FDTD method, etc.). The inverse matrix of the obtained matrix is ​​then calculated, and a vector "E" representing the observed radar signal for each channel is obtained. z By multiplying by ", we obtain a vector "n" representing the distribution of the scatterer, and by plotting this vector according to the grid label "j", we obtain an image. TIFF2026076465000003.tif25166 Thus, equation (2) becomes equation (6). E si=Σ j G ij n pj …(6) According to equation (6), the length of "j" can be the same as the number of grid points, "n", and the length of the observed data can also be as small as "n". The specific components of the matrix are determined by the following equation (7). G ij =g zzij (cosΦ p H incyj -sinΦ p H incxj ) …(7) Here “Φ p The angle is the assumed primary direction of the normal vector measured from the x-axis. This way, the size of the channel matrix G becomes n × n, reducing computational cost and allowing for lightweight operation. We obtain an image of the region thus defined.

[0013] Next, a specific example of the configuration of the imaging device 100 described above will be explained. The imaging device 100 shown in Figure 1 includes a geometry information acquisition unit 110, a propagation characteristics calculation unit 120, an observation result acquisition unit 130, and an imaging processing unit 140.

[0014] The geometry information acquisition unit 110 acquires geometry information. The geometry information acquisition unit 110 acquires geometry information representing the environment of the imaging target area, which is an area outside the line of sight. Geometry information represents the environment of the imaging target area, which is a non-line-of-sight (NOS) area, and indicates the spatial shape and position of the environment. When the imaging target area is the NOS area (Non-Line-of-Sight) shown in Figure 3, the geometry information includes, for example, the shape of corners and walls, and the relative position relative to the radar. Geometry information can be obtained using, for example, images captured by a camera, measurement results from LiDAR (Light Detection and Ranging), or 3D maps.

[0015] The propagation characteristics calculation unit 120 calculates the radio wave propagation characteristics in the environment of the imaging target area using the geometry information and the observation conditions of the observation device that performs observations using radio waves. The propagation characteristics calculation unit 120 calculates the radio wave propagation characteristics between the transmit and receive channels of the observation device and the target object that is likely to be in the line of sight area. The propagation characteristics calculation unit 120 derives and outputs a channel matrix representing the radio wave propagation characteristics.

[0016] The observation result acquisition unit 130 acquires observation results, which are the results of observations performed on the imaging target area. Specifically, the observation result acquisition unit 130 acquires the results of observations of the imaging target area by the observation device.

[0017] The imaging processing unit 140 performs imaging processing to generate an image of the out-of-line region using the radio wave propagation characteristics calculated by the propagation characteristics calculation unit 120 and the observation results from the observation device.

[0018] In addition to the above configuration, the imaging device 100 includes a control unit (not shown), a storage unit (not shown), and a communication unit (not shown). A control unit (not shown) controls the entire imaging device 100 and its individual components. For example, the control unit starts the imaging device 100 according to an external command. The control unit also controls the state of the imaging device 100 (operating state = state such as startup, shutdown, or sleep). A storage unit (not shown) stores the data used by the imaging device 100. For example, the storage unit stores the output (output data) from each component of the imaging device 100 and outputs the requested data to the requesting component. The communication unit (not shown) communicates with external devices. For example, it communicates between the imaging device 100 and peripheral devices (for example, other mobile-mounted devices if the imaging device 100 is mounted on a mobile device). For example, if the imaging device 100 and the mobile-mounted device are not connected by a wire, the communication unit (not shown) has the function of communicating between the imaging device 100 and the mobile-mounted device. The communication unit (not shown) also has the function of communicating with an external device, such as a server device. The control unit (not shown), storage unit (not shown), and communication unit (not shown) are the same in the embodiments described later.

[0019] Next, we will describe an example of a system configuration that includes an imaging device. The imaging system 1(1A) has means for acquiring environmental information (geometry) and has a computational function to calculate a channel matrix G representing propagation characteristics based on that information. The imaging system 1(1A) shown in Figure 2 comprises an imaging device 100(100A), a geometry information acquisition device 200, an observation device 300, and an output device 400.

[0020] The geometry information acquisition device 200 is a means for acquiring information (geometry information) about the environment of the imaging target area, and includes, for example, a camera, a lidar, and a 3D map. The geometry information acquisition device 200 acquires geometry information representing the environment of the imaging target area, which is an area outside the line of sight. The geometry information acquisition device 200 acquires geometry information using, for example, images captured by a camera, measurement results from LiDAR (Light Detection And Ranging), and 3D maps. Although the geometry information acquisition device 200 is shown as an independent device in Figure 2, it may be composed of multiple different devices. Alternatively, the geometry information acquisition device 200 may be configured inside the imaging device 100 (100A). In this case, the geometry information acquisition device 200 may be configured integrally with, for example, the geometry information acquisition unit 110.

[0021] The observation device 300 is a device that performs observations using radio waves. This explanation assumes that the observation device 300 is a radar device. The observation device 300 is configured to allow changes in observation conditions. The observation device 300 has a hardware configuration that allows changes to, for example, the configuration related to the transmission and reception of radio waves. The observation device 300 allows changes to the arrangement of transmitting and receiving elements, etc.

[0022] The output device 400 receives and acquires the image output by the imaging device 100 (100A). The output device 400 is, for example, a device that constitutes an autonomous driving system, an autonomous robot, and an advanced driver-assistance system (ADAS), and processes the image output by the imaging device 100 (100A). Alternatively, the output device 400 may be, for example, a display device, a touch panel display, or a terminal device.

[0023] The imaging device 100 (100A) performs imaging processing using the geometry information acquired by the geometry information acquisition device 200 and the observation results for each observation condition from the observation device 300, generates an image (including the out-of-line-of-sight area), and outputs it to the output device 400. The imaging device 100 (100A) shown in Figure 2 is configured similarly to the imaging device 100 shown in Figure 1, and includes a geometry information acquisition unit 110, a propagation characteristics calculation unit 120, an observation result acquisition unit 130, and an imaging processing unit 140.

[0024] The geometry information acquisition unit 110 shown in Figure 2 receives and acquires geometry information acquired by the geometry information acquisition device 200 from the geometry information acquisition device 200.

[0025] The imaging processing unit 140 shown in Figure 2 performs imaging processing to generate an image of the out-of-line region using the radio wave propagation characteristics calculated by the propagation characteristics calculation unit 120 and the observation results from the observation device 300, and outputs the generated image as image data to the output destination device 400.

[0026] This document describes an example of processing for an imaging system and an imaging device according to Embodiment 1 of this disclosure. Figure 4 is a flowchart showing an example of processing in an imaging system including an imaging apparatus according to Embodiment 1 of this disclosure. The processing of imaging device 100 and imaging device 100(100A) differs in that imaging device 100(100A) can explicitly state that the source of information used in processing is the geometry information acquisition device 200 and the observation device 300, and that the destination of the image data is the output destination device 400. However, since the internal processing is the same for both, an example of the processing of imaging device 100A will be explained here as a representative example. The process shown in Figure 4 includes an imaging method using an imaging device. For example, the imaging system 1 (1A) and imaging device 100 (100A) shown in Figure 2 start the process shown in Figure 4 ("Start") when a pre-set start condition is met, such as when an external command is received.

[0027] In imaging system 1(1A), the geometry information acquisition process is then performed (step ST1000 "Geometry Information Acquisition"). In the geometry information acquisition process of step ST1000, the geometry information acquisition device 200 of imaging system 1(1A) acquires geometry information representing the environment of the imaging target area, which is an area beyond line of sight, using, for example, images captured by a camera, measurement results by LiDAR (Light Detection And Ranging), a 3D map, etc. The geometry information acquisition device 200 acquires geometry information sequentially.

[0028] The imaging device 100 (100A) of the imaging system 1 (1A) then performs geometry information reception processing (step ST1110 "Geometry Information Reception"). In the geometry information reception processing of step ST1110, the geometry information acquisition unit 110 of the imaging device 100 (100A) receives and acquires geometry information representing the environment of the imaging target area, which is an out-of-line-of-sight area, from the geometry information acquisition device 200. The geometry information acquisition unit 110 outputs the geometry information to the propagation characteristics calculation unit 120.

[0029] The imaging device 100 (100A) of the imaging system 1 (1A) then performs propagation characteristics calculation processing (step ST1120 "Propagation Characteristics Calculation"). In the propagation characteristics calculation processing of step ST1120, the propagation characteristics calculation unit 120 of the imaging device 100 (100A) calculates the radio wave propagation characteristics in the environment of the imaging target area using the geometry information and the observation conditions of the observation device 300 that performs observation using radio waves. The propagation characteristics calculation unit 120 further derives a channel matrix G that shows the radio wave propagation characteristics and outputs the channel matrix G to the imaging processing unit 140.

[0030] Here, we will explain an example of the detailed processing involved in calculating propagation characteristics. Figure 5 is a flowchart showing an example of the propagation characteristics calculation process in the imaging apparatus of this disclosure. When the propagation characteristics calculation unit 120 of the imaging device 100 (100A) receives geometry information and observation conditions of the observation device 300, it starts the process shown in Figure 5 ("Start"). The propagation characteristics calculation unit 120 of the imaging device 100 (100A) then performs numerical simulation processing (step ST1121 "Numerical Simulation"). In the numerical simulation processing of step ST1121, the propagation characteristics calculation unit 120 models the geometry based on the acquired geometry information and performs numerical simulation (ray tracing, moment method, FDTD method, etc.). The propagation characteristics calculation unit 120 of the imaging device 100 (100A) then performs a channel matrix calculation process (step ST1122 "Channel Matrix Calculation"). In the channel matrix calculation process of step ST1122, the propagation characteristics calculation unit 120 calculates the components of the channel matrix G by numerical simulation processing. The propagation characteristics calculation unit 120 of the imaging device 100 (100A) then completes the process shown in Figure 5 ("complete").

[0031] Returning to the explanation of Figure 4. The observation device 300 of the imaging system 1 (1A) then performs observation processing (step ST1130 "Observation"). In the observation processing of step ST1130, when observation parameters indicating observation conditions are given, the observation device 300 performs observations based on the observation conditions indicated by the observation parameters. The observation device 300 outputs an observation signal indicating the observation result to the imaging device 100 (100A).

[0032] The imaging device 100 (100A) of the imaging system 1 (1A) then performs an observation result acquisition process (step ST1140 "Observation Result Acquisition"). In the observation result acquisition process of step ST1140, the observation result acquisition unit 130 of the imaging device 100 (100A) acquires the observation results from the observation device 300. The observation result acquisition unit 130 acquires the observation results by receiving an observation signal indicating the observation results output by the observation device 300.

[0033] The imaging device 100 (100A) of the imaging system 1 (1A) then performs imaging processing (step ST1150 "imaging processing"). In the imaging processing of step ST1150, the imaging processing unit 140 of the imaging device 100 (100A) performs imaging processing to generate an image of the out-of-line region using the radio wave propagation characteristics calculated by the propagation characteristics calculation unit 120 and the observation results from the observation device 300.

[0034] In the imaging system 1(1A), output destination processing is then performed (step ST1200 "output destination processing"). In the output destination processing of step ST1200, when the imaging processing unit 140 of the imaging device 100(100A) in the imaging system 1(1A) outputs image data, the output destination device 400 receives the image data and performs processing using the image data. Specifically, the output destination device 400 analyzes, for example, whether the image contains a target object.

[0035] The imaging device 100 (100A) of imaging system 1 (1A) then terminates the process shown in Figure 4 ("terminate").

[0036] This embodiment shows an example configuration that includes the following: (1) A geometry information acquisition unit acquires geometry information representing the environment of the imaging target area, which is outside the line of sight. A propagation characteristics calculation unit calculates the radio wave propagation characteristics in the environment of the imaging target region using the aforementioned geometry information and the observation conditions of an observation device that performs observations using radio waves. An observation result acquisition unit that acquires observation results from the aforementioned observation device, An imaging processing unit that performs imaging processing to generate an image of the beyond-line-of-sight region using the radio wave propagation characteristics calculated by the propagation characteristics calculation unit and the observation results from the observation device, An imaging device equipped with this device. By adopting this configuration, the present disclosure has the effect of providing an imaging device that enables imaging of objects that may be located in areas outside the line of sight.

[0037] This embodiment shows an example configuration that includes the following: (10) An observation device that performs observations using radio waves and is configured to allow changes in observation conditions, A geometry information acquisition device that acquires geometric information representing the environment of the imaging target area, which is outside the line of sight, A geometry information acquisition unit that receives geometry information acquired by the geometry information acquisition device, A propagation characteristics calculation unit calculates the radio wave propagation characteristics in the environment of the imaging target region using the geometry information and the observation conditions of the observation device. An observation result acquisition unit that acquires observation results from the aforementioned observation device, And, An imaging processing unit performs imaging processing to generate an image of the beyond-line-of-sight region using the radio wave propagation characteristics calculated by the propagation characteristics calculation unit and the observation results from the observation device. An imaging device having, An imaging system equipped with this system. By adopting this configuration, the present disclosure has the effect of providing an imaging system that enables imaging of objects that may be located in areas outside the line of sight.

[0038] This embodiment shows an example configuration that includes the following: (11) An imaging method for generating an image using an imaging device, A geometry information reception step that acquires geometry information representing the environment of the imaging target area which is an area outside the line of sight, A propagation characteristics calculation step that calculates the radio wave propagation characteristics in the environment of the imaging target region using the aforementioned geometry information and the observation conditions of an observation device that performs observations using radio waves, An observation result acquisition step of acquiring observation results by the aforementioned observation device, An imaging processing step is performed to generate an image of the out-of-line region using the radio wave propagation characteristics calculated in the propagation characteristics calculation step and the observation results from the observation device. Imaging methods are provided. By adopting this configuration, the present disclosure has the effect of providing an imaging method that enables imaging of objects that may be located in areas outside the line of sight.

[0039] Embodiment 2. In the above-described embodiment 1, an example of the configuration of the basic form of the present disclosure was explained. In Embodiment 2, we will describe an example configuration in which the accuracy of imaging is further improved by operating the observation device under suitable observation conditions. In Embodiment 2, for components of Embodiment 2 that are the same as those of Embodiment 1 already described, redundant explanations will be omitted as appropriate.

[0040] Next, an example of the configuration of an imaging system including an imaging apparatus according to Embodiment 2 of this disclosure will be described. The imaging system has a hardware configuration that allows for changes in the frequency to be acquired, the elements used, and the placement of the elements. The observation device receives a command that includes observation parameters, which are the observation conditions, and changes the frequency to be acquired, the elements used, and the placement of the elements. The observation parameters are, for example, values ​​that indicate the frequency to be acquired, the elements used, and the placement of the elements. The imaging system according to Embodiment 2 operates the imaging system described in Embodiment 1, which has already been described, under the observation conditions indicated by one or a set of selected observation parameters, calculates a channel matrix, repeatedly changes the observation parameters and calculates the channel matrix, repeats until a channel matrix that yields a better image is obtained, adaptively changes the hardware configuration according to the resulting observation parameters, performs the observation, and obtains an image. As a specific example, the channel matrix is ​​subjected to singular value decomposition, truncated at a certain threshold, and the inverse of the matrix is ​​taken to obtain an image of the scatterer. The singular value decomposition of the channel matrix G of the transfer matrix is ​​“G = UΣV * This is the result. Here, "U" represents the left singular vector, "V" represents the matrix constructed by arranging the right singular vectors, and "Σ" represents "Σ=diag(σ1, σ2,···, σ i ,···)” indicates a singular value σ i This is a matrix in which the elements are arranged in descending order. TIFF2026076465000004.tif37166

[0041] Figure 6 shows an example of the configuration of an imaging system including an imaging apparatus according to Embodiment 2 of this disclosure. In addition to the functions of imaging system 1 already described, imaging system 1(1B) has the function of operating the observation instrument under observation conditions suitable for imaging. The imaging system 1(1B) shown in Figure 6 comprises an imaging device 100(100B), a geometry information acquisition device 200, an observation device 300, and an output device 400. The geometry information acquisition device 200 and the output destination device 400 are configured in the same manner as the geometry information acquisition device 200 and the output destination device 400 described earlier.

[0042] The observation device 300 shown in Figure 6, in addition to the observation device 300 already described, also receives commands for observation conditions from the imaging device 100 (100B) and operates according to those conditions. The observation device 300 has a configuration that allows the frequency to be acquired, the elements used, and the placement positions of the elements to be changed within a certain range. The observation device 300 changes the frequency to be acquired, the elements used, and the placement positions of the elements in response to a command that includes observation parameters, which are the observation conditions.

[0043] The imaging device 100(100B), in addition to the imaging device 100(100A) described above, also has the function of calculating singular values ​​of the channel matrix connecting each grid point in the imaging region with the transmit / receive channel based on the transfer characteristic calculation results. Furthermore, the imaging device 100(100B) has the function of repeatedly calculating singular values ​​by changing the observation conditions of the observation device 300. The imaging device 100(100B) adaptively changes the observation conditions of the observation device 300 so that the distribution of singular values ​​of the channel matrix satisfies pre-set conditions.

[0044] The imaging device 100 (100B) shown in Figure 6 includes a geometry information acquisition unit 110, a propagation characteristics calculation unit 120, an observation result acquisition unit 130, an imaging processing unit 140, an observation condition command unit 150, and a processing command unit 170. The geometry information acquisition unit 110, propagation characteristics calculation unit 120, observation result acquisition unit 130, and imaging processing unit 140 are configured in the same manner as the geometry information acquisition unit 110, propagation characteristics calculation unit 120, observation result acquisition unit 130, and imaging processing unit 140 described above.

[0045] The observation condition command unit 150 calculates singular values ​​in the channel matrix that connects each grid point in the imaging target area to the transmit / receive channel of the observation device 300, using the calculated propagation characteristics for each observation condition, and operates the observation device 300 using observation conditions in which the distribution of singular values ​​in the channel matrix satisfies the conditions set in advance. The observation conditions include at least one of the following: the number of observation elements in the observation device, the position of the observation elements, or the observation frequency. The observation condition command unit 150 shown in Figure 6 includes an observation parameter output unit 161, a channel matrix singular value calculation unit 162, and an observation parameter optimization unit 163.

[0046] The observation parameter output unit 161 outputs observation parameters for each observation condition. The observation parameter output unit 161 outputs observation parameters for each observation condition using a plurality of pre-set observation conditions. The observation parameter output unit 161 provides the observation parameters to the observation device 300.

[0047] The channel matrix singular value calculation unit 162 calculates singular values ​​of the channel matrix. For each observation condition, the channel matrix singular value calculation unit 162 uses the calculated propagation characteristics to calculate singular values ​​of the channel matrix that connect each grid point in the imaging target area with the transmit / receive channel of the observation device 300.

[0048] The observation parameter optimization unit 163 determines observation parameters suitable for observation. The observation parameter optimization unit 163 operates the observation device 300 using observation conditions that satisfy the pre-set conditions for the distribution of singular values ​​in the channel matrix.

[0049] The processing command unit 170 commands the imaging device 100 (100B) to perform an operation. The processing command unit 170 repeatedly performs the following: acquisition of geometry information by the geometry information acquisition unit 110, calculation of radio wave propagation characteristics by the propagation characteristics calculation unit 120, and determination of observation conditions by the observation condition command unit 150.

[0050] This document describes an example of processing for an imaging system and an imaging device according to Embodiment 2 of this disclosure. Figure 7 is a flowchart showing an example of processing in an imaging system including an imaging apparatus according to Embodiment 2 of this disclosure. The process shown in Figure 7 includes an imaging method using an imaging device. For example, the imaging system 1 (1B) and imaging device 100 (100B) shown in Figure 6 start the process shown in Figure 7 ("Start") when a pre-set start condition is met, such as when an external command is received.

[0051] In imaging system 1(1B), the geometry information acquisition process is then performed (step ST2000 "Geometry Information Acquisition"). In the geometry information acquisition process of step ST2000, the geometry information acquisition device 200 of imaging system 1(1B) acquires geometry information representing the environment of the imaging target area, which is an area beyond line of sight, using, for example, images captured by a camera, measurement results by LiDAR (Light Detection And Ranging), a 3D map, etc. The geometry information acquisition device 200 acquires geometry information sequentially.

[0052] The imaging device 100 (100B) of the imaging system 1 (1B) then performs geometry information reception processing (step ST2110 "Geometry Information Reception"). In the geometry information reception processing of step ST2110, the geometry information acquisition unit 110 of the imaging device 100 (100B) receives and acquires geometry information representing the environment of the imaging target area, which is an out-of-line-of-sight area, from the geometry information acquisition device 200. The geometry information acquisition unit 110 outputs the geometry information to the propagation characteristics calculation unit 120.

[0053] The imaging device 100(100B) of the imaging system 1(1B) then performs propagation characteristics calculation processing (step ST2120 "Propagation Characteristics Calculation"). In the propagation characteristics calculation processing of step ST2120, the propagation characteristics calculation unit 120 of the imaging device 100(100B) calculates the radio wave propagation characteristics in the environment of the imaging target area using the geometry information and the observation conditions of the observation device 300 that performs observation using radio waves. The propagation characteristics calculation unit 120 further derives a channel matrix G that shows the radio wave propagation characteristics and outputs the channel matrix G to the imaging processing unit 140.

[0054] The imaging device 100 (100B) then performs observation condition output processing (step ST2130 "Observation Condition Output"). In the observation condition output processing of step ST2130, the observation condition command unit 150 of the imaging device 100 (100B) calculates singular values ​​of the channel matrix that connects each grid point of the imaging target area to the transmit / receive channel of the observation device using the calculated propagation characteristics for each observation condition, and operates the observation device 300 using observation conditions in which the distribution of singular values ​​of the channel matrix satisfies the conditions set in advance.

[0055] Here, we will explain the details of the process for determining the observation conditions in the observation condition output processing. Figure 8 is a flowchart showing an example of the process for determining observation conditions in the imaging apparatus of this disclosure. When the observation condition command unit 150 of the imaging device 100 (100B) starts the process shown in Figure 8 ("Start"), it then executes the observation parameter optimization start process (step ST2131 "Start observation parameter optimization"). The observation parameter optimization process is repeated until the termination conditions described in the embodiment described later are met.

[0056] The observation condition command unit 150 of the imaging device 100 (100B) then performs channel matrix acquisition processing for each observation parameter (step ST2132 "Acquire channel matrix for each observation parameter"). In the channel matrix acquisition processing of step ST2132, the observation parameter output unit 161 of the observation condition command unit 150 outputs the observation parameters for each observation condition to the observation device 300, and sequentially provides label information to identify each observation condition to the propagation characteristics calculation unit 120. The observation device 300 performs observations according to the observation conditions indicated in the observation parameters for each observation condition. The propagation characteristics calculation unit 120 calculates the radio wave propagation characteristics for each observation condition and outputs the channel matrix G representing the radio wave propagation characteristics to the channel matrix singular value calculation unit 162 of the observation condition command unit 150.

[0057] The observation condition command unit 150 of the imaging device 100 (100B) then performs a channel matrix singular value calculation process for each observation parameter (step ST2133 "Calculate channel matrix singular values ​​for each observation parameter"). In the channel matrix singular value calculation process of step ST2133, the channel matrix singular value calculation unit 162 of the observation condition command unit 150 calculates singular values ​​of the channel matrix that connect each grid point of the imaging target area with the transmit / receive channel of the observation device 300 for each observation condition, using the calculated propagation characteristics. The channel matrix singular value calculation unit 162 outputs the singular values ​​of the channel matrix to the observation parameter optimization unit 163.

[0058] The observation condition command unit 150 of the imaging device 100 (100B) then performs an optimization determination process (step ST2134 "Optimization Determination"). In the optimization determination process of step ST2134, the observation parameter optimization unit 163 of the observation condition command unit 150 determines observation conditions in which the distribution of singular values ​​in the channel matrix satisfies pre-set conditions. The observation parameter optimization unit 163 outputs the determined observation conditions to the observation device 300, thereby operating the observation device 300 with optimized observation conditions. The processing command unit 170 repeatedly executes the acquisition of geometry information by the geometry information acquisition unit 110, the calculation of radio wave propagation characteristics by the propagation characteristics calculation unit 120, and the determination of observation conditions by the observation condition command unit 150. The processing command unit 170 repeatedly executes the processing until the pre-set conditions are met.

[0059] The observation condition command unit 150 of the imaging device 100 (100B) then executes the observation parameter optimization completion process (step ST2135 "Observation parameter optimization completion").

[0060] The observation condition command unit 150 of the imaging device 100 (100B) then performs an observation parameter determination process (step ST2136 "Observation parameter determination"). In the observation parameter determination process of step ST2136, the observation parameter optimization unit 163 of the observation condition command unit 150 determines the observation parameters, which are the optimized observation conditions.

[0061] The observation condition command unit 150 of the imaging device 100 (100B) then terminates the process shown in Figure 8 ("terminate").

[0062] Let's return to the explanation of Figure 7. The imaging device 100 (100B) then performs observation command processing (step ST2140 "Observation Command"). In the observation command processing of step ST2140, the observation condition command unit 150 outputs observation parameters to the observation device 300.

[0063] The observation device 300 of imaging system 1 (1B) then performs observation processing (step ST2150 "Observation"). In the observation processing of step ST2150, when observation parameters indicating observation conditions are given, the observation device 300 performs observations based on the observation conditions indicated by the observation parameters. The observation device 300 outputs an observation signal indicating the observation result to imaging device 100 (100B).

[0064] The imaging device 100 (100B) of the imaging system 1 (1B) then performs an observation result acquisition process (step ST2160 "Observation Result Acquisition"). In the observation result acquisition process of step ST2160, the observation result acquisition unit 130 of the imaging device 100 (100B) acquires the observation results from the observation device 300. The observation result acquisition unit 130 acquires the observation results by receiving an observation signal indicating the observation results output by the observation device 300.

[0065] The imaging device 100 (100B) then performs imaging processing (step ST2170 "imaging processing"). In the imaging processing of step ST2170, the imaging processing unit 140 of the imaging device 100 (100B) performs imaging processing to generate an image of the out-of-line region using the radio wave propagation characteristics calculated by the propagation characteristics calculation unit and the observation results from the observation device.

[0066] The imaging device 100 (100B) then performs output destination processing (step ST2200 "output destination processing"). In the output destination processing of step ST2200, when the imaging processing unit 140 of the imaging device 100 (100A) in the imaging system 1 (1A) outputs image data, the output destination device 400 receives the image data and performs processing using the image data. Specifically, the output destination device 400 analyzes, for example, whether the image contains a target object.

[0067] The imaging device 100 (100B) then performs a termination determination process (step ST2310 "Termination?"). In the termination determination process of step ST2310, a control unit (not shown) of the imaging device 100 (100B) determines whether to terminate the processing of the imaging device 100 (100B). The control unit (not shown) determines whether to terminate the processing of the imaging device 100 (100B) according to, for example, an external termination command or execution program. If the control unit (not shown) determines that the imaging device 100 (100B) has not finished processing (step ST2310 "NO"), the process proceeds to step ST2000, and the process is repeated from step ST2000. If a control unit (not shown) determines that the imaging device 100 (100B) has finished processing (step ST2310 "YES"), the imaging device 100 (100B) then finishes the processing shown in Figure 7 ("Finish").

[0068] Here, we will explain the results of our verification of the effects of the above-described configuration. Figure 9 is a diagram showing a first example of a simulation of processing in an imaging apparatus according to Embodiment 2 of the present disclosure, where Figure 9A shows a singular value distribution and Figure 9B illustrates an example of image resolution. Figure 10 is a diagram showing a second example of a simulation of processing in an imaging apparatus according to Embodiment 2 of the present disclosure, where Figure 10A shows a singular value distribution and Figure 10B illustrates an example of image resolution. Figures 9 and 10 show the results of simulations in which the observation parameters were actually changed to image metal plate scatterers in the out-of-line region (dashed lines in Figures 9B and 10B). It can be seen that the image resolution is improved in Figure 10, where the distribution of singular values ​​is gentler (49 points at equal intervals from 1.5 to 2.5 GHz receiving frequencies, and 9 points at 0.1 m intervals between receiving points), compared to Figure 9, where the distribution is steeper (21 points at equal intervals from 1.5 to 2.5 GHz receiving frequencies, and 21 points at 0.05 m intervals between receiving points).

[0069] The imaging system, including the aforementioned imaging device, sequentially acquires environmental information, simulates the radio wave propagation characteristics of the environment, and allows for changes to the hardware configuration of the radar system's transmission and reception. By adaptively optimizing and changing the hardware configuration according to the radio wave propagation characteristics obtained from the simulation, the amount of information in the acquired signal can be increased, noise can be reduced, and stable detection and imaging can be achieved. Furthermore, even if the radar itself moves, a stable image can be obtained by repeating the same procedure.

[0070] This embodiment shows an example configuration that includes the following: (2) In the imaging apparatus described in (1) above, For each observation condition, the observation condition command unit calculates singular values ​​in the channel matrix that connects each grid point in the imaging target area to the transmit / receive channel of the observation device using the calculated propagation characteristics, and operates the observation device using observation conditions where the distribution of singular values ​​in the channel matrix satisfies the conditions set in advance. An imaging device further equipped with [features / features]. This disclosure thus has the effect of providing an imaging device that enables stable imaging of areas outside the line of sight.

[0071] This embodiment further illustrates an example configuration including the following: (3) In the imaging apparatus described in (2) above, The observation conditions include at least one of the following: the number of observation elements in the observation device, the position of the observation elements, or the observation frequency. An imaging apparatus characterized by the following features. As a result, this disclosure further provides an imaging device that enables observation under more favorable observation conditions when the observation device is a radar device, thereby enabling more stable imaging of areas beyond the line of sight. Furthermore, by applying the above configuration to an imaging system including an imaging device, or to the above imaging method, the same effects as described above can be achieved.

[0072] This embodiment further illustrates an example configuration including the following: (4) In the imaging apparatus described in (2) or (3) above, Acquisition of geometry information by the geometry information acquisition unit, Calculation of radio wave propagation characteristics by the aforementioned propagation characteristics calculation unit, And, Determination of observation conditions by the aforementioned observation condition command unit, A processing command unit that repeatedly executes the process, An imaging device further characterized by having the following features. As a result, this disclosure has the effect of providing an imaging device that can perform adaptive observations moment by moment and stably obtain high-precision images of areas beyond the line of sight. Furthermore, by applying the above configuration to an imaging system including an imaging device, or to the above imaging method, the same effects as described above can be achieved.

[0073] Embodiment 3. In the above-described embodiment 2, a configuration was described in which observation conditions suitable for imaging can be determined. Embodiment 3 describes an example of a configuration that allows for the determination of more suitable observation conditions. In Embodiment 3, for components related to Embodiment 3 that are the same as those related to Embodiment 1 or Embodiment 2 already described, redundant explanations will be omitted as appropriate.

[0074] The imaging system according to Embodiment 3 is an imaging system described in Embodiment 2, wherein the imaging device provides the objective function and optimization conditions for optimizing the observation parameters using the following equation (8). "m" is the number of singular values ​​sorted in descending order used during imaging. "C" is the singular value truncation threshold set at this time. Here, the observation parameters are repeatedly changed and searched so that the number of singular values ​​for which the singular value distribution of the transfer matrix is ​​greater than or equal to the set threshold is maximized. This parameter search can be done using any method, such as Monte Carlo or a genetic algorithm. The process ends when a predetermined number of iterations is reached or when the objective function F falls below the set threshold. Next, an example of the configuration of an imaging apparatus according to Embodiment 3 of this disclosure will be described. The configuration of the imaging device according to Embodiment 3 differs from the observation condition command unit 150 in the imaging device 100 (100B) described earlier. The observation condition command unit 150 operates the observation device to satisfy the objective function F of the conditions expressed in the following equation (8), using singular values ​​calculated in the same manner as the observation condition command unit 150 described above. maximize F, F=m st σ m >C …(8) In the above equation, “σ m " represents the singular values ​​sorted in descending order of the channel matrix. "m" is the number of singular values. "C" is the pre-set singular value censoring threshold.

[0075] This embodiment further illustrates an example configuration including the following: (5) In the imaging apparatus described in any of (2), (3), or (4) above, The aforementioned observation condition command unit is: Using the calculated singular values, the observation device is operated to satisfy the objective function F expressed in the following equation (a): An imaging apparatus characterized by the following features. maximize F, F=m st σ m>C …(a) In the above equation, “σ m ": These are singular values ​​sorted in descending order of the channel matrix. “m”: This represents the number of singular values. "C": This is a pre-set threshold for terminating outliers. This disclosure further has the effect of providing an imaging device that can be adaptively and optimally configured. Furthermore, by applying the above configuration to an imaging system including an imaging device, or to the above imaging method, the same effects as described above can be achieved.

[0076] Embodiment 4. In the above-described embodiment 3, a first example of the objective function and conditions was explained. Embodiment 4 further describes a second example of the objective function and conditions. In Embodiment 4, for components related to Embodiment 4 that are the same as those related to Embodiment 1, Embodiment 2, or Embodiment 3 already described, redundant explanations will be omitted as appropriate.

[0077] The imaging system according to Embodiment 4 is the imaging system described in Embodiment 2, in which the objective function for optimizing the observation parameters is given by the following equation (9). Here, the observation parameters are repeatedly changed and searched so that the expected value of the gradient up to a set number m of singular values ​​in the channel matrix is ​​minimized. The parameter search may be performed using discrete methods such as the Monte Carlo method or a genetic algorithm, or a gradient method using the variation of the objective function with respect to changes in the observation parameters may be used. Next, an example of the configuration of an imaging apparatus according to Embodiment 4 of this disclosure will be described. The configuration of the imaging device according to Embodiment 4 differs from the observation condition command unit 150 in the imaging device 100 (100B) described earlier. The observation condition command unit 150 uses the calculated singular value to operate the observation device so as to satisfy the objective function F of the condition expressed in equation (9). TIFF2026076465000005.tif28166

[0078] This embodiment further illustrates an example configuration including the following: (6) In the imaging apparatus described in any of (2), (3), or (4) above, The aforementioned observation condition command unit is: Using the calculated singular values, the observation device is operated to satisfy the objective function F of the conditions expressed in equation (b) below. An imaging apparatus characterized by the following features. τ F,F=Σ i=1~m |σ i+1 -σ i | …(b) In the above equation, “σ i " represents the singular values ​​of the channel matrix sorted in descending order. “m” is the predetermined number of singular values ​​to transcribe. This disclosure further has the effect of providing an imaging device that can be adaptively and optimally configured. Furthermore, by applying the above configuration to an imaging system including an imaging device, or to the above imaging method, the same effects as described above can be achieved.

[0079] Embodiment 5. In the above-described embodiments 3 and 4, we described a configuration in which observation conditions suitable for imaging can be determined. Embodiment 5 describes an example of a configuration that further improves image accuracy. In Embodiment 5, for components related to Embodiment 5 that are the same as those described in Embodiments 1, 2, 3, or 4, redundant explanations will be omitted as appropriate.

[0080] Next, an example of the configuration of an imaging apparatus according to Embodiment 5 of this disclosure will be described. The configuration of the imaging apparatus according to Embodiment 5 differs from the imaging apparatus 100 (100A, 100B) described earlier in that the imaging processing unit 140 is different. The imaging processing unit 140 performs l1-norm regularization when executing imaging processing so as to satisfy the function M of the condition expressed in equation (10) below. ≥ M,M=||E s -Gn||2 2 +κ||n||1…(10) In the above equation, “E s " is E s ={E si} is the electric field component of the acquired signal for each channel. "n" is n={n j} is the image distribution of the target. "G" is G={G ij} is the channel matrix. "κ" is a pre-set parameter. || ||2 is the l2 norm. || ||1 is the l1 norm.

[0081] The imaging apparatus according to Embodiment 5 uses l1-norm minimization in the matrix solution method when obtaining an image in the imaging apparatus described above. This reflects the sparsity of the scatterer distribution and improves the accuracy of the image.

[0082] This embodiment further illustrates an example configuration including the following: (7) In the imaging apparatus described in any of (1), (2), (3), (4), (5), or (6) above, The imaging processing unit is When performing imaging processing, l1-norm regularization is performed so that the function M satisfies the condition expressed in equation (c) below. An imaging apparatus characterized by the following features. ≥ M,M=||E s -Gn||2 2 +κ||n||1…(c) In the above equation, “E s " is E s ={E si This is the electric field component of the acquired signal for each channel. “n” is n={n j This is the image distribution of the target. “G” is G={G ij This is a channel matrix. "κ" is a parameter that is set in advance. || ||2 is the l2 norm. || ||1 is the l1 norm. As a result, this disclosure can further provide an imaging device that reflects the sparsity of the scatterer distribution and enables improved image accuracy. Furthermore, by applying the above configuration to an imaging system including an imaging device, or to the above imaging method, the same effects as described above can be achieved.

[0083] Embodiment 6. In the above-described embodiment 5, a method of performing l1-norm minimization (l1-norm regularization) as the matrix solution method in imaging processing was explained. Embodiment 6 describes an example of a configuration in which total variation is used for solving the matrix in imaging processing. In Embodiment 6, for components related to Embodiment 6 that are the same as those described in Embodiments 1, 2, 3, 4, or 5, redundant explanations will be omitted as appropriate.

[0084] Next, an example of the configuration of an imaging apparatus according to Embodiment 6 of this disclosure will be described. The configuration of the imaging apparatus according to Embodiment 6 differs from the imaging apparatus 100 (100A, 100B) described earlier in that the imaging processing unit 140 is different. The imaging processing unit 140 performs imaging processing to generate an image of the out-of-line region using the radio wave propagation characteristics calculated by the propagation characteristics calculation unit 120 and the observation results from the observation device 300, and outputs the generated image as image data to the output device 400. When performing imaging processing, the imaging processing unit 140 performs total variation regularization to satisfy the function M of the condition expressed in equation (11). ≥ M,M=||E s -Gn||2 2 +ηΣ {i,j} |n i -n j |1…(11) In the above equation, the second term of “M” represents the gradient of “n”, and the sum is taken for adjacent {i,j} combinations in the grid when the imaging region is discretized, avoiding duplication. “η” is a pre-set parameter. || ||² is the l² norm.

[0085] The imaging apparatus according to Embodiment 6 uses total variation in the matrix solution method when obtaining an image in the imaging apparatus described earlier. This emphasizes the boundary between the free space that gives the image and the object, thereby improving the accuracy of the image.

[0086] This embodiment further illustrates an example configuration including the following: (8) In the imaging device according to any one of the above (1), (2), (3), (4), (5), or (6), the imaging processing unit performs total variation regularization so as to satisfy a function M of the conditions represented by the following formula (d) in the execution of imaging processing. An imaging device characterized by this. minimize M, M = ||E s -Gn||2 2 +ηΣ {i,j} |n i -n j |1…(d) In the above formula, the second term of “M” means the gradient of “n”, and the sum is taken without duplication for combinations of adjacent {i, j} in the grid when the imaging area is discretized. “η” is a parameter set in advance. || ||2 is the l2 norm. Thereby, the present disclosure can further provide an imaging device that emphasizes the free space giving the image and the boundary of the object, and enables improvement of the accuracy of the image. In addition, the present disclosure applies the above configuration to an imaging system including an imaging device or the above imaging method, thereby achieving the same effect as the above effect.

[0087] Embodiment 7. The above-described Embodiments 3, 4, 5, and 6 may be combined. [[ID=4I]]In Embodiment 7, a configuration example of a form in which Embodiments 3, 4, 5, and 6 are combined will be described. In Embodiment 7, among the components according to Embodiment 7, for components similar to those according to Embodiment 1, Embodiment 2, Embodiment 3, Embodiment 4, Embodiment 5, or Embodiment 6, which have already been described, duplicate explanations will be omitted as appropriate.

[0088] The imaging system according to Embodiment 7 is configured by combining Embodiments 3, 4, 5, and 6, which have already been described. The imaging system 1 described below determines the observation parameters in Embodiment 3 or Embodiment 4, performs singular value decomposition, and truncates the singular values, and then performs conditional minimization of the resulting matrix using the following equation (12). In equation (12) below, imaging is performed by appropriately setting the threshold C for truncation singularities or the number of singularities m and the minimization parameters κ and η. Next, an example of the configuration of an imaging apparatus according to Embodiment 7 of this disclosure will be described. The configuration of the imaging apparatus according to Embodiment 7 differs from the imaging apparatus 100 (100A, 100B) described earlier in that the imaging processing unit 140 is different. The imaging processing unit 140 performs imaging processing to generate an image of the out-of-line region using the radio wave propagation characteristics calculated by the propagation characteristics calculation unit 120 and the observation results from the observation device 300, and outputs the generated image as image data to the output device 400. In executing the imaging processing, the imaging processing unit 140 terminates the calculation of singularities and generates the image using a combination of l1 norm regularization and total variation regularization so as to satisfy the function M of the condition expressed in the following equation (12). TIFF2026076465000006.tif38166

[0089] This embodiment further illustrates an example configuration including the following: (9) In the imaging apparatus described in any of (1), (2), (3), (4), (5), or (6) above, The imaging processing unit is In the execution of the imaging process, the calculation of singularities is terminated and an image is generated using a combination of l1 norm regularization and total variation regularization, such that the function M of the condition expressed in equation (e) below is satisfied. An imaging apparatus characterized by the following features. TIFF2026076465000007.tif47166 This disclosure has the added benefit of providing an optimized imaging apparatus. Furthermore, by applying the above configuration to an imaging system including an imaging device, or to the above imaging method, the same effects as described above can be achieved.

[0090] Here, we will describe the hardware configuration required to realize the functions of this disclosure. Figure 11 shows a first example of a hardware configuration for realizing the functions of this disclosure. Figure 12 shows a second example of a hardware configuration for realizing the functions of this disclosure. The imaging apparatus 100 (100A, 100B) of this disclosure is implemented by hardware as shown in Figure 11 or Figure 12, respectively.

[0091] Each imaging device 100 (100A, 100B) is composed of, for example, a processor 10001, a memory 10002, an input / output interface 10003, and a communication circuit 10004, as shown in Figure 11. The processor 10001 and memory 10002 are, for example, components installed in a computer. Memory 10002 stores a program that causes the computer to function as the geometry information acquisition unit 110, propagation characteristics calculation unit 120, observation result acquisition unit 130, imaging processing unit 140, observation condition command unit 150, observation parameter output unit 161, channel matrix singular value calculation unit 162, observation parameter optimization unit 163, processing command unit 170, and a control unit (not shown) of the imaging device 100 (100A, 100B). By the processor 10001 reading and executing the program stored in memory 10002, the functions of the geometry information acquisition unit 110, propagation characteristics calculation unit 120, observation result acquisition unit 130, imaging processing unit 140, observation condition command unit 150, observation parameter output unit 161, channel matrix singular value calculation unit 162, observation parameter optimization unit 163, processing command unit 170, and a control unit (not shown) of the imaging device 100 (100A, 100B) are realized. Furthermore, a storage unit (not shown) is realized by memory 10002 or other memory (not shown). Furthermore, a communication unit (not shown) is realized by the communication circuit 10004.

[0092] Processor 10001 uses, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a microprocessor, a microcontroller, or a DSP (Digital Signal Processor). Memory 10002 may be a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable Read Only Memory), or flash memory; it may be a magnetic disk such as a hard disk or flexible disk; it may be an optical disk such as a CD (Compact Disc) or DVD (Digital Versatile Disc); or it may be a magneto-optical disk. The processor 10001 and the memory 10002 or the communication circuit 10004 are connected in a way that allows them to transmit data to each other. Furthermore, the processor 10001, the memory 10002, and the communication circuit 10004 are connected to other hardware via the input / output interface 10003 in a way that allows them to transmit data to each other.

[0093] Alternatively, the functions of the geometry information acquisition unit 110, propagation characteristic calculation unit 120, observation result acquisition unit 130, imaging processing unit 140, observation condition command unit 150, observation parameter output unit 161, channel matrix singular value calculation unit 162, observation parameter optimization unit 163, processing command unit 170, and control unit (not shown) in the imaging device 100 (100A, 100B) may be realized by a dedicated processing circuit 20001, as shown in Figure 12.

[0094] The processing circuit 20001 may utilize, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field-Programmable Gate Array), a SoC (System-on-a-Chip), or a system LSI (Large-Scale Integration). Furthermore, a storage unit (not shown) is realized by memory 20002 or other memory (not shown). Memory 20002 may be a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable Read Only Memory), or flash memory; it may be a magnetic disk such as a hard disk or flexible disk; it may be an optical disk such as a CD (Compact Disc) or DVD (Digital Versatile Disc); or it may be a magneto-optical disk. Furthermore, a communication unit (not shown) is realized by the communication circuit 20004. The processing circuit 20001 and the memory 20002 or the communication circuit 20004 are connected in a way that allows them to transmit data to each other. Furthermore, the processing circuit 20001, the memory 20002, and the communication circuit 20004 are connected in a way that allows them to transmit data to other hardware via the input / output interface 20003. Furthermore, the functions of the geometry information acquisition unit 110, propagation characteristic calculation unit 120, observation result acquisition unit 130, imaging processing unit 140, observation condition command unit 150, observation parameter output unit 161, channel matrix singular value calculation unit 162, observation parameter optimization unit 163, processing command unit 170, and control unit (not shown) in the imaging device 100 (100A, 100B) may be implemented by separate processing circuits, or they may be implemented together in a single processing circuit.

[0095] Alternatively, some functions of the geometry information acquisition unit 110, propagation characteristic calculation unit 120, observation result acquisition unit 130, imaging processing unit 140, observation condition command unit 150, observation parameter output unit 161, channel matrix singular value calculation unit 162, observation parameter optimization unit 163, processing command unit 170, and control unit (not shown) in the imaging device 100 (100A, 100B) may be implemented by the processor 10001 and memory 10002, while the remaining functions are implemented by the processing circuit 20001.

[0096] Within the scope of this disclosure, it is possible to freely combine the embodiments, modify any component of each embodiment, or omit any component of each embodiment.

[0097] This disclosure enables imaging of objects that may exist in areas beyond the line of sight, and is therefore suitable for use in imaging devices that use radar in, for example, autonomous driving, autonomous robots, or advanced driver-assistance systems. [Explanation of Symbols]

[0098] 1(1A,1B) Imaging system, 100(100A,100B) Imaging device, 110 Geometry information acquisition unit, 120 Propagation characteristic calculation unit, 130 Observation result acquisition unit, 140 Imaging processing unit, 150 Observation condition command unit, 161 Observation parameter output unit, 162 Channel matrix singular value calculation unit, 163 Observation parameter optimization unit, 170 Processing command unit, 200 Geometry information acquisition device, 300 Observation device, 400 Output destination device, 10001 Processor, 10002 Memory, 10003 Input / output interface, 10004 Communication circuit, 20001 Processing circuit, 20002 Memory, 20003 Input / output interface, 20004 Communication circuit.

Claims

1. A geometry information acquisition unit acquires geometry information representing the environment of the imaging target area, which is outside the line of sight. A propagation characteristics calculation unit calculates the radio wave propagation characteristics in the environment of the imaging target region using the aforementioned geometry information and the observation conditions of an observation device that performs observations using radio waves. An observation result acquisition unit that acquires observation results from the aforementioned observation device, An imaging processing unit that performs imaging processing to generate an image of the beyond-line-of-sight region using the radio wave propagation characteristics calculated by the propagation characteristics calculation unit and the observation results from the observation device, An imaging device equipped with this device.

2. In the imaging apparatus according to claim 1, For each observation condition, the observation condition command unit calculates singular values ​​in the channel matrix that connects each grid point in the imaging target area to the transmit / receive channel of the observation device using the calculated propagation characteristics, and operates the observation device using observation conditions where the distribution of singular values ​​in the channel matrix satisfies the conditions set in advance. An imaging device further characterized by having the following features.

3. In the imaging apparatus according to claim 2, The observation conditions include at least one of the following: the number of observation elements in the observation device, the position of the observation elements, or the observation frequency. An imaging apparatus characterized by the following features.

4. In the imaging apparatus according to claim 2 or claim 3, Acquisition of geometry information by the geometry information acquisition unit, Calculation of radio wave propagation characteristics by the aforementioned propagation characteristics calculation unit, And, Determination of observation conditions by the aforementioned observation condition command unit, A processing command unit that repeatedly executes the process, An imaging device further characterized by having the following features.

5. In the imaging apparatus according to claim 2 or claim 3, The aforementioned observation condition command unit is: Using the calculated singular values, the observation device is operated to satisfy the objective function F of the conditions expressed in the following equation (a). An imaging apparatus characterized by the following features. maximize F, F=m s.t. σ ｍ >C …(a) In the above equation, "σ ｍ " represents the singular values ​​of the channel matrix sorted in descending order. "m" is the number of singular values. "C" is a predetermined threshold for terminating outliers.

6. In the imaging apparatus according to claim 2 or claim 3, The aforementioned observation condition command unit is: Using the calculated singular values, the observation device is operated to satisfy the objective function F of the conditions expressed in equation (b) below. An imaging apparatus characterized by the following features. mmimize F,F=S ｉ＝１～ｍ |s ｉ＋１ -s ｉ | …(b) In the above equation, "σ ｉ " represents the singular values ​​of the channel matrix sorted in descending order. "m" is the predetermined number of singular values ​​to be truncated.

7. In the imaging apparatus according to any one of claims 1 to 3, In the imaging apparatus described above, The imaging processing unit is In performing the imaging process, the function M satisfies the condition expressed in equation (c) below. １ Perform norm regularization. An imaging apparatus characterized by the following features. minimize M,M=||E ｓ -Gn|| ２ ２ +κ||n|| １ …(c) In the above equation, "E ｓ " is E ｓ = {E ｓｉ This is the electric field component of the acquired signal for each channel. "n" is n = {n ｊ This is the image distribution of the target. "G" is G = {G ｉｊ This is a channel matrix. "κ" is a parameter that is set in advance. || || ２ is ２ It is a norm. || || １ is １ It is a norm.

8. In the imaging apparatus according to any one of claims 1 to 3, The imaging processing unit is During the imaging process, total variation regularization is performed so as to satisfy the function M of the condition expressed in equation (d) below. An imaging apparatus characterized by the following features. ________||. ｓ || ２ ２ _ηΣ ｛ｉ，ｊ｝ |� ｉ  ｊ | １ ...() In the above equation, The second term of "M" represents the gradient of "n". The sum is calculated by avoiding duplication for adjacent {i, j} combinations in the grid obtained when the imaging region is discretized. "η" is a parameter that is set in advance. || || ２ is ２ It is a norm.

9. In the imaging apparatus according to any one of claims 1 to 3, The imaging processing unit is In the execution of the imaging process, the calculation of singularities is terminated so that the function M of the condition expressed in the following equation (e) is satisfied, l １ Images are generated using a combination of norm regularization and total variation regularization. An imaging apparatus characterized by the following features.

10. An observation device that performs observations using radio waves and is configured to allow changes in observation conditions, A geometry information acquisition device that acquires geometric information representing the environment of the imaging target area, which is outside the line of sight, An imaging apparatus comprising: a propagation characteristics calculation unit that calculates radio wave propagation characteristics in the environment of the imaging target region using the geometry information and the observation conditions of the observation device; an observation result acquisition unit that acquires observation results from the observation device; and an imaging processing unit that performs imaging processing to generate an image of the beyond-line-of-sight region using the radio wave propagation characteristics calculated by the propagation characteristics calculation unit and the observation results from the observation device, An imaging system equipped with this system.

11. An imaging method for generating an image using an imaging device, A geometry information reception step that acquires geometry information representing the environment of the imaging target area which is an area outside the line of sight, A propagation characteristics calculation step that calculates the radio wave propagation characteristics in the environment of the imaging target region using the aforementioned geometry information and the observation conditions of an observation device that performs observations using radio waves, An observation result acquisition step of acquiring observation results by the aforementioned observation device, An imaging processing step is performed to generate an image of the out-of-line region using the radio wave propagation characteristics calculated in the propagation characteristics calculation step and the observation results from the observation device. Imaging methods are provided.

Images