Image reproduction device, image reproduction method, and dangerous object detection system

Abstract

An image reproduction device (3) is configured to comprise: an observation data acquisition unit (11) that acquires observation data of the surface of a subject from a depth camera (1) that observes said surface; and a reception signal acquisition unit (12) that acquires, from a radar (2) that receives reflected waves from the subject, reception signals of the reflected waves. The image reproduction device (3) further comprises: a three-dimensional position calculation unit (13) that calculates three-dimensional positions of a point cloud of interest on the surface of the subject on the basis of the observation data acquired by the observation data acquisition unit (11), and calculates three-dimensional positions of a point cloud of interest inside the subject on the basis of the three-dimensional positions of the surface point cloud of interest; and an image generation unit (14) that calculates, by using the reception signals acquired by the reception signal acquisition unit (12), image values of the inside point cloud of interest that is present at the three-dimensional positions calculated by the three-dimensional position calculation unit (13), and generates an inverse synthetic aperture radar image, which is an image having the image values of the inside point cloud of interest.

Description

Image playback device, image playback method, and hazardous materials detection system

[0001] This disclosure relates to an image playback device, an image playback method, and a hazardous materials detection system.

[0002] There is an image reproduction device that calculates the image values ​​of a point cloud of interest on the surface of a subject and generates an inverse synthetic aperture radar image (hereinafter referred to as "ISAR (Inverse Synthetic Aperture Radar) image") which is an image containing the image values ​​of the point cloud of interest. The ISAR image generated by the image reproduction device can be used, for example, to detect dangerous objects held by the subject. As such an image reproduction device, for example, Patent Document 1 discloses an image reproduction device that generates an ISAR image based on observation data from a depth camera that observes the surface of a subject and the received signal from a radar that receives reflected waves from the subject.

[0003] Special Publication No. 2021-505891

[0004] The image reproduction device disclosed in Patent Document 1 has the problem that it can generate an ISAR image of the surface of a subject, but cannot generate an ISAR image of the inside of the subject. Therefore, even if the ISAR image generated by the image reproduction device is monitored, it may not be possible to detect dangerous objects hidden inside the subject. Examples of the inside of a subject include the deep parts of the clothing worn by the subject, or the inside of the body of the subject.

[0005] This disclosure was made to solve the above-mentioned problems and aims to provide an image playback device that can generate ISAR images of the inside of a subject.

[0006] The image playback device according to this disclosure includes an observation data acquisition unit that acquires surface observation data from a depth camera that observes the surface of an object, and a reception signal acquisition unit that acquires a received signal of reflected waves from a radar that receives reflected waves from the object. The image playback device also includes a three-dimensional position calculation unit that calculates the three-dimensional position of a group of points of interest inside the object based on the observation data acquired by the observation data acquisition unit and calculates the three-dimensional position of a group of points of interest inside the object based on the three-dimensional position of the group of points of interest on the surface, and an image generation unit that uses the received signal acquired by the reception signal acquisition unit to calculate the image values ​​of the internal group of points of interest located at the three-dimensional position calculated by the three-dimensional position calculation unit, and generates an inverse synthetic aperture radar image which is an image having the image values ​​of the internal group of points of interest.

[0007] According to this disclosure, it is possible to generate an inverse synthetic aperture radar image of the inside of a subject.

[0008] This is a configuration diagram showing a hazardous materials detection system including an image playback device 3 according to Embodiment 1. This is a configuration diagram showing the image playback device 3 according to Embodiment 1. This is a hardware configuration diagram showing the hardware of the image playback device 3 according to Embodiment 1. This is a hardware configuration diagram of a computer when the image playback device 3 is implemented by software or firmware, etc. This is an explanatory diagram showing the position x of the subject at observation time t. This is an explanatory diagram showing the point cloud of interest on the surface of the subject and the point cloud of interest inside the subject, respectively. This is a flowchart showing the image playback method, which is the processing procedure of the image playback device 3. This shows the depth camera 1 and surface node n in three-dimensional space. ｓｕｒｆ This is an explanatory diagram showing the correspondence with (i). This is an explanatory diagram showing an example of an ISAR image that contains a handgun, which is a dangerous object.

[0009] To provide a more detailed explanation of this disclosure, the forms for implementing this disclosure will be described below with reference to the attached drawings.

[0010] Embodiment 1. Figure 1 is a configuration diagram showing a hazardous materials detection system including an image playback device 3 according to Embodiment 1. Figure 2 is a configuration diagram showing the image playback device 3 according to Embodiment 1. Figure 3 is a hardware configuration diagram showing the hardware of the image playback device 3 according to Embodiment 1. In Figure 1, the hazardous materials detection system includes a depth camera 1, a radar 2, and an image playback device 3.

[0011] The depth camera 1 is implemented, for example, by a camera with a depth sensor or an RGBD (Red Green Blue Depth) camera. The depth camera 1 observes the surface of the object. The depth camera 1 outputs surface observation data at multiple observation times to the image playback device 3. The radar 2 irradiates the object with electromagnetic waves and receives the reflected waves from the object. The radar 2 outputs the received signal of the reflected waves to the image playback device 3. In the hazardous material detection system shown in Figure 1, the radar 2 irradiates the object with electromagnetic waves. However, this is just one example, and other devices (not shown) may irradiate the object with electromagnetic waves, and the radar 2 may receive the reflected waves from the object.

[0012] As shown in Figure 2, the image playback device 3 comprises an observation data acquisition unit 11, a received signal acquisition unit 12, a three-dimensional position calculation unit 13, an image generation unit 14, and a hazardous material detection unit 15. The image playback device 3 acquires surface observation data from the depth camera 1 at multiple observation times and acquires received signals of reflected waves from the radar 2 at multiple observation times. Based on the surface observation data and the received signals of reflected waves, the image playback device 3 generates an inverse synthetic aperture radar image (hereinafter referred to as "ISAR image"). Based on the ISAR image, the image playback device 3 detects hazardous materials.

[0013] The observation data acquisition unit 11 is implemented, for example, by the observation data acquisition circuit 21 shown in Figure 3. The observation data acquisition unit 11 acquires observation data of the surface of the subject from the depth camera 1 at multiple observation times. The observation data acquisition unit 11 outputs the observation data to the 3D position calculation unit 13.

[0014] The received signal acquisition unit 12 is implemented, for example, by the received signal acquisition circuit 22 shown in Figure 3. The received signal acquisition unit 12 acquires received signals of reflected waves from the radar 2 at multiple observation times. The received signal acquisition unit 12 outputs the received signals to the image generation unit 14.

[0015] The 3D position calculation unit 13 is implemented, for example, by the 3D position calculation circuit 23 shown in Figure 3. The 3D position calculation unit 13 acquires observation data for multiple observation times from the observation data acquisition unit 11. Based on the observation data acquired by the observation data acquisition unit 11, the 3D position calculation unit 13 calculates the 3D position of the point cloud of interest on the surface of the object. Based on the 3D position of the point cloud of interest on the surface of the object, the 3D position calculation unit 13 calculates the 3D position of the point cloud of interest inside the object.

[0016] Specifically, the 3D position calculation unit 13 calculates the 3D position of the target point group on the surface at the first observation time based on the observation data from the first observation time among the observation data from multiple observation times. The 3D position calculation unit 13 calculates the displacement of the target point group on the surface at the second and subsequent observation times based on the observation data from the second and subsequent observation times. Then, the 3D position calculation unit 13 calculates the 3D position of the target point group on the surface at the second and subsequent observation times from the 3D position of the target point group on the surface at the first observation time and the displacement of the target point group on the surface at the second and subsequent observation times. The 3D position calculation unit 13 calculates the displacement of the target point group inside at the second and subsequent observation times based on the displacement of the target point group on the surface at the second and subsequent observation times. The 3D position calculation unit 13 then calculates the 3D position of the internal point group at subsequent observation times from the 3D position of the point group on the surface at the first observation time and the displacement of the position of the internal point group at subsequent observation times. The 3D position calculation unit 13 outputs position data indicating the 3D position of the point group on the surface of the subject and position data indicating the 3D position of the internal point group to the image generation unit 14.

[0017] The image generation unit 14 is implemented, for example, by the image generation circuit 24 shown in Figure 3. The image generation unit 14 acquires a received signal from the received signal acquisition unit 12 and acquires position data from the 3D position calculation unit 13. Using the received signal, the image generation unit 14 calculates the image values ​​of the point group of interest located at the 3D position indicated by the position data, and generates an ISAR image, which is an image containing the image values ​​of the point group of interest.

[0018] Specifically, the image generation unit 14 calculates the distance spectrum at each observation time from the received signals at each observation time acquired by the received signal acquisition unit 12. Based on the distance spectrum at each observation time, the image generation unit 14 calculates the image values ​​of the target point group on the surface and inside at the three-dimensional position calculated by the three-dimensional position calculation unit 13. The image generation unit 14 generates an ISAR image by assigning the image values ​​of the target point group to the three-dimensional position calculated by the three-dimensional position calculation unit 13. The image generation unit 14 outputs image data showing the ISAR image to the hazardous material detection unit 15.

[0019] The hazardous materials detection unit 15 is implemented, for example, by a hazardous materials detection circuit 25. The hazardous materials detection unit 15 acquires image data representing the ISAR image from the image generation unit 14. The hazardous materials detection unit 15 performs hazardous materials detection processing based on the ISAR image represented by the image data. Examples of hazardous materials include knives and handguns.

[0020] In Figure 2, it is assumed that each of the components of the image playback device 3—the observation data acquisition unit 11, the received signal acquisition unit 12, the 3D position calculation unit 13, the image generation unit 14, and the hazardous material detection unit 15—is implemented by dedicated hardware as shown in Figure 3. That is, it is assumed that the image playback device 3 is implemented by an observation data acquisition circuit 21, a received signal acquisition circuit 22, a 3D position calculation circuit 23, an image generation circuit 24, and a hazardous material detection circuit 25. Each of the observation data acquisition circuit 21, the received signal acquisition circuit 22, the 3D position calculation circuit 23, the image generation circuit 24, and the hazardous material detection circuit 25 can be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0021] The components of the image playback device 3 are not limited to those realized by dedicated hardware; the image playback device 3 may also be realized by software, firmware, or a combination of software and firmware. The software or firmware is stored as a program in the computer's memory. The computer refers to the hardware that executes the program, and includes, for example, a CPU (Central Processing Unit), GPU (Graphics Processing Unit), central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, processor, or DSP (Digital Signal Processor).

[0022] FIG. 4 is a hardware configuration diagram of a computer when the image playback device 3 is realized by software, firmware, or the like. When the image playback device 3 is realized by software, firmware, or the like, a program for causing a computer to execute the respective processing procedures in the observation data acquisition unit 11, the reception signal acquisition unit 12, the three-dimensional position calculation unit 13, the image generation unit 14, and the dangerous object detection unit 15 is stored in the memory 31. Then, the processor 32 of the computer executes the program stored in the memory 31.

[0023] Further, FIG. 3 shows an example in which each component of the image playback device 3 is realized by dedicated hardware, and FIG. 4 shows an example in which the image playback device 3 is realized by software, firmware, or the like. However, this is only an example, and some components of the image playback device 3 may be realized by dedicated hardware and the remaining components may be realized by software, firmware, or the like.

[0024] Next, the operation of the dangerous object detection system shown in FIG. 1 will be described. As shown in FIG. 1, the subject moves, for example, across in front of each of the depth camera 1 and the radar 2. FIG. 5 is an explanatory diagram showing the position x of the subject at the observation time t. In FIG. 5, the position of the subject at the observation time t = t ０ is x ０ , the position of the subject at the observation time t = t １ is x １ , the position of the subject at the observation time t = t ２ is x ２ , the position of the subject at the observation time t = t ３ is x ３ . In the example of FIG. 5, the position x ０ is the reference position with respect to the positions x １ , x ２ , x ３ . x ＴＸ is the position of the transmission antenna of the radar 2, and x ＲＸ is the position of the reception antenna of the radar 2.

[0025] The depth camera 1 observes the surface of the subject. That is, as shown in Figure 6, the depth camera 1 observes the cluster of points of interest on the surface of the subject at observation time t. The depth camera 1 outputs the observation data of the cluster of points of interest on the surface to the image playback device 3. Figure 6 is an explanatory diagram showing the cluster of points of interest on the surface of the subject and the cluster of points of interest inside the subject. In Figure 6, the cluster of points of interest on the surface of the subject is at surface node n. ｓｕｒｆ It is represented by (i). i = 1, ..., I. In the example in Figure 6, I = 13. In Figure 6, the group of points of interest inside the subject is the internal node n ｉｎ，ｍ (i) is represented as follows: m = 1, ..., M. In the example in Figure 6, M = 3. The internal node n with a large value of m ｉｎ，ｍ (i) indicates an internal node located deeper within the subject. In the example in Figure 6, internal node n ｉｎ，ｍ (i) The density of surface node n ｓｕｒｆ (i) The density is sparser than (i). Also, the internal nodes n have a large value of m. ｉｎ，ｍ (i) the internal node n ｉｎ，ｍ (i) has a low density.

[0026] Radar 2 is synchronized with depth camera 1, for example, by an external trigger. Radar 2 irradiates the subject with electromagnetic waves and the observation time t from the subject ｓ It receives the reflected wave. Radar 2, at observation time t ｓ The received signal f(t) of the reflected wave ｓ The output (,k) is sent to the image playback device 3. s is the frame number, where s = 0, 1, 2, 3, ... k is the wavenumber.

[0027] Figure 7 is a flowchart showing the image playback method, which is the processing procedure of the image playback device 3. The observation data acquisition unit 11 receives the observation time t from the depth camera 1. ０ Surface node n ｓｕｒｆ (i) Observation data is acquired (step ST1 in Figure 7). The observation data acquisition unit 11 acquires observation time t ０ Surface node n ｓｕｒｆ (i) The observation data is output to the 3D position calculation unit 13. The observation data from the depth camera 1 is obtained as shown in the following equation (1) at the observation time t０ Surface node n ｓｕｒｆ (i) contains data showing the polar coordinates of the three-dimensional coordinate system. ｉ ､φ ｉ For details, see Figure 8. ｓｕｒｆ (i) = (rsinθ) ｉ cosφ ｉ ,rsinθ ｉ sinφ ｉ ,rcosθ ｉ (1) Figure 8 shows the depth camera 1 and surface node n in three-dimensional space. ｓｕｒｆ This is an explanatory diagram showing the correspondence with (i).

[0028] Furthermore, the observation data acquisition unit 11 receives the observation time t from the depth camera 1. １， t ２， t ３ Surface node n ｓｕｒｆ (i) Obtain the observation data (step ST1 in Figure 7). Observation time t １， t ２， t ３ Surface node n ｓｕｒｆ (i) Observation data is obtained at observation time t ０ Surface node n ｓｕｒｆ (i) contains data showing the displacement from the original position. This displacement is, for example, observed at time t. ０ Surface node n ｓｕｒｆ (i) includes the amount of translation and rotation from (i). This displacement is expressed, for example, in dual quaternion format. The observation data acquisition unit 11 records the observation time t １， t ２， t ３ Surface node n ｓｕｒｆ (i) The observation data is output to the 3D position calculation unit 13.

[0029] The received signal acquisition unit 12 receives the observation time t from the radar 2. ｓ The received signal f(t) of the reflected wave ｓ The received signal acquisition unit 12 acquires the received signal f(t) (step ST2 in Figure 7). ｓ The output of ,k) is sent to the image generation unit 14.

[0030] The 3D position calculation unit 13 receives the observation time t from the observation data acquisition unit 11. ０， t １， t ２， t ３ Surface node n ｓｕｒｆ (i) Observation data is acquired. The 3D position calculation unit 13 calculates the observation time t ０ Surface node n ｓｕｒｆ (i) Polar coordinates of the 3D coordinate system and observation time t (t = t １， t ２， t ３ ) Surface node n ｓｕｒｆ (i) From the displacement amount, the observation time t (t = t １， t ２， t ３ ) Surface node n ｓｕｒｆ (i) The polar coordinates of the three-dimensional coordinates are calculated (step ST3 in Figure 7). The three-dimensional position calculation unit 13 calculates the observation time t (t = t ０， t １， t ２， t ３ ) Surface node n ｓｕｒｆ (i) The position data representing the polar coordinates of the three-dimensional coordinates is output to the image generation unit 14.

[0031] The three-dimensional position calculation unit 13 calculates the observation time t (t=t １， t ２， t ３ ) Surface node n ｓｕｒｆ (i) By interpolating the displacement amount in the direction of the object, the observed time t (t = t １， t ２， t ３ ) internal node n ｉｎ，ｍ (i) Calculate the displacement amount (step ST4 in Figure 7).

[0032] The following describes the internal node n calculated by the 3D position calculation unit 13. ｉｎ，ｍ The process for calculating the displacement in (i) will be explained in detail. A dual quaternion is a number q expressed in the form of equation (2) below, using three types of elements i, j, and k having the following property (1), and one type of element ε having the following property (2).

[0033] In equation (2), pｗ , p ｘ , p ｙ , p ｚ , q ｗ , q ｘ , q ｙ , q ｚ are real numbers.

[0034] As a calculation method of interpolation using the displacement expressed in the dual quaternion form, DQB (Dual Quaternion Blending) is known. The three-dimensional position calculation unit 13, for example, uses the calculation method of DQB to calculate the displacement of the internal node n １， t ２， t ３ (i) at the observation time t (t = t ｉｎ，ｍ ). When calculating the displacement using the calculation method of DQB, the three-dimensional position calculation unit 13 sets the three-dimensional reference position coordinates of the internal node as x, and the reference posture of the subject (the posture of the subject at the observation time t ０ ) the set of "nodes for which the displacement has been calculated" existing in the vicinity of the coordinate x of the internal node at that time as N(x). The three-dimensional position calculation unit 13 calculates the displacement q, which is the displacement of the internal node n ｉｎ，ｍ (i), according to the following formula (3).

[0035] In formulas (3) and (4), ω ｋ (x) is the weight coefficient of the k-th peripheral node, and q ｋ is the dual quaternion representing the displacement of the k-th peripheral node. The weight coefficient ω ｋ (x) is a function of the distance between the coordinate x of the internal node of interest and the coordinate x ｋ of the k-th peripheral node in the reference posture, and σ in formula (4) is a real coefficient. The normalized() function is a normalization operation for making the norm of the dual quaternion q representing the final displacement equal to 1.

[0036] The three-dimensional position calculation unit 13, based on the displacement q of the internal node n ｉｎ，ｍ (i), at the observation time t (t = t ０， t １， t ２， t ３) internal node n ｉｎ，ｍ (i) Calculate the polar coordinates of the 3D coordinates (Step ST5 in Figure 7). Internal node n ｉｎ，ｍ The polar coordinates of the three-dimensional coordinate system in (i) are expressed as shown in equation (5) below.

[0037] In equation (5), α is a real coefficient greater than 1.

[0038] Here, when the coordinates x = [x, y, z] of an internal node are transformed using a dual quaternion q, the x shown in equation (6) below is obtained. ＤＱ x is calculated, ＤＱ Using this, the dual quaternion x' is formed as shown in equation (7) below. ＤＱ The result is calculated using the dual quaternion x'. ＤＱ The expression inside ε is in the form shown in equation (8) below. From equation (8), the polar coordinate x' after the coordinate transformation can be found as x' = [x', y', z'].

[0039]

[0040] The three-dimensional position calculation unit 13 calculates the observation time t (t=t ０， t １， t ２， t ３ ) internal node n ｉｎ，ｍ (i) The position data showing the polar coordinates of the three-dimensional coordinates is output to the image generation unit 14. The three-dimensional position calculation unit 13 has an internal node n ｉｎ，ｍ When outputting position data indicating the polar coordinates of the three-dimensional coordinates of (i) to the image generation unit 14, internal node n ｉｎ，ｍ (i) The polar coordinates of the three-dimensional coordinates are converted to the coordinate system of radar 2, and position data showing the polar coordinates after the coordinate system conversion is output to the image generation unit 14. The coordinate system conversion process itself is a well-known technique, so a detailed explanation is omitted.

[0041] The image generation unit 14 receives the observed time t (t = t) from the received signal acquisition unit 12. ０， t １， t ２， t ３ ) Received signal f(t ｓThe image generation unit 14 obtains the observation time t(t=t) from the 3D position calculation unit 13. ０， t １， t ２， t ３ ) Surface node n ｓｕｒｆ (i) Position data showing the polar coordinates of the 3D coordinates, and observation time t (t = t ０， t １， t ２， t ３ ) internal node n ｉｎ，ｍ (i) The position data showing the polar coordinates of the three-dimensional coordinates is obtained. The image generation unit 14 receives the received signal f(t ｓ Perform an inverse Fourier transform on (t) and obtain the distance spectrum F(t) from the inverse Fourier transform result. ｓ The distance spectrum F(t) is calculated. R is the average one-way distance from the radar to the subject. The image generation unit 14 calculates the distance spectrum F(t) ｓ Using R) and position data, the image value G for each group of nodes of interest is set as follows, as shown in equation (9): surface node n ｓｕｒｆ (i) and internal node n ｉｎ，ｍ (i) Calculate the respective image values ​​G (step ST6 in Figure 7).

[0042] In equation (10), r ｓ is, t = t ｓ At that time, the radar and surface node n ｓｕｒｆ (i) or internal node n ｉｎ，ｍ This is the round-trip distance between (i) and (i).

[0043] The image generation unit 14 has a surface node n ｓｕｒｆ (i) and internal node n ｉｎ，ｍ (i) Each image value G is given by t = t ０ By assigning the three-dimensional position of each node in the basic posture at that time, an ISAR image in the basic posture is generated (step ST7 in Figure 7). The image generation unit 14 outputs image data showing the ISAR image to the hazardous material detection unit 15.

[0044] The hazardous material detection unit 15 acquires image data representing the ISAR image from the image generation unit 14. Based on the ISAR image represented by the image data, the hazardous material detection unit 15 performs a hazardous material detection process (step ST8 in Figure 7). The hazardous material detection process based on the ISAR image is a well-known technique, so a detailed explanation is omitted. If the hazardous material detection unit 15 detects a hazardous material, it displays a message indicating that a hazardous material has been detected on a display device (not shown), for example. Figure 9 is an explanatory diagram showing an example of an ISAR image in which a handgun, which is a hazardous material, is visible.

[0045] In the above embodiment 1, the image playback device 3 is configured to include an observation data acquisition unit 11 that acquires surface observation data from a depth camera 1 that observes the surface of an object, and a reception signal acquisition unit 12 that acquires a received signal of reflected waves from a radar 2 that receives reflected waves from the object. The image playback device 3 also includes a 3D position calculation unit 13 that calculates the 3D position of the point cloud of interest inside the object based on the observation data acquired by the observation data acquisition unit 11 and the 3D position of the point cloud of interest inside the object based on the 3D position of the point cloud of interest on the surface, and an image generation unit 14 that uses the received signal acquired by the reception signal acquisition unit 12 to calculate the image values ​​of the point cloud of interest inside that are located at the 3D position calculated by the 3D position calculation unit 13, and generates an inverse synthetic aperture radar image which is an image having the image values ​​of the point cloud of interest inside.Therefore, the image playback device 3 can generate an inverse synthetic aperture radar image of the inside of an object.

[0046] In Embodiment 1, the image playback device 3 is configured to include a hazardous material detection unit 15 that detects hazardous materials based on an inverse synthetic aperture radar image generated by the image generation unit 14. Therefore, the image playback device 3 can detect hazardous materials present inside the subject.

[0047] In Embodiment 1, the image playback device 3 was configured such that the density of the point cloud of interest inside the subject was sparser than the density of the point cloud of interest on the surface of the subject. Therefore, the image playback device 3 can reduce the computational load compared to when the density of the point cloud of interest inside the subject is the same as the density of the point cloud of interest on the surface of the subject.

[0048] It should be noted that this disclosure allows for modifications of any component of the embodiment, or the omission of any component of the embodiment.

[0049] The image playback device described herein can generate an ISAR image of the inside of a subject and is suitable for use in image playback devices and the like.

[0050] 1 Depth camera, 2 Radar, 3 Image playback device, 11 Observation data acquisition unit, 12 Received signal acquisition unit, 13 3D position calculation unit, 14 Image generation unit, 15 Hazardous material detection unit, 21 Observation data acquisition circuit, 22 Received signal acquisition circuit, 23 3D position calculation circuit, 24 Image generation circuit, 25 Hazardous material detection circuit, 31 Memory, 32 Processor.

Claims

1. An image reproduction device comprising: an observation data acquisition unit that acquires observation data of the surface of an object from a depth camera that observes the surface of the object; a reception signal acquisition unit that acquires a received signal of the reflected wave from a radar that receives the reflected wave from the object; a three-dimensional position calculation unit that calculates the three-dimensional position of a group of points of interest on the surface of the object based on the observation data acquired by the observation data acquisition unit, and calculates the three-dimensional position of a group of points of interest inside the object based on the three-dimensional position of the group of points of interest on the surface; and an image generation unit that uses the received signal acquired by the reception signal acquisition unit to calculate the image values ​​of the group of points of interest inside the object that are located at the three-dimensional position calculated by the three-dimensional position calculation unit, and generates an inverse synthetic aperture radar image which is an image having the image values ​​of the group of points of interest inside the object.

2. The image playback device according to claim 1, characterized in that the observation data acquisition unit acquires observation data of the surface at multiple observation times from the depth camera, the three-dimensional position calculation unit calculates the three-dimensional position of the point cluster of interest on the surface at the first observation time based on the observation data of the first observation time from among the observation data of the multiple observation times, calculates the amount of displacement of the position of the point cluster of interest on the surface at the second and subsequent observation times based on the observation data of the second and subsequent observation times, and calculates the three-dimensional position of the point cluster of interest on the surface at the second and subsequent observation times from the three-dimensional position of the point cluster of interest on the surface at the first observation time and the amount of displacement of the position of the point cluster of interest on the surface at the second and subsequent observation times.

3. The image playback device according to claim 2, characterized in that the three-dimensional position calculation unit calculates the displacement amount of the position of the internal point group at the second and subsequent observation times based on the displacement amount of the position of the point group of interest on the surface at the second and subsequent observation times, and calculates the three-dimensional position of the internal point group of interest at the second and subsequent observation times from the three-dimensional position of the point group of interest on the surface at the first observation time and the displacement amount of the position of the internal point group of interest at the second and subsequent observation times.

4. The image playback device according to claim 3, characterized in that the three-dimensional position calculation unit calculates the displacement amount of the internal point group at the second and subsequent observation times by interpolating the displacement amount of the position of the point group at the surface at the second and subsequent observation times in the direction inward of the subject.

5. The image reproduction device according to any one of claims 1 to 4, characterized in that the receiving signal acquisition unit acquires the received signals of the reflected waves from the radar at the plurality of observation times, the image generation unit calculates the distance spectrum at each observation time from the received signals at each observation time acquired by the receiving signal acquisition unit, calculates the image values ​​of the internal point of interest group located at the three-dimensional position calculated by the three-dimensional position calculation unit based on the distance spectrum at each observation time, and assigns the image values ​​of the internal point of interest group to the three-dimensional position calculated by the three-dimensional position calculation unit to generate the inverse synthetic aperture radar image.

6. The image reproduction device according to any one of claims 1 to 5, further comprising a hazardous material detection unit that performs hazardous material detection processing based on an inverse synthetic aperture radar image generated by the image generation unit.

7. The image reproduction device according to any one of claims 1 to 6, characterized in that the density of the point cloud of interest inside the subject is sparser than the density of the point cloud of interest on the surface of the subject.

8. An image reproduction method comprising: an observation data acquisition unit acquiring observation data of the surface of an object from a depth camera that observes the surface of the object; a reception signal acquisition unit acquiring a received signal of the reflected wave from a radar that receives the reflected wave from the object; a three-dimensional position calculation unit calculating the three-dimensional position of the point cloud of interest on the surface of the object based on the observation data acquired by the observation data acquisition unit, and calculating the three-dimensional position of the point cloud of interest inside the object based on the three-dimensional position of the point cloud of interest on the surface; and an image generation unit using the received signal acquired by the reception signal acquisition unit to calculate the image values ​​of the point cloud of interest inside the object that are located at the three-dimensional position calculated by the three-dimensional position calculation unit, and generating an inverse composite aperture radar image which is an image having the image values ​​of the point cloud of interest inside the object.

9. A hazardous materials detection system comprising: a depth camera for observing the surface of a subject; a radar for receiving reflected waves from the subject; an observation data acquisition unit for acquiring observation data of the surface from the depth camera; a reception signal acquisition unit for acquiring a received signal of the reflected waves from the radar; a three-dimensional position calculation unit for calculating the three-dimensional position of a group of points of interest on the surface of the subject based on the observation data acquired by the observation data acquisition unit, and for calculating the three-dimensional position of a group of points of interest inside the subject based on the three-dimensional position of the group of points of interest on the surface; an image generation unit for calculating the image values ​​of the group of points of interest inside the subject that are located at the three-dimensional position calculated by the three-dimensional position calculation unit using the received signal acquired by the reception signal acquisition unit, and for generating an inverse synthetic aperture radar image which is an image having the image values ​​of the group of points of interest inside the subject; and a hazardous materials detection unit for performing hazardous materials detection processing based on the inverse synthetic aperture radar image generated by the image generation unit.

Images