Target detection device, target detection method, and program

Abstract

To provide a target detection apparatus capable of reliably detecting a target regardless of a background.SOLUTION: A target detection apparatus of an embodiment is a target detection apparatus capable of being mounted on a flying object. This target detection apparatus comprises an information acquisition unit, an imaging range calculation unit, an imaging apparatus, an estimated image generation unit, a feature map generation unit, and a determination unit. The information acquisition unit acquires position information and posture angle information of the flying object in flight. The imaging range calculation unit calculates an imaging range based on the position information and the posture angle information. The imaging apparatus captures the imaging range and sequentially generates frame images. The estimated image generation unit generates a predicted frame image by predicting the next frame image based on the displacement between the imaging range and the frame image. The feature map generation unit calculates the difference between the latest frame image and the predicted frame image and cancels a background component from the latest frame image. The determination unit determines a target from the frame image from which the background component has been canceled.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] An embodiment of the present invention relates to a target detection device, a target detection method, and a program. [Background technology]

[0002] Guided flying objects that fly toward a target after being launched from the ground are known. Some of these flying objects are equipped with a target detection device that can detect an approaching target while the flying object itself is in flight. The target detection device detects the target by processing image data acquired by an imaging device such as an infrared camera, and extracting features such as brightness value and shape. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 5713939 Summary of the Invention [Problem to be solved by the invention]

[0004] When a target approaches from a low altitude, a situation may occur in which the target is overlooked from a higher altitude than the target. If an attempt is made to capture an image of the target from a flying object in this situation, the complex background caused by the ground surface will take up most of the captured image. In such cases, the background captured will have a higher brightness value than the target, or a background similar in size and shape to the target, increasing the possibility of erroneously detecting the background as the target. SUMMARY OF THE DISCLOSURE An object of the present invention is to provide a target detection device, a target detection method, and a program that are capable of reliably detecting a target regardless of the background. [Means for solving the problem]

[0005] A target detection device according to an embodiment is a target detection device that can be mounted on a flying object. The target detection device includes an information acquisition unit, an imaging range calculation unit, an imaging device, an estimated image generation unit, a feature map generation unit, and a determination unit. The information acquisition unit acquires position information and attitude angle information of the flying object during flight. The imaging range calculation unit calculates the imaging range based on the position information and attitude angle information. The imaging device captures the imaging range and sequentially generates frame images. The estimated image generation unit predicts the next frame image based on the amount of displacement between the imaging range and the frame image to generate a predicted frame image. The feature map generation unit calculates the difference between the latest frame image and the predicted frame image, and cancels background components from the latest frame image. The determination unit determines a target from the frame image from which the background components have been canceled. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a functional block diagram showing an example of a target detection system according to an embodiment. [Diagram 2] FIG. 2 is a functional block diagram showing an example of the processor 32 shown in FIG. [Diagram 3] FIG. 3 is a diagram showing an example of a region calculated by the imaging range calculation unit 32b. [Figure 4] FIG. 4 is a diagram for explaining generation of a predicted frame image. [Diagram 5] FIG. 5 is a diagram for explaining the process of the feature map generating unit 32d. [Figure 6] FIG. 6 is a flowchart showing an example of a processing procedure of the target detection device 30. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, embodiments will be described with reference to the accompanying drawings. Fig. 1 is a functional block diagram showing an example of a target detection system according to an embodiment. The target detection system 1 shown in Fig. 1 can be mounted on, for example, a guided flying object. The target detection system 1 includes an imaging device 10, an inertial device 20, and a target detection device 30. The target detection system 1 detects targets by applying image processing to image data acquired by the imaging device 10. In the image processing, the imaging range of the imaging device 10 is calculated from position information and attitude angle information of the target detection system 1 acquired by the inertial device 20, and the position coordinates of this imaging range on the earth's surface are used.

[0008] The imaging device 10 outputs captured image data to the target detection device 30. The imaging device 10 is, for example, a camera that captures images of a scene including a target in a time-series manner. The imaging device 10 may be, for example, an infrared camera equipped with an imaging element having sensitivity in the infrared region. Alternatively, the imaging device 10 may be a camera equipped with an imaging element having sensitivity in the visible light region.

[0009] The inertial device 20 is a sensor that acquires the position and attitude angle of the flying object on which the target detection device 30 is mounted. The inertial device 20 may be, for example, an inertial sensor that calculates the position and rotation from an accelerometer and an angular velocity meter. The inertial device 20 may be, for example, a GPS (Global Positioning System) that calculates the position using a positioning signal from a satellite.

[0010] The target detection device 30 includes an input device 31, a processor 32, a memory 33, a storage device 34, and an output device 35. In other words, the target detection device 30 is a computer.

[0011] The input device 31 is an interface device for inputting image data sequentially acquired by the imaging device 10 to the processor 32. The input device 31 is an interface device for inputting the position and attitude angles of the target detection system 1 sequentially acquired by the inertial device 20 to the processor 32.

[0012] The processor 32 executes a target detection program 34a stored in the storage device to detect a target in the captured image from the captured image, position, and attitude angle information acquired via the input device 31. The processor 32 is, for example, a CPU (Central Processing Unit). The processor 32 may be an MPU, a GPU, an ASIC, an FPGA, or the like. The processor 32 may be a single CPU or the like, or multiple CPUs or the like.

[0013] The memory 33 includes a read only memory (ROM) and a random access memory (RAM). The ROM stores a startup program for the target detection device 30 and various parameters necessary for the operation of the target detection device 30. The RAM is used as a working memory for various processes by the processor 32. For example, the RAM is used to store multiple captured images acquired via the input device 31.

[0014] The storage device 34 is a storage device that stores a target detection program 34a. The storage device 34 may be a flash memory or the like.

[0015] The output device 35 is an interface device for outputting information on the target detected by the processor 32 to the outside.

[0016] Fig. 2 is a functional block diagram showing an example of the processor 32 in Fig. 1. The processor 32 includes, as its processing functions, an imaging control unit 32a, an imaging range calculation unit 32b, an estimated image generation unit 32c, a feature map generation unit 32d, a target determination unit 32e, and an output control unit 32f. For example, the processor 32 executes a target detection program 34a to function as the imaging control unit 32a, the imaging range calculation unit 32b, the estimated image generation unit 32c, the feature map generation unit 32d, the target determination unit 32e, and the output control unit 32f.

[0017] The imaging control unit 32a controls the operation of the imaging device 10. The imaging control unit 32a controls the imaging device 10, for example, the directivity angle of the imaging direction.

[0018] The imaging range calculation unit 32b sequentially calculates, for each pixel, the coordinates of the ground surface in the reference coordinate system imaged by the imaging device 10. The imaging range calculation unit 32b calculates the amount of displacement of the imaging range for each pixel between time series.

[0019] Fig. 3 is a diagram showing an example of an area calculated by the imaging range calculation unit 32b. In Fig. 3, a reference coordinate system 501 is an inertial coordinate system, and its origin is set to, for example, the launch point of the flying object or the position 502a of the flying object in a previous frame. The line of sight direction and imaging range at the time of the previous frame are indicated by dotted lines, and the line of sight direction and imaging range at the time of the latest frame are indicated by solid lines.

[0020] Based on the directional angle of the imaging device 10 instructed by the imaging control unit 32a to the imaging device 10 and the position and attitude angle of the target detection system 1 acquired by the inertial device 20, the imaging range calculation unit 32b calculates the line of sight direction 503a of the past frame and the line of sight direction 503b of the latest frame in the reference coordinate system 501.

[0021] The imaging range calculation unit 32b calculates a past frame imaging range 504a and a latest frame imaging range 504b in the reference coordinate system 501 based on the calculated line of sight direction and position information of the target detection system 1 acquired by the inertial device 20. The imaging displacement amount 505 is calculated as a difference for each pixel between the past frame imaging range 504a and the latest frame imaging range 504b.

[0022] Returning to FIG. 2, the explanation will be continued. The estimated image generating unit 32c generates an image (predicted frame image) that is to be captured in the latest frame by converting the image captured in the past frame based on the imaging displacement amount 505. That is, the estimated image generating unit 32c generates a predicted frame image by predicting the next frame image based on the calculated imaging range and the displacement amount between the past frame image.

[0023] Fig. 4 is a diagram for explaining generation of a predicted frame image. Fig. 4 shows a method for generating an image to be captured in the latest frame when the amount of imaging displacement 505 is a translation of the captured image in the vertical and horizontal directions. 4, the latest frame imaging range 602 is an area shifted rightward and downward by the amount of imaging displacement 505 with respect to the past frame image 601. Based on this, the estimated image generating unit 32c translates (shifts) each of the pixel coordinates of the past frame image 601 by the amount of imaging displacement 505 to generate a predicted frame image 605 of the latest frame. At this time, an invalid area 604a where no destination exists in the past frame image 601 and an invalid area 604b where no source exists in the predicted frame image 605 are generated. Since no image exists in these invalid areas 604a and 604b, the effective field of view is narrowed by the amount of imaging displacement 505.

[0024] 4, the case where the imaging displacement amount 505 of all pixels is translated the same amount in the vertical and horizontal directions of the image is illustrated. On the other hand, in an actual system, the displacement amount in the direction in which the imaging background expands, the rotational displacement amount, and the displacement amount for each pixel may differ. In this case, homography transformation is applied.

[0025] That is, a homography matrix is ​​found from a past frame image 601 and a latest frame imaging range 602, and a homography transformation is performed on an effective area 603a of the past frame image 601, thereby generating a predicted frame image 605.

[0026] Returning to FIG. 2, the explanation will be continued. The feature map generating unit 32d cancels the background component from the captured image by calculating the difference between the captured image of the latest frame acquired from the imaging device 10 and the predicted frame image 605 generated by the estimated image generating unit 32c. This emphasizes the target component to be detected.

[0027] Fig. 5 is a diagram for explaining the processing of the feature map generating unit 32d. In Fig. 5, first, a difference map 702 between the latest frame image 701 and the predicted frame image 605 is calculated.

[0028] The amount of displacement of the predicted frame image 605 is calculated with the entire imaging range as the ground surface background. On the other hand, since the flying target flies at an altitude higher than the ground surface, the motion parallax between the background component and the target component in the image captured by the imaging device 10 is different. Therefore, the background component can be canceled by assuming the ground surface in the image and taking the difference between the predicted frame image 605, which has been aligned with the background, and the latest frame image 701. Furthermore, the target component with a different motion parallax remains without being canceled.

[0029] At this time, the target components remaining in the difference map 702 contain positive component pixels and negative component pixels, and so are divided into a positive component map 703 and a negative component map 704. The target imaged in the positive component map 703 is the target component of the latest frame image 701, so the position at which the target is imaged is the latest target position. On the other hand, the target imaged in the negative component map 704 is a position transformed by the amount of displacement of the background component from the previous frame image 601, so does not indicate the latest target position.

[0030] Therefore, the estimated image generating unit 32c inversely transforms the negative component map 704 to generate a negative component map (inverse transformation). Specifically, a transformation inverse to the transformation for generating the predicted frame image 605 from the past frame image 601 is performed on the negative component map 704, a process is performed to return the target imaging position to the position of the past frame image 601, and the absolute value is taken to generate the negative component map (inverse transformation). Finally, the background component is cancelled by taking the sum of the positive component map 703 and the negative component map (inverse transformation) 705. As a result, a feature map 706 in which the target pixel is emphasized is generated.

[0031] That is, the feature map generating unit 32d divides the difference map 702 into a positive component map 703 in which positive component pixels are extracted, and a negative component map 704 in which negative component pixels are extracted and the absolute values ​​of the pixel values ​​are mapped. The feature map generating unit 32d also inversely transforms the negative component map 704 to return the target pixel coordinates. Furthermore, the feature map generating unit 32d calculates the sum of the negative component map (inverse transformation) 705 and the positive component map 703 to generate a feature map in which the intensity of the target component is emphasized.

[0032] Returning to FIG. 2, the explanation will be continued. The target determination unit 32e acquires a feature map in the latest frame from the feature map generation unit 32d. Then, the target determination unit 32e determines the coordinates of the final target in the image from the feature map. For example, the pixel with the maximum image luminance value is set as the target. In addition, the feature map may be subjected to known image processing such as binarization processing and edge detection, and the target coordinates may be determined based on the results.

[0033] The output control unit 32f calculates an error angle from the pixel coordinates of the target determined by the target determination unit 32e. The output control unit 32f performs control to output the error angle information to the outside. For example, the output control unit 32f outputs the target error angle information to the control system (propulsion system, steering mechanism, etc.) of the guided missile via the output device 35.

[0034] 6 is a flowchart showing an example of a processing procedure of the target detection device 30. The processing of FIG. In step S101 , the processor 32 acquires the imaging direction from the imaging device 10 , and acquires the position and attitude angle of the target detection system 1 from the inertial device 20 .

[0035] In step S102, the processor 32 calculates the imaging range from the acquired imaging direction, position, and attitude angle. In step S103 , the processor 32 acquires the captured image of the latest frame from the imaging device 10 , and acquires the captured image of the previous frame from the memory 33 .

[0036] In step S104, the processor 32 generates an image (predicted frame image 605) that is estimated to be captured in the latest frame, based on the calculated imaging range and the captured image of the acquired past frame. In step S105, the processor 32 cancels the background from the generated predicted frame image 605 and the captured image of the latest frame acquired, and generates a feature map in which the target is emphasized.

[0037] In step S106, the processor 32 determines a target pixel coordinate from the generated feature map, for example, determining the pixel having the highest value as the target pixel coordinate. In step S107, the processor 32 calculates an error angle from the determined target pixel coordinates, and outputs the error angle via the output device 35 to the outside.

[0038] As described above, according to the embodiment, it is possible to generate a feature map in which the background is cancelled from the captured image of the latest frame and the target is emphasized. This makes it possible to detect the target with high accuracy even if there is a complex background caused by the ground surface.

[0039] Therefore, according to the embodiment, it is possible to accurately detect a target even when a complex background caused by the ground surface is captured. In other words, it is possible to provide a target detection device, a target detection method, and a program that can reliably detect a target regardless of the background.

[0040] The present invention is not limited to the above embodiment. For example, at least one of the imaging control unit 32a, the imaging range calculation unit 32b, the estimated image generation unit 32c, the feature map generation unit 32d, the target determination unit 32e, and the output control unit 32f shown in Fig. 2 may be configured by dedicated hardware or the like.

[0041] Although the embodiment has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. This novel embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. This embodiment and its modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0042] 1...target detection system, 10...imaging device, 20...inertial device, 30...target detection device, 31...input device, 32...processor, 32a...imaging control unit, 32b...imaging range calculation unit, 32c...estimated image generation unit, 32d...feature map generation unit, 32e...target determination unit, 32f...output control unit, 33...memory, 34...storage device, 34a...target detection program, 35...output device.

Claims

1. A target detection device that can be mounted on an aircraft, an information acquisition unit that acquires position information and attitude angle information of the flying object during flight; an imaging range calculation unit that calculates an imaging range based on the position information and the attitude angle information; an imaging device that captures the imaging range and sequentially generates frame images; an estimated image generating unit that predicts a next frame image based on an amount of displacement between the imaging range and the frame image, and generates a predicted frame image; a feature map generating unit that calculates a difference between a latest frame image and the predicted frame image and cancels background components from the latest frame image; a determination unit for determining a target from the frame image in which the background component has been cancelled, The feature map generation unit Calculating a difference between the latest frame image and the predicted frame image to generate a difference map in which background components are cancelled from the latest frame image and target components remain; Dividing the difference map into a positive component map in which pixels of positive components are extracted, and a negative component map in which pixels of negative components are extracted and absolute values ​​of pixel values ​​are mapped, inversely transforming the negative component map to return the target pixel coordinates; A target detection device that calculates the sum of the inverse transformed negative component map and the positive component map to generate a feature map in which the intensity of the target component is emphasized.

2. 2. The target detection device according to claim 1, wherein the imaging range calculation section assumes that the frame image is the ground surface and calculates imaging position coordinates in a reference coordinate system for each pixel of the frame image based on the position information and attitude angle information.

3. 3. The target detection device according to claim 2, wherein the imaging range calculation section calculates the imaging position coordinates in time series, and calculates a displacement amount of image-forming pixel coordinates between any time series in the imaging position coordinates.

4. 3. The target detection device according to claim 2, wherein the estimated image generating section generates the predicted frame image by performing image processing and conversion on an image frame previously captured by the imaging device based on the amount of displacement of the imaging position coordinates.

5. A method for detecting a target using a target detection device that can be mounted on a flying object, comprising: The target detection device acquires position information and attitude angle information of the flying object during flight; a target detection device calculating an imaging range based on the position information and the attitude angle information; the target detection device captures the imaging range with an imaging device to sequentially generate frame images; a target detection device predicting a next frame image based on an amount of displacement between the imaging range and the frame image to generate a predicted frame image; A target detection device calculates a difference between a latest frame image and the predicted frame image, and cancels a background component from the latest frame image; A target detection device determines a target from the frame image in which the background component has been cancelled; a target detection device calculating a difference between the latest frame image and the predicted frame image, and generating a difference map in which a background component is cancelled from the latest frame image and a target component remains; A target detection device divides the difference map into a positive component map in which pixels of positive components are extracted, and a negative component map in which pixels of negative components are extracted and absolute values ​​of pixel values ​​are mapped; a target detection device inversely transforming the negative component map to return pixel coordinates of the target; The target detection method includes a target detection device calculating a sum of the inverse transformed negative component map and the positive component map to generate a feature map in which the intensity of the target component is emphasized.

6. A program executable by a computer mountable on a flying object, An instruction to cause the computer to acquire position information and attitude angle information of the flying object during flight; an instruction to cause the computer to calculate an imaging range based on the position information and the attitude angle information; An instruction to the computer to cause an imaging device to capture the imaging range and sequentially generate frame images; an instruction to cause the computer to predict a next frame image based on an amount of displacement between the imaging range and the frame image, and generate a predicted frame image; instructions to the computer to calculate a difference between a latest frame image and the predicted frame image to cancel background components from the latest frame image; instructions for causing the computer to determine a target from the frame image with the background component cancelled; instructions to cause the computer to calculate a difference between the latest frame image and the predicted frame image to generate a difference map from the latest frame image in which background components are cancelled and target components remain; An instruction to cause the computer to divide the difference map into a positive component map in which pixels of positive components are extracted, and a negative component map in which pixels of negative components are extracted and absolute values ​​of pixel values ​​are mapped; instructions to cause the computer to inverse transform the negative component map to return target pixel coordinates; and instructions for causing the computer to calculate a sum of the inverse transformed negative component map and the positive component map to generate a feature map in which the intensity of the target component is enhanced.

Images