Tracking device and tracking method

Abstract

To provide a tracking device capable of suppressing a calculation load even when detecting and tracking a plurality of targets simultaneously. [Solution] The tracking device 10A comprises an imaging processing unit 1 that captures an image of a target that is a candidate for tracking, a target detection processing unit 4 that detects a bounding box, which is the area where the target exists from the image, and a target position, which is the position of the target corresponding to the center point of the bounding box, and a tracking processing unit 6A that calculates an estimated position of the tracked target by state estimation processing based on the target position and a previous target position, which is a target position that precedes the target position in the chronological order, and outputs the calculated estimated position as the estimated position of the tracked target, and the target detection processing unit 4 cuts out multiple cut-out areas from the image and detects a bounding box and target position for each cut-out area.

Description

[Technical Field]

[0001] The present disclosure relates to a tracking device and a tracking method for tracking a target. [Background technology]

[0002] 2. Description of the Related Art Techniques have been proposed for tracking targets such as ships and vehicles based on images captured by a camera.

[0003] The image processing device described in Patent Document 1 tracks a target using a template that serves as a reference image for tracking the target. This image processing device performs template matching by moving the template within a search range, and tracks the target by estimating the position of the part with the highest correlation value as the tracking position. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5772572 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology of Patent Document 1 has a problem in that the calculation load increases when multiple targets are detected and tracked simultaneously.

[0006] The present disclosure has been made in consideration of the above, and aims to provide a tracking device that can reduce the calculation load even when detecting and tracking multiple targets simultaneously. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the object, a tracking device disclosed herein includes an imaging processing unit that captures an image of a target that is a candidate for tracking, and a target detection processing unit that detects from the image a bounding box that is an area where the target exists and a target position that is the position of the target corresponding to the center point of the bounding box.The tracking device disclosed herein also includes a tracking processing unit that calculates an estimated position of the tracking target by a state estimation process based on the target position and a previous target position that is a target position that precedes the target position in the chronological order, and outputs the calculated estimated position as the estimated position of the tracking target.The target detection processing unit cuts out multiple cutout regions from the image and detects a bounding box and a target position for each cutout region. [Effects of the Invention]

[0008] The tracking device according to the present disclosure has the advantage of being able to reduce the calculation load even when detecting and tracking multiple targets simultaneously. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating a configuration of a tracking device according to a first embodiment; [Figure 2] 1 is a flowchart showing a processing procedure of a process executed by a tracking device according to a first embodiment; [Figure 3] FIG. 1 is a diagram for explaining a process executed by an image scale conversion unit of a tracking device according to a first embodiment; [Figure 4] FIG. 2 is a diagram illustrating a process executed by an image scale conversion unit of the tracking device according to the first embodiment; [Figure 5] FIG. 1 is a diagram for explaining a process executed by a target detection processing unit of a tracking device according to a first embodiment; [Figure 6] FIG. 2 is a flowchart for explaining a process executed by a target detection processing unit of a tracking device according to a first embodiment; [Figure 7] FIG. 3 is a flowchart for explaining a process executed by a target detection processing unit of a tracking device according to a first embodiment; [Figure 8] FIG. 10 is a diagram illustrating a configuration of a tracking device according to a second embodiment. [Figure 9] 10 is a flowchart showing a processing procedure of a process executed by a tracking device according to a second embodiment. [Figure 10] FIG. 1 is a diagram illustrating a process executed by a feature extraction processing unit of a tracking device according to a second embodiment. [Figure 11] FIG. 2 is a diagram illustrating a process executed by a feature extraction processing unit of a tracking device according to a second embodiment. [Figure 12] FIG. 10 is a diagram illustrating a configuration example of a processing circuit provided in the tracking device according to the first and second embodiments when the processing circuit is realized by a processor and a memory. [Figure 13] FIG. 10 is a diagram illustrating a configuration example of a processing circuit included in a tracking device according to first and second embodiments, in the case where the processing circuit is configured with dedicated hardware. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, a tracking device and a tracking method according to an embodiment of the present disclosure will be described in detail with reference to the accompanying drawings.

[0011] Embodiment 1 1 is a diagram illustrating the configuration of a tracking device according to a first embodiment. Tracking device 10A is a device that tracks a tracking target such as a ship, a vehicle, etc. In the following, a case where tracking device 10A tracks a ship on the water will be described, but tracking device 10A may also track a tracking target other than a ship.

[0012] The tracking device 10A includes an imaging processing unit 1, a boundary detection unit 2, an image scale conversion unit 3, a target detection processing unit 4, and a tracking processing unit 6A. Note that the tracking device 10A does not necessarily have to include the boundary detection unit 2 and the image scale conversion unit 3.

[0013] The imaging processing unit 1 captures time-series images including one or more targets to be tracked. For example, the imaging processing unit 1 collects visible light or infrared light from the targets using an imaging lens, converts the collected visible light or infrared light into an electrical signal using an imaging element, and converts the electrical signal into a digital signal using an analog-to-digital converter, thereby obtaining an image including the targets. Note that the imaging lens and imaging element may be external devices separate from the tracking device 10A. In this case, the imaging processing unit 1 receives the electrical signal from the imaging element and obtains an image including the targets from the electrical signal. The following describes a case where the imaging processing unit 1 captures an image including multiple targets.

[0014] The image including the target captured by the image capturing processor 1 is a two-dimensional image. Hereinafter, an "image that may include the target" will be referred to as a "frame." The image capturing processor 1 transmits the captured frame to the boundary detection unit 2.

[0015] The boundary detection unit 2 detects the coordinates of a boundary line (for example, a horizontal line) that indicates whether or not a target exists in the frame received from the image capture processing unit 1. The boundary line detected by the boundary detection unit 2 is the boundary line between an area in the frame where a target may exist and an area where a target does not exist. Hereinafter, the coordinates of the boundary line detected by the boundary detection unit 2 will be referred to as boundary line coordinates.

[0016] If the target is a ship on the water, the area above the horizon within the frame may not contain any ships, but the area below the horizon may contain ships. The boundary detection unit 2 transmits the frame and the detected boundary coordinates to the image scale conversion unit 3 and the target detection processing unit 4.

[0017] The image scale conversion unit 3 performs image scale conversion using the frame captured by the imaging processing unit 1 and the boundary coordinates detected by the boundary detection unit 2 so that a target within a specific size (pixel size) range is present in the frame after image scale conversion. That is, the image scale conversion unit 3 performs image scale conversion so that a target within a predetermined size range (a target of a size to be detected) is included in the frame after image scale conversion. In other words, the image scale conversion unit 3 performs image scale conversion to match an area on the frame where a target of the specific size may be present. For example, the image scale conversion unit 3 performs image scale conversion so that a target larger than the specific size (a target of a desired minimum size) is included in the frame after image scale conversion. Note that the image scale conversion unit 3 may also perform image scale conversion so that a target smaller than the specific size is included in the frame after image scale conversion. The size range preset in the image scale conversion unit 3 is the size range of targets to be detected by the tracking device 10A.

[0018] The image scale conversion unit 3 calculates a conversion ratio for image scale conversion based on a preset maximum detection size, the size of a cutout area for detection, etc. The maximum detection size is the maximum size of a target to be detected. In other words, the maximum detection size is the maximum size of a target to be detected, and the tracking device 10A detects targets that are equal to or smaller than the maximum detection size. The cutout area is an area that the target detection processing unit 4 cuts out from a frame. The tracking device 10A cuts out multiple cutout areas from a frame, and detects a target for each cutout area. Details of the maximum detection size and the cutout areas will be described later. The image scale conversion unit 3 performs image scale conversion using the calculated conversion ratio. Note that the image scale conversion unit 3 may also perform image scale conversion using a preset conversion ratio.

[0019] The image scale conversion unit 3 performs image scale conversion on the frame including the boundary line by also performing image scale conversion on the boundary line coordinates detected by the boundary line detection unit 2. The image scale conversion unit 3 transmits the frame after image scale conversion and the boundary line coordinates to the target detection processing unit 4.

[0020] The target detection processing unit 4 detects targets using either the boundary line coordinates and frame (frame without image scale conversion) received from the boundary line detection unit 2 or the boundary line coordinates and frame (frame after image scale conversion) received from the image scale conversion unit 3. Based on the boundary line coordinates, the target detection processing unit 4 sets the area below the horizon as the target detection range (target detection range), and detects targets from the target detection range. The target detection processing unit 4 is preset with a setting for the direction of the area located in the target detection range. For example, if a ship is detected, the target detection processing unit 4 is set to set the area below the horizon (boundary line) as the target detection range, and if an airplane is detected, the target detection processing unit 4 is set to set the area above the horizon or skyline as the target detection range. The target detection processing unit 4 cuts out clipping areas from the target detection range of the frame and detects targets for each clipping area. The targets detected by the target detection processing unit 4 are the tracking targets.

[0021] The target detection processing unit 4 cuts out multiple cut-out regions from the frame so that adjacent cut-out regions partially overlap each other. The size of the cut-out region is the size of the region that the target detection processing unit 4 cuts out from the frame at specific intervals. The size of the cut-out region used by the target detection processing unit 4 is set in advance in the target detection processing unit 4. The size (vertical dimension and horizontal dimension) of the cut-out region is larger than the expected size (vertical dimension and horizontal dimension) of the target.

[0022] The target detection processing unit 4 detects, for each cutout area, a target area (bounding box) in which a target to be tracked exists, and the center point of the bounding box. That is, the target detection processing unit 4 detects a bounding box and the center point of the bounding box for each cutout area based on either a frame without image scale conversion or a frame with image scale conversion. In this way, the target detection processing unit 4 detects a target for each cutout area according to the size of the target, and detects the target's bounding box and center point.

[0023] A bounding box is an area (e.g., a rectangular area) that contains a target. The target detection processing unit 4 detects a bounding box and a center point for each target. The bounding box detected by the target detection processing unit 4 may overlap with other bounding boxes in the frame.

[0024] The position of the target (target position) is the center point of the bounding box detected by the target detection processing unit 4. The target detection processing unit 4 transmits the center point of the bounding box (target position) to the tracking processing unit 6A.

[0025] The tracking processing unit 6A calculates an estimate of the position of the tracking target (estimated position of the tracking target) by state estimation processing based on the target position received from the target detection processing unit 4 and the target position (previous target position) of the previous frame (e.g., the frame immediately before) which is a frame before the frame containing this target position (current frame).

[0026] Tracking processing unit 6A calculates the target position of the first bounding box in the (X+2)th frame (estimated position of the tracking target) based on the target position of the first bounding box in the Xth (X is a natural number) frame and the target position of the first bounding box in the (X+1)th frame. Here, the first bounding box in the Xth frame, the first bounding box in the (X+1)th frame, and the first bounding box in the (X+2)th frame are all bounding boxes of the same target.

[0027] Tracking processing unit 6A transmits the estimated position of the tracking target to an external device such as a display device. In this way, tracking device 10A of the first embodiment detects a tracking target (target) for each clipped region and calculates an estimated position for the detected tracking target.

[0028] Next, an operation example of the tracking device 10A will be described. Fig. 2 is a flowchart showing a processing procedure of processing executed by the tracking device according to the first embodiment. The imaging processing unit 1 of the tracking device 10A captures frames (step S10). As a result, the imaging processing unit 1 obtains frames in time series.

[0029] The boundary detection unit 2 detects the boundary coordinates of a boundary line indicating whether or not a target may exist in the frame captured by the image capture processing unit 1 (step S20). When detecting the boundary coordinates, the boundary detection unit 2 may use information such as the position where the image capture processing unit 1 is attached.

[0030] The image scale conversion unit 3 performs image scale conversion using the frame captured by the image capture processing unit 1 and the boundary line coordinates detected by the boundary line detection unit 2 so that a target within a specific size range is present on the frame after the image scale conversion (step S30). The image scale conversion unit 3 may use a preset maximum detection size used for target detection when performing this image scale conversion. Note that the tracking device 10A does not need to perform image scale conversion of the frame. In this case, the tracking device 10A may not need to include the image scale conversion unit 3. Furthermore, the tracking device 10A may not need to detect boundary line coordinates. In this case, the tracking device 10A may not need to include the boundary line detection unit 2.

[0031] The target detection processing unit 4 cuts out a plurality of cut-out regions from the frame, and detects a target (tracking target) for each cut-out region.

[0032] 3 is a first diagram for explaining the processing executed by the image scale conversion unit of the tracking device according to the first embodiment. FIG. 3 shows one frame F1 that includes multiple ships 21 and a horizon 30, which is a boundary line detected by the boundary detection unit 2. Ships located below the horizon 30 in this frame F1 tend to appear larger the closer they are to the horizon 30. The image scale conversion unit 3 performs image scale conversion to make these large targets larger so that they can be detected.

[0033] Fig. 4 is a second diagram for explaining the processing executed by the image scale conversion unit of the tracking device according to the first embodiment. Fig. 4 shows a state in which the image scale conversion unit 3 performs image scale conversion to produce a frame F2 that is smaller than the frame F1 shown in Fig. 3.

[0034] The image scale conversion unit 3 calculates a conversion ratio for image scale conversion based on a preset maximum detection size, the size of a cutout region for detection, and the like.

[0035] The maximum detection size is the maximum size of the bounding box when detecting the bounding box of the target ship 21. The tracking device 10A detects targets up to the size set in the maximum detection size. The maximum detection size is set in advance in the image scale conversion unit 3 according to the size of the target expected after image scale conversion. In other words, the maximum detection size (vertical and horizontal dimensions) is set in advance in the image scale conversion unit 3 so that it is larger than the size (vertical and horizontal dimensions) of all ships 21 expected after image scale conversion. The image scale conversion unit 3 calculates the conversion ratio for image scale conversion based on the preset maximum detection size and the size of the cut-out area larger than the maximum detection size.

[0036] For example, if the size of the cutout area is small, the size of the ship 21 located at the bottom of the frame may be larger than the maximum detection size and the size of the cutout area. The image scale conversion unit 3 calculates a conversion ratio for image scale conversion so that the size of the ship 21 is smaller than the maximum detection size and the size of the cutout area, even if the ship 21 is located at the bottom of the frame. The image scale conversion unit 3 performs image scale conversion based on the conversion ratio for image scale conversion. As a result, the target included in the frame after image scale conversion becomes smaller than the maximum detection size and the size of the cutout area.

[0037] The image scale conversion unit 3 may perform image scale conversion using a preset conversion ratio for image scale conversion. In this case, the conversion ratio for image scale conversion is preset in the image scale conversion unit 3 based on the size of the ships 21 expected before the image scale conversion so that the size of all ships 21 after the image scale conversion is smaller than the maximum detection size and the cutout area.

[0038] Also, here, the image scale conversion unit 3 reduces the size of the frame to detect the area below the horizon 30, which is the boundary line, but the image scale conversion unit 3 may also reduce the size of the frame to detect the area above the boundary line. For example, if the target is a flying object such as an airplane, the image scale conversion unit 3 may reduce the size of the frame to detect the area above the horizon 30 or the horizon line.

[0039] The image scale conversion unit 3 may also enlarge the frame to detect an area above or below the boundary line. That is, the image scale conversion unit 3 may enlarge or reduce the frame to detect one of the areas divided by the boundary line, depending on the target to be detected.

[0040] The target detection processing unit 4 detects the target, the bounding box, and the center point of the bounding box using either the boundary line coordinates and frame received from the boundary line detection unit 2 or the boundary line coordinates and frame received from the image scale conversion unit 3.

[0041] The target detection processing unit 4 extracts crop regions from a frame, detects targets from each crop region, and sets bounding boxes for the detected targets. For example, the target detection processing unit 4 uses common object recognition technologies such as SSD (Single Shot Multibox Detector) or YOLO (You Only Look Once) v (version) 2 to detect targets and the center points of bounding boxes. The following describes an example of the operation of the target detection processing unit 4 when detecting targets using YOLO v2.

[0042] The target detection processing unit 4 detects the center point of the bounding box (step S40) based on the frame including the boundary line coordinates that have been image scaled by the image scale conversion unit 3. Note that the target detection processing unit 4 may also detect the center point of the bounding box based on the boundary line coordinates and frame (a frame that has not been image scaled) received from the boundary line detection unit 2.

[0043] Fig. 5 is a first diagram for explaining the processing executed by the target detection processing unit of the tracking device according to the first embodiment. Fig. 5 explains the processing executed by the target detection processing unit 4 to detect a target using a frame F1 on which image scale conversion has not been performed.

[0044] FIG. 5 shows an example of processing in which the target detection processing unit 4 detects targets near the boundary line using the boundary line coordinates detected by the boundary line detection unit 2 and a frame F1 including these boundary line coordinates.

[0045] The target detection processing unit 4 cuts out cutout regions at regular intervals along the horizontal line 30, which is a boundary line, for example. The target detection processing unit 4 may also cut out cutout regions at regular intervals along the edge of the frame (the lower side of the frame F1), for example. The target detection processing unit 4 may also perform both the process of cutting out cutout regions at regular intervals along the horizontal line 30 and the process of cutting out cutout regions at regular intervals along the edge of the frame F1.

[0046] The target detection processing unit 4 determines, for example, the range where adjacent clipping regions overlap (hereinafter referred to as clipping overlap region) based on a preset maximum detection size. The target detection processing unit 4 determines the size of the clipping overlap region so that the size of the clipping overlap region is larger than the maximum detection size of the target. In other words, the target detection processing unit 4 determines the size of the clipping overlap region so that the clipping overlap region can encompass the maximum size target.

[0047] Note that Fig. 5 shows a case where the size of the cut-out overlapping region is smaller than the maximum detection size of the target. Fig. 5 also shows a case where the target detection processing unit 4 cuts out cut-out regions 51A to 51D, indicated by dashed lines, from frame F1.

[0048] The target detection processing unit 4 detects a target in each of the cutout regions 51 A to 51 D. The target detection processing unit 4 detects a bounding box and the center point of the bounding box for the detected target.

[0049] Fig. 6 is a second diagram for explaining the processing executed by the target detection processing unit of the tracking device according to the first embodiment. Fig. 6 explains the processing executed by the target detection processing unit 4 to detect a target using the frame F2 on which image scale conversion has been performed.

[0050] 6 shows an example of processing in which the target detection processing unit 4 detects a target near the edge of frame F2 (below frame F2) using frame F2 that has been image scale converted by the image scale conversion unit 3. The target detection processing unit 4 determines the size of the cut-out overlap region by processing similar to the processing described in FIG.

[0051] The target detection processing unit 4 cuts out cutout regions at regular intervals along the edge of the frame (the lower side of frame F2), for example. The target detection processing unit 4 may also cut out cutout regions at regular intervals along the horizontal line 30, which is a boundary line, for example. The target detection processing unit 4 may also perform both the process of cutting out cutout regions at regular intervals along the horizontal line 30 and the process of cutting out cutout regions at regular intervals along the edge of frame F2.

[0052] FIG. 6 shows a case where the target detection processing unit 4 cuts out cut-out regions 51E to 51G indicated by dashed lines from the frame F2.

[0053] The target detection processing unit 4 detects a target in each of the cutout regions 51E to 51G. The target detection processing unit 4 detects a bounding box and the center point of the bounding box for the detected target.

[0054] Fig. 7 is a third diagram for explaining the processing executed by the target detection processing unit of the tracking device according to the first embodiment. Fig. 7 shows a bounding box 71 set by the target detection processing unit 4 for frame F1 in Fig. 5 for which image scale conversion has not been performed, and a center point 81 of the bounding box 71.

[0055] The target detection processing unit 4 may set a bounding box and a center point of the bounding box for frame F2 that has undergone the image scale conversion shown in Fig. 6. In this case, the target detection processing unit 4 may calculate the bounding box 71 and center point 81 of the bounding box 71 shown in Fig. 7 by dividing the set bounding box and center point of the bounding box by the conversion ratio of the image scale conversion (by multiplying by the reciprocal of the conversion ratio).

[0056] The tracking processing unit 6A calculates an estimated position of the tracking target (step S50) by performing a state estimation process based on the transition of the target position for the target (tracking target) detected by the target detection processing unit 4. Specifically, the tracking processing unit 6A calculates an estimated position of the tracking target (target position in the frame immediately following the current frame) by performing a state estimation process based on the target position (center point of the bounding box) in the current frame (hereinafter referred to as the current frame) detected by the target detection processing unit 4 and the target position (previous target position) in the previous frame before the current frame.

[0057] The tracking processing unit 6A may use the center point of the bounding box and an estimated value of the speed of the center point in calculating the estimated position of the tracking target. In this case, the tracking processing unit 6A estimates the speed of the center point of the bounding box based on the distance traveled by the center point of the bounding box and the time required for the movement. The tracking processing unit 6A estimates the estimated position of the tracking target (next target position) based on the center point of the bounding box and the estimated value of the speed of the center point.

[0058] Furthermore, the tracking processing unit 6A may use a technique such as GNN (Global Nearest Neighbor), JPDA (Joint Probabilistic Data Association), or MHT (Multiple Hypothesis Tracking) in calculating the estimated position of the tracking target.

[0059] In this way, in the first embodiment, tracking device 10A can detect a target without the user having to specify the target, so the user does not have to specify the target in advance. Furthermore, tracking device 10A does not perform template matching, so the calculation load can be reduced even when detecting multiple targets.

[0060] Furthermore, since tracking device 10A cuts out cutout areas from a frame and detects a target from each cutout area, the number of processes to be performed until a target is detected is determined once the frame size and the cutout area size are determined. Therefore, tracking device 10A can easily estimate the processing load until a target is detected, regardless of the number of targets included in the frame, which is the imaging result. Furthermore, since tracking device 10A does not vary the processing load when detecting a target, it is possible to prevent a decrease in the processing rate for target detection and to prevent some targets from becoming undetectable.

[0061] Furthermore, since tracking device 10A can detect targets regardless of the number of targets and the arrangement of targets in an imaged frame, it is possible to reduce the delay time from image capture to tracking, and prevent a decrease in processing rate and a decrease in tracking performance.

[0062] According to the first embodiment, tracking device 10A extracts an extraction region from a frame and detects a target for each extraction region, thereby reducing fluctuations in the calculation load and the process of detecting overlapping targets, and improving the determination accuracy. As a result, tracking device 10A can suppress the calculation load and stably detect and track targets, even when detecting and tracking multiple targets simultaneously.

[0063] Embodiment 2 Next, a second embodiment will be described with reference to Fig. 8 to Fig. 11. In the second embodiment, feature points of target candidates (tracking target candidates) and feature amounts at the feature points are detected, and the feature amounts of the current frame are compared with the feature amounts of the frame n frames prior, thereby eliminating feature points other than the feature points of the target candidates.

[0064] Fig. 8 is a diagram showing the configuration of a tracking device according to embodiment 2. Among the components in Fig. 8, components that achieve the same functions as those of tracking device 10A according to embodiment 1 shown in Fig. 1 are assigned the same reference numerals, and redundant explanations will be omitted.

[0065] Similar to tracking device 10A, tracking device 10B of embodiment 2 tracks targets such as ships, vehicles, etc. Tracking device 10B includes an imaging processing unit 1, a boundary detection unit 2, an image scale conversion unit 3, a target detection processing unit 4, a tracking processing unit 6B, a detection comparison processing unit 7, and a feature extraction processing unit (feature extraction unit) 8. In embodiment 2, targets detected by target detection processing unit 4 are target candidates, and not all targets detected by target detection processing unit 4 become tracking targets.

[0066] In tracking device 10B, the image capture processing unit 1 transmits the captured frames to boundary detection unit 2 and feature extraction processing unit 8. Furthermore, target detection processing unit 4 of tracking device 10B detects a target for each clipping region and detects the coordinates of the target's bounding box. The coordinates of the bounding box are the coordinates of the vertices of the bounding box (vertex coordinates) and the coordinates of the center point of the bounding box (center point coordinates). Target detection processing unit 4 transmits the vertex coordinates and center point coordinates of the bounding box to detection comparison processing unit 7.

[0067] The feature amount extraction processing unit 8 detects a plurality of feature points from each frame received from the imaging processing unit 1. An example of a feature point is a point (vertex coordinate) at a corner position of a target included in the frame. The feature amount extraction processing unit 8 also calculates the speed of each extracted feature point. The speed of the feature points calculated by the feature amount extraction processing unit 8 will be described later.

[0068] The feature extraction processing unit 8 extracts a feature for each of the extracted feature points. Note that the feature extraction processing unit 8 may extract a feature for a frame that has been subjected to rotation processing, enlargement processing, and reduction processing.

[0069] The feature extraction processing unit 8 extracts features using, for example, the ORB (Oriented FAST and Rotated BRIEF) method or the A-KAZE (Accelerated KAZE) method. Examples of features include color and brightness. Note that the features may also include information on the speed of the feature points.

[0070] The feature extraction processing unit 8 determines whether or not a feature point and a feature amount exist in a frame n frames before the current frame (n is a natural number). If the feature extraction processing unit 8 determines that a feature point and a feature amount do not exist in the frame n frames before the current frame, it determines that the feature points included in the current frame do not contain feature points to be matched.

[0071] When it is determined that feature points and feature amounts exist in the frame n frames prior, the feature amount extraction processing unit 8 performs corresponding point matching (matching of feature points between frames) between the current frame containing the detected feature points and the frame n frames prior in which the feature points and feature amounts exist. For example, if there are 80 feature points in the frame n frames prior and 100 feature points in the current frame, the feature amount extraction processing unit 8 performs corresponding point matching between the 80 feature points and the 100 feature points.

[0072] The feature amount extraction processing unit 8 performs corresponding point matching based on the feature amount for each feature point. The feature amount extraction processing unit 8 deletes feature points that are not present in the frame n frames before from the feature points in the current frame.

[0073] Alternatively, the feature extraction processing unit 8 may perform corresponding point matching based on the feature amounts for each feature point, calculate the matching correlation based on the difference in feature amounts between two frames, and delete from the current frame any feature points for which the matching correlation is lower than a reference value. In this case, the feature extraction processing unit 8 determines that a feature point for which the difference in feature amount between two frames is equal to or less than the reference value is the target feature point. Furthermore, the feature extraction processing unit 8 determines that a feature point for which the difference in feature amount between two frames is greater than the reference value is not the target feature point and deletes it.

[0074] A feature point with low matching correlation may be a feature point that does not exist in the frame n frames prior. In other words, the feature extraction processing unit 8 may determine that, among feature points that exist in the current frame, feature points that do not exist in the frame n frames prior are feature points with low matching correlation, and delete them from the current frame. In this way, the feature extraction processing unit 8 deletes feature points other than the feature points of the target candidates from among the multiple feature points detected from the current frame based on the result of corresponding point matching.

[0075] The feature extraction processing unit 8 also estimates the optical flow between frames for corresponding feature points in two frames, the current frame and the frame n frames prior. The feature extraction processing unit 8 detects the movement of the feature points between the two frames and estimates the velocities (vectors) of the feature points corresponding to the movement of the feature points, thereby estimating the optical flow of the feature points (velocities of the feature points) between the two frames. The feature extraction processing unit 8 may also perform corresponding point matching using the positions and velocities (optical flows) of each of the extracted feature points.

[0076] The feature extraction processing unit 8 may also perform clustering processing of the extracted feature points based on the position and velocity of each of the extracted feature points. That is, the feature extraction processing unit 8 may cluster the extracted feature points for each target. In this case, the feature extraction processing unit 8 sets some of the extracted feature points as a feature point group. For example, the feature extraction processing unit 8 assigns feature points with the same positional relationship and velocity to the same class. Note that if the feature extraction processing unit 8 does not perform clustering processing of the feature points, it is not necessary to estimate the optical flow of the feature points.

[0077] The feature extraction processing unit 8 sets, for example, a group of feature points in which the difference in speed of the feature points between the current frame and the frame n frames before is equal to or less than a reference value and the amount of change in the positional relationship of the feature points is equal to or less than a reference value, into one class.

[0078] A group of feature points included in one class set in the feature extraction processing unit 8 is a group of feature points of one target. Examples of clustering processing performed by the feature extraction processing unit 8 include processing using the k-means method or the variational Bayes method. The feature extraction processing unit 8 calculates the center of gravity of each class and transmits it to the detection and comparison processing unit 7. When not performing clustering processing, the feature extraction processing unit 8 transmits the positions of the extracted feature points to the detection and comparison processing unit 7.

[0079] When the detection comparison processing unit 7 receives the position of a feature point from the feature amount extraction processing unit 8, it compares the vertex coordinates and center point coordinates of the bounding box received from the target detection processing unit 4 with the position of the feature point received from the feature amount extraction processing unit 8. Based on the comparison result, the detection comparison processing unit 7 suppresses erroneous detection of the target by deleting feature points that have been erroneously detected from the feature points received from the feature amount extraction processing unit 8. Note that the detection comparison processing unit 7 may compare the position of the feature point with either the vertex coordinates or center point coordinates of the bounding box.

[0080] The detection comparison processing unit 7 determines that a feature point whose position received from the feature amount extraction processing unit 8 is not within the range of the bounding box (the area surrounded by the vertex coordinates) is an erroneously detected feature point and deletes it. In addition, the detection comparison processing unit 7 determines that a feature point whose distance between the position received from the feature amount extraction processing unit 8 and the center point of the bounding box is greater than a reference value is an erroneously detected feature point and deletes it.

[0081] The detection comparison processing unit 7 deletes bounding boxes that have no feature points inside them due to the deletion process as erroneously detected bounding boxes. The detection comparison processing unit 7 determines a bounding box that still has feature points after the deletion process (a bounding box that was not deleted) as the target bounding box. The detection comparison processing unit 7 transmits the center point coordinates of the bounding box determined as the target to the tracking processing unit 6B.

[0082] When the detection comparison processing unit 7 receives the position of the center of gravity of a class from the feature extraction processing unit 8, it compares the vertex coordinates and center coordinates of the bounding box received from the target detection processing unit 4 with the position of the center of gravity. Based on the comparison result, the detection comparison processing unit 7 suppresses erroneous detection of targets by deleting the center of gravity of feature points that have been erroneously detected from the center of gravity received from the feature extraction processing unit 8. Note that the detection comparison processing unit 7 may compare the position of the center of gravity with either the vertex coordinates or center coordinates of the bounding box.

[0083] For example, the detection comparison processing unit 7 determines that a centroid point whose position received from the feature amount extraction processing unit 8 is outside the range of the bounding box is a centroid point of an erroneously detected feature point and deletes it. Also, for example, the detection comparison processing unit 7 determines that a feature point whose distance between the centroid point position received from the feature amount extraction processing unit 8 and the center point of the bounding box is greater than a reference value is a centroid point of an erroneously detected feature point and deletes it.

[0084] The detection comparison processing unit 7 deletes a bounding box whose center of gravity has disappeared as a result of the deletion process as an erroneously detected bounding box. The detection comparison processing unit 7 determines a bounding box whose center of gravity remains after the deletion process (a bounding box that has not been deleted) as the target bounding box. The detection comparison processing unit 7 transmits the center point coordinates of the bounding box determined as the target to the tracking processing unit 6B.

[0085] The detection comparison processing unit 7 may compare the position of the center of gravity of the bounding box with the positions of the feature points or the position of the center of gravity of the class. In this case, the target detection processing unit 4 calculates the position of the center of gravity of the bounding box in advance.

[0086] In this way, the detection comparison processing unit 7 suppresses false detection of a target by comparing at least one piece of information, namely, the vertex coordinates, center coordinates, and center of gravity coordinates of the bounding box, with the position of the target's feature point or the position of the center of gravity of the class. The detection comparison processing unit 7 deletes any bounding box determined to be a false detection from all bounding boxes in the frame, and transmits the center coordinates (target positions) of the remaining bounding boxes to the tracking processing unit 6B.

[0087] The tracking processing unit 6B of the tracking device 10B calculates an estimated position of the tracking target by a state estimation process based on the target position in the current frame received from the detection comparison processing unit 7 and the target position in the previous frame (e.g., the frame immediately before the current frame).

[0088] Tracking processing unit 6B transmits the estimated position of the tracking target to an external device such as a display device. In this way, tracking device 10B of the second embodiment tracks the tracking target after deleting any erroneously detected targets based on the feature amounts of the target.

[0089] Next, an example of the operation of the tracking device 10B will be described. Fig. 9 is a flowchart showing the processing procedure of the processing executed by the tracking device according to the second embodiment. The image capture processing unit 1, boundary detection unit 2, image scale conversion unit 3, and target detection processing unit 4 execute the processing of steps S10 to S40 described in the first embodiment. As a result, the tracking device 10B detects the center point coordinates of the bounding box. In this way, the image capture processing unit 1, boundary detection unit 2, image scale conversion unit 3, and target detection processing unit 4 execute the same processing as the tracking device 10A of the first embodiment. Furthermore, the target detection processing unit 4 detects the vertex coordinates of the bounding box.

[0090] The feature extraction processing unit 8 receives frames from the imaging processing unit 1 (step S110). The feature extraction processing unit 8 detects a plurality of feature points from each frame received from the imaging processing unit 1 (step S120).

[0091] The feature amount extraction processing unit 8 extracts feature amounts for each feature point from the feature points using, for example, the ORB method (step S130). Furthermore, the feature amount extraction processing unit 8 extracts the positions of each of the extracted feature points.

[0092] The feature extraction processing unit 8 determines whether or not there are feature points and feature amounts in a frame n frames before the current frame (step S140). The frame n frames before the current frame is a frame that occurs before the current frame and includes the multiple feature points detected by the feature extraction processing unit 8 in step S120.

[0093] If it is determined that no feature points or feature amounts exist in the frame n frames before (No in step S140), the feature amount extraction processing unit 8 determines that the feature points included in the current frame do not contain any feature points to be matched. In this case, the tracking device 10B returns to step S110. Then, the tracking device 10B executes the processes of steps S110 to S130. In this case, the feature amount extraction processing unit 8 receives the next frame from the imaging processing unit 1 (step S110). Then, the feature amount extraction processing unit 8 extracts feature points from the next frame (step S120) and extracts feature amounts of these feature points (step S130).

[0094] When it is determined that feature points and feature amounts exist in the frame n frames prior (step S140, Yes), the feature amount extraction processing unit 8 performs corresponding point matching between two frames: the current frame including the feature points detected in step S120, and the frame n frames prior in which results for the feature points and feature amounts exist (step S150).

[0095] Based on the result of the corresponding point matching, the feature amount extraction processing unit 8 deletes feature points other than the feature points of the target candidates from the multiple feature points detected in step S120 (multiple feature points detected from the current frame). Additionally, the feature amount extraction processing unit 8 may delete feature points with low matching correlation.

[0096] After performing corresponding point matching, the feature extraction processing unit 8 estimates the optical flow between frames for corresponding feature points between the current frame and the frame n frames before (step S160). The feature extraction processing unit 8 performs clustering processing on the feature points based on the positions and velocities of the extracted feature points (the optical flow estimation results). The feature extraction processing unit 8 then calculates the center of gravity of each class and sends it to the detection and comparison processing unit 7.

[0097] The feature extraction processing unit 8 may transmit the positions of the feature points to the detection and comparison processing unit 7. In this case, the feature extraction processing unit 8 does not need to execute the process of estimating the optical flow and the process of calculating the center of gravity of each class.

[0098] After estimating the optical flow, the feature extraction processing unit 8 determines whether or not there is a next frame to be processed (step S170). If the feature extraction processing unit 8 determines that there is a next frame to be processed (step S170, Yes), the tracking device 10B returns to step S110. Then, the tracking device 10B executes the processes of steps S110 to S170.

[0099] Tracking device 10B repeats the processes of steps S110 to S170 until it determines that there is no next frame to process. When it determines that there is no next frame to process (step S170, No), feature extraction processing unit 8 ends the feature extraction process.

[0100] The detection comparison processing unit 7 compares the vertex coordinates and center point coordinates of the bounding box received from the target detection processing unit 4 with the center of gravity of each class received from the feature extraction processing unit 8, and based on the comparison results, deletes any erroneously detected bounding boxes to suppress erroneous detection of targets.

[0101] Fig. 10 is a first diagram for explaining the processing executed by the feature amount extraction processing unit of the tracking device according to the second embodiment. In Fig. 10, for a frame F1 containing nine target ships, feature points 35 detected by the feature amount extraction processing unit 8 through the feature amount extraction processing are indicated by black circles. Fig. 10 shows a case where the feature amount extraction processing unit 8 has extracted the corner positions of the targets included in frame F1 as feature points 35.

[0102] Fig. 11 is a second diagram for explaining the processing executed by the feature extraction processing unit of the tracking device according to the second embodiment. Fig. 11 shows the result of clustering processing performed by the feature extraction processing unit 8 on the feature points in Fig. 10 for each target. In Fig. 11, classes C1 to C7 obtained by the clustering processing are indicated by dashed lines, and the center of gravity of each class C1 to C7 is indicated by a cross.

[0103] It should be noted that when the feature extraction processing unit 8 executes clustering processing, it is not necessarily possible to cluster all targets. Figure 11 shows a case where the feature extraction processing unit 8 sets seven classes for nine ships.

[0104] The detection comparison processing unit 7 compares the vertex coordinates and center coordinates of the target bounding box obtained by the target detection processing unit 4 with the feature point positions or class center of gravity points obtained by the feature extraction processing unit 8, and suppresses erroneous detection of targets by deleting erroneously detected feature points.

[0105] The tracking processing unit 6B of the tracking device 10B calculates an estimated position of the tracking target by a state estimation process based on the target position in the current frame received from the detection comparison processing unit 7 and the target position in the previous frame immediately preceding the current frame. The tracking processing unit 6B may also use the GNN, JPDA, or MHT technology in calculating the estimated position of the tracking target.

[0106] Next, the hardware configuration of the tracking devices 10A and 10B will be described. The tracking devices 10A and 10B are realized by a processing circuit. The processing circuit may be a processor and memory that executes a program stored in a memory, or may be dedicated hardware.

[0107] 12 is a diagram illustrating an example of the configuration of a processing circuit when the processing circuit included in the tracking device according to the first and second embodiments is realized by a processor and a memory. Since the tracking devices 10A and 10B have similar hardware configurations, the hardware configuration of the tracking device 10A will be described here.

[0108] The processing circuit 90 shown in FIG. 12 includes a processor 91 and a memory 92. When the processing circuit 90 is configured with the processor 91 and the memory 92, each function of the processing circuit 90 is realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a tracking program and stored in the memory 92. In the processing circuit 90, each function is realized by the processor 91 reading and executing the tracking program stored in the memory 92. That is, the processing circuit 90 includes the memory 92 for storing the tracking program that results in the processing of the tracking device 10A being executed. This tracking program can also be said to be a program that causes the tracking device 10A to execute each function realized by the processing circuit 90. This tracking program may be provided by a computer-readable recording medium on which the tracking program is recorded, or by other means such as a communication medium.

[0109] The tracking program can also be said to be a program that causes tracking device 10A to execute the processes of steps S10 to S50 in Fig. 2. Here, processor 91 is, for example, a CPU (Central Processing Unit), a processing device, an arithmetic device, a microprocessor, a microcomputer, or a DSP (Digital Signal Processor). Also, memory 92 is, for example, a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable ROM), or EEPROM (Electrically EPROM), a magnetic disk, a flexible disk, an optical disk, a compact disk, a minidisk, or a DVD (Digital Versatile Disc).

[0110] FIG. 13 is a diagram illustrating a configuration example of a processing circuit included in the tracking device according to the first and second embodiments, when the processing circuit is configured with dedicated hardware. The processing circuit 93 illustrated in FIG. 13 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The processing circuit 93 may be partially realized with dedicated hardware and partially realized with software or firmware. In this way, the processing circuit 93 can realize each of the above-described functions by dedicated hardware, software, firmware, or a combination thereof.

[0111] As described above, according to the second embodiment, tracking device 10B detects the feature points and feature amounts of the target, and compares the feature amounts of the current frame with the feature amounts of the frame n frames prior, thereby eliminating feature points other than the feature points of the target candidate, thereby enabling accurate tracking of the detected target.

[0112] In parallel with the target detection process, tracking device 10B classifies the feature points in the image into classes and calculates the center of gravity of the classes. Tracking device 10B then compares the center of gravity of the classes with the vertex coordinates and center coordinates of the bounding boxes to remove any erroneously detected bounding boxes, thereby improving the target detection accuracy.

[0113] The configurations shown in the above embodiments are merely examples, and may be combined with other known technologies, or different embodiments may be combined with each other. It is also possible to omit or modify parts of the configurations as long as they do not deviate from the gist of the invention.

[0114] Various aspects of the present disclosure are summarized below as appendices.

[0115] (Appendix 1) an imaging processing unit that captures an image of a target that is a candidate for a tracking target; a target detection processing unit that detects a bounding box, which is an area where the target exists, from the image and a target position, which is a position of the target corresponding to a center point of the bounding box; a tracking processing unit that calculates an estimated position of the tracking target by a state estimation process based on the target position and a previous target position that is a target position that precedes the target position in the time series, and outputs the calculated estimated position as the estimated position of the tracking target; Equipped with the target detection processing unit cuts out a plurality of cutout regions from the image, and detects the bounding box and the target position for each cutout region; A tracking device characterized by: (Appendix 2) a feature extraction unit that extracts a plurality of feature points from the image; a detection comparison processing unit that deletes erroneously detected bounding boxes from the bounding boxes based on the coordinates of the bounding boxes and the positions of the feature points, and determines the remaining bounding boxes as target bounding boxes; Furthermore, the tracking processing unit calculates an estimated position of the tracking target by the state estimation process for the bounding box determined by the detection comparison processing unit; 2. The tracking device according to claim 1, (Appendix 3) the feature extraction unit further extracts feature amounts of the feature points, and removes erroneously detected bounding boxes from the bounding boxes based on a correlation between the feature amounts and feature amounts that precede the feature amounts in time series. 3. The tracking device according to claim 2, (Appendix 4) the feature extraction unit deletes feature points whose correlation is lower than a reference value from the feature points; the detection comparison processing unit deletes the erroneously detected feature points based on the coordinates of the bounding boxes and the positions of the feature points, and deletes the bounding boxes in which the feature points do not exist as erroneously detected bounding boxes. 4. The tracking device according to claim 3, (Appendix 5) the feature extraction unit clusters the feature points and calculates a center of gravity of the clustered feature points; the detection comparison processing unit deletes the erroneously detected center of gravity point based on the coordinates of the bounding box and the position of the center of gravity point, and deletes the bounding box in which the center of gravity point does not exist as the erroneously detected bounding box; 4. The tracking device according to claim 3, (Appendix 6) The coordinates of the bounding box include at least one piece of information of vertex coordinates of the bounding box, coordinates of the center point, and coordinates of the center of gravity of the bounding box. 6. The tracking device according to claim 4 or 5, (Appendix 7) a boundary detection unit that detects boundary coordinates that are coordinates of a boundary between an area in the image where the target may exist and an area where the target does not exist, the target detection processing unit detects the bounding box and the target position based on the boundary line coordinates and the image; 7. A tracking device according to any one of claims 1 to 6. (Appendix 8) an image scale conversion unit that performs image scale conversion of the image based on the boundary line and the image so that a target within a predetermined size range is included in a frame after image scale conversion; the target detection processing unit detects the bounding box and the target position based on the boundary line coordinates after image scale conversion and the image; 8. The tracking device according to claim 7, (Appendix 9) the target detection processing unit cuts out a plurality of cutout regions from the image, and detects the bounding box and the target position for each cutout region; 9. A tracking device according to any one of appendices 1 to 8. (Appendix 10) the target detection processing unit determines an overlapping range between adjacent clipping regions based on a maximum detection size when detecting the bounding box. 10. The tracking device according to claim 9, (Appendix 11) the target detection processing unit sets a target detection range, which is a detection range of the target, based on the boundary line coordinates and the image, and detects the bounding box and the target position from an area within the target detection range. 9. The tracking device according to claim 7 or 8. (Appendix 12) The target detection processing unit detects the target using a general object recognition technology. 12. A tracking device according to any one of claims 1 to 11. (Appendix 13) an imaging processing step in which the tracking device captures an image of a target that is a candidate for a tracking target; a target detection processing step in which the tracking device detects a bounding box, which is an area in which the target exists, from the image, and a target position, which is a position of the target corresponding to a center point of the bounding box; a tracking processing step in which the tracking device calculates an estimated position of the tracking target by a state estimation process based on the target position and a previous target position that is a target position that is previous in time series, and outputs the calculated estimated position as the estimated position of the tracking target; Including, In the target detection processing step, the tracking device cuts out a plurality of cutout regions from the image, and detects the bounding box and the target position for each cutout region. A tracking method characterized by: [Explanation of symbols]

[0116] 1 imaging processing unit, 2 boundary detection unit, 3 image scale conversion unit, 4 target detection processing unit, 6A, 6B tracking processing unit, 7 detection comparison processing unit, 8 feature extraction processing unit, 10A, 10B tracking device, 21 ship, 30 horizon, 35 feature points, 51A to 51G cut-out area, 71 bounding box, 90, 93 processing circuit, C1 to C7 classes, F1, F2 frames.

Claims

1. an imaging processing unit that captures an image of a target that is a candidate for a tracking target; a target detection processing unit that detects a bounding box, which is an area where the target exists, from the image and a target position, which is a position of the target corresponding to a center point of the bounding box; a tracking processing unit that calculates an estimated position of the tracking target by a state estimation process based on the target position and a previous target position that is a target position that precedes the target position in the time series, and outputs the calculated estimated position as the estimated position of the tracking target; Equipped with the target detection processing unit cuts out a plurality of cutout regions from the image, and detects the bounding box and the target position for each cutout region; A tracking device characterized by:

2. a feature extraction unit that extracts a plurality of feature points from the image; a detection comparison processing unit that deletes erroneously detected bounding boxes from the bounding boxes based on the coordinates of the bounding boxes and the positions of the feature points, and determines the remaining bounding boxes as target bounding boxes; Furthermore, the tracking processing unit calculates an estimated position of the tracking target by the state estimation process for the bounding box determined by the detection and comparison processing unit; 2. The tracking device according to claim 1.

3. the feature extraction unit further extracts feature amounts of the feature points, and removes erroneously detected bounding boxes from the bounding boxes based on a correlation between the feature amounts and feature amounts that precede the feature amounts in time series.

3. The tracking device according to claim 2.

4. the feature extraction unit deletes feature points whose correlation is lower than a reference value from the feature points; the detection comparison processing unit deletes any erroneously detected feature points based on the coordinates of the bounding boxes and the positions of the feature points, and deletes any bounding box in which the feature points are not present as an erroneously detected bounding box.

4. The tracking device according to claim 3.

5. the feature extraction unit clusters the feature points and calculates a center of gravity of the clustered feature points; the detection comparison processing unit deletes the erroneously detected center of gravity based on the coordinates of the bounding box and the position of the center of gravity, and deletes, as the erroneously detected bounding box, any bounding box in which the center of gravity does not exist.

4. The tracking device according to claim 3.

6. The coordinates of the bounding box include at least one piece of information of vertex coordinates of the bounding box, coordinates of the center point, and coordinates of the center of gravity of the bounding box.

6. The tracking device according to claim 4 or 5.

7. a boundary detection unit that detects boundary coordinates that are coordinates of a boundary between an area in the image where the target may exist and an area where the target does not exist, the target detection processing unit detects the bounding box and the target position based on the boundary line coordinates and the image; 2. The tracking device according to claim 1.

8. an image scale conversion unit that performs image scale conversion of the image based on the boundary line and the image so that a target within a predetermined size range is included in a frame after image scale conversion; the target detection processing unit detects the bounding box and the target position based on the boundary line coordinates after image scale conversion and the image; 8. The tracking device according to claim 7.

9. the target detection processing unit cuts out a plurality of cutout regions from the image, and detects the bounding box and the target position for each cutout region; 2. The tracking device according to claim 1.

10. the target detection processing unit determines an overlapping range between adjacent clipping regions based on a maximum detection size when detecting the bounding box.

10. The tracking device according to claim 9.

11. the target detection processing unit sets a target detection range, which is a detection range of the target, based on the boundary line coordinates and the image, and detects the bounding box and the target position from an area within the target detection range.

9. The tracking device according to claim 7 or 8.

12. The target detection processing unit detects the target using a general object recognition technology.

2. The tracking device according to claim 1.

13. an imaging processing step in which the tracking device captures an image of a target that is a candidate for a tracking target; a target detection processing step in which the tracking device detects a bounding box, which is an area in which the target exists, from the image, and a target position, which is a position of the target corresponding to a center point of the bounding box; a tracking processing step in which the tracking device calculates an estimated position of the tracking target by a state estimation process based on the target position and a previous target position that is a target position that is previous in time series, and outputs the calculated estimated position as the estimated position of the tracking target; Including, In the target detection processing step, the tracking device cuts out a plurality of cutout regions from the image and detects the bounding box and the target position for each cutout region. A tracking method characterized by:

Images