Image processing device, image processing method, and program

Abstract

To improve tracking accuracy by suppressing occurrence of erroneous tracking.SOLUTION: An image processing device detects, from an image, a first area to be tracked and a second area to be associated with the first area, and determines whether the first area is occluded based on the second area. The image processing device then tracks the first area after it is determined that the first area has been occluded, based on a determination result of the occlusion for the first area and the second area.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to image processing technology for tracking a subject.

Background Art

[0002] Techniques for extracting a specific subject area from images supplied in a time series and tracking the subject area are used for identifying a human face area, a human body area, etc. in a moving image. In digital still cameras and digital video cameras, subject tracking technology is used for focus and exposure automatic adjustment for a subject.

[0003] Patent Document 1 discloses a technique for automatically tracking a specific subject using template matching. According to the technique of this Patent Document 1, a partial image obtained by cutting out an image area including a specific subject is used as a template, and by calculating an area having a high similarity to the template, a specific subject can be tracked. On the other hand, in template matching processing, similarity of pixel patterns and color histograms is used, but when the types, sizes, and appearances of tracking targets are diverse, there is a possibility that the tracking targets cannot be fully captured. In particular, when the tracking target is shielded, etc., it often happens that a target different from the tracking target is erroneously tracked. In contrast, Patent Document 2 discloses a method of performing two types of subject detections on the same subject and tracking while selecting which of the results of the two types of subject detections to use for tracking. In the technique of Patent Document 2, more accurate tracking is attempted by using the results of two types of subject detections.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, even when the technique of Patent Document 2 is used, for example, when another subject similar to the subject to be tracked intersects with the subject to be tracked, it may become impossible to accurately determine which of these subjects was the tracking target. For this reason, another similar subject may be erroneously tracked as the tracking target, resulting in a decrease in tracking accuracy.

[0006] Therefore, an object of the present invention is to suppress the occurrence of erroneous tracking and improve tracking accuracy.

Means for Solving the Problems

[0007] The image processing apparatus of the present invention includes: detection means for detecting, from an image, a first region to be tracked and a second region associated with the first region; determination means for determining whether the first region is occluded based on the second region; and tracking means for tracking the first region after it is determined by the determination means that the first region is occluded, based on the determination result of occlusion of the first region and the second region.

Effects of the Invention

[0008] According to the present invention, it is possible to suppress the occurrence of erroneous tracking and improve tracking accuracy.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments according to the present invention will be described with reference to the drawings. Each of the embodiments described hereinafter does not limit the present invention, and not all combinations of the features described in the present embodiment are essential for the solution means of the present invention. The configuration of the embodiment can be appropriately modified or changed according to the specifications of the apparatus to which the present invention is applied and various conditions (usage conditions, usage environment, etc.). Also, some of the embodiments described later may be appropriately combined to form a configuration. In the following embodiments, redundant descriptions of the same or similar configurations and processing steps will be omitted.

[0011] <First Embodiment> FIG. 1(a) is a diagram showing a hardware configuration example of the image processing apparatus according to the present embodiment, and shows a configuration example of a general-purpose information processing apparatus including a CPU 131, a memory 132, an input unit 133, a storage unit 134, a display unit 135, and a communication unit 136. In the present embodiment, in the information processing apparatus shown in FIG. 1(a), each functional configuration of the image processing apparatus 100 as shown in FIG. 1(b) is realized.

[0012] In the information processing apparatus shown in Fig. 1(a), the CPU 131 is a control device that overall controls the entire apparatus. The storage unit 134 stores a control program for the CPU 131 to control the information processing apparatus, an image processing program for realizing image processing related to each functional unit of the image processing apparatus 100 shown in Fig. 1(b), and the like. Further, the storage unit 134 can also store an image acquired by photographing. The memory 132 is a memory for expanding the program read by the CPU 131 from the storage unit 134 and executing processing. Also, the memory 132 is used as a temporary storage area for temporarily storing image data and the like that are targets of various processes.

[0013] The display unit 135 is a display device that displays images, texts, and the like. The input unit 133 is a device provided with at least any one of an input key, a pointing device for the screen display on the display unit 135, a touch panel, and the like. A user who uses the information processing apparatus can input an instruction operation and the like via the input unit 133, and the CPU 131 performs processing according to the instruction operation from the user.

[0014] The communication unit 136 performs communication wirelessly or by wire. The communication unit 136 transmits and receives images, programs, and other data via a communication path. The CPU 131 performs image processing and the like, which will be described later, on the image received via the communication unit 136.

[0015] As described above, the hardware configuration shown in Fig. 1(a) has components similar to the hardware components mounted on an information processing apparatus such as a general PC (personal computer). Therefore, various functions realized in the image processing apparatus 100 shown in Fig. 1(b) can be implemented as software (image processing program) operating on a PC. That is, the CPU 131 realizes the processing related to each functional unit of the image processing apparatus 100 shown in Fig. 1(b) by executing the image processing program according to this embodiment. Of course, each functional unit of the image processing apparatus 100 shown in Fig. 1(b) may be realized as a circuit configuration.

[0016] Hereinafter, an overview of each functional unit of the image processing apparatus 100 according to the present embodiment shown in FIG. 1(b) will be described. The imaging device 110 includes an optical system, an imaging element, etc., and captures images (moving images) in chronological order for each frame and outputs them to the image acquisition unit 101. In the case of the present embodiment, as the imaging device 110, for example, a digital camera, a surveillance camera, etc. are assumed. The image acquisition unit 101 of the image processing apparatus 100 acquires images in chronological order for each frame via the communication unit 136 described above. Note that the image processing apparatus 100 may be included in a digital camera, a surveillance camera, etc., but in the present embodiment, it is assumed that the image processing apparatus 100 and the imaging device 110 are independent devices. Also, the images acquired by the image acquisition unit 101 are not limited to the images for each frame in chronological order captured by the imaging device 110, and images for each frame in chronological order generated by various image generation devices may be acquired.

[0017] The tracking unit 102 tracks a specific tracking target by detecting a first region, which is an image region of the specific tracking target, from each image for each frame acquired by the image acquisition unit 101. In the present embodiment, an example of detecting an image region of a specific object (subject), which is the tracking target, from each image as the first region will be given. In the following description, the specific object (subject) detected as the first region of the tracking target from the image will be referred to as the tracking target object. The tracking unit 102 estimates in which region of each image acquired by the image acquisition unit 101 the specific tracking target object exists, performs tracking processing to sequentially track the tracking target object in each image for each frame, and outputs the tracking result for each frame.

[0018] The object detection unit 103 detects a candidate region (third region) that can be a second region associated with the first region from each image for each frame acquired by the image acquisition unit 101. In the case of this embodiment, as the candidate region, an image region of an object (subject) different from the object to be tracked, which can be an associated object described later, is detected. In the following description, an object that can be an associated object will be referred to as a candidate object. The object detection unit 103 detects in each image for each frame acquired by the image acquisition unit 101 in which region the candidate object exists, and outputs the detection result of the candidate object estimated for each of those frames.

[0019] The linking unit 104 receives the tracking result of the object to be tracked for each frame by the tracking unit 102 and the detection result of the candidate object for each frame by the object detection unit 103. Then, the linking unit 104 sets, among the candidate objects detected by the object detection unit 103, a candidate object whose relationship with the object to be tracked satisfies a predetermined condition as an object associated with that object to be tracked. In this embodiment, among the candidate objects, an object associated with the object to be tracked will be referred to as an associated object. The linking unit 104 links a candidate object as an associated object to the object to be tracked when at least one of the relative positional relationship between the object to be tracked and the candidate object and the relative size relationship between the object to be tracked and the candidate object satisfies a predetermined condition respectively. Details of the predetermined condition will be described later. Note that when there is no candidate object among the candidate objects detected by the object detection unit 103 whose relationship with the object to be tracked satisfies a predetermined condition, it is assumed that no associated object has been detected.

[0020] As described above, in this embodiment, among the candidate objects detected by the object detection unit 103, a candidate object determined by the linking unit 104 to satisfy a predetermined condition in relation to the object to be tracked is linked to the object to be tracked as an associated object. That is, among the functional units of the image processing apparatus 100 shown in Fig. 1(b), the object detection unit 103 and the linking unit 104 realize the function as an association unit that detects a second region associated with the first region and associates it with the first region.

[0021] The feature acquisition unit 105 extracts features from the image regions of each candidate object detected by the object detection unit 103. That is, in the feature acquisition unit 105, the features of the image regions of the related objects linked to the object to be tracked by the linking unit 104 are also extracted. The features obtained from these image regions are used for tracking recovery based on the occlusion determination in the occlusion determination unit 106 and the related object feature matching process in the matching unit 107, which will be described later.

[0022] The occlusion determination unit 106 determines whether the object to be tracked is occluded by other objects or the like in the image. In the case of this embodiment, the occlusion determination unit 106 determines whether the object to be tracked is occluded based on the presence or absence of related objects or the temporal change of the features obtained by the feature acquisition unit 105 from the regions of the related objects. For example, when the temporal change (change per frame) of the features obtained by the feature acquisition unit 105 from the region of the related object linked to the object to be tracked exceeds a predetermined change threshold, the occlusion determination unit 106 determines that the object to be tracked is occluded. Also, for example, when the related object changes from a state where it was detected by the linking unit 104 to a state where it is not detected, the occlusion determination unit 106 may also determine that the object to be tracked is occluded. Then, the occlusion determination unit 106 outputs the occlusion determination result. The details of the occlusion determination in the occlusion determination unit 106 will be described later.

[0023] The matching unit 107 searches for the region of the related object based on the occlusion determination result by the occlusion determination unit 106 and the features obtained by the feature acquisition unit 105 from the image regions of the related object and each other candidate object, and performs a matching process for correctly correcting or re-tracking the tracking result. In the case of this embodiment, when the occlusion determination unit 106 determines that the object to be tracked is occluded, the matching unit 107 performs a related object feature matching process to determine whether there is a candidate object having features similar to the features of the related object linked to the object to be tracked before the occlusion. Then, the matching unit 107 outputs the related object feature matching result. The details of the related object feature matching process will be described later.

[0024] The integration unit 108 receives the tracking result of the object to be tracked by the trailing unit 102 and the related object feature matching result by the matching unit 107, and performs tracking result integration processing based on them. Details of the tracking result integration processing in the integration unit 108 will be described later.

[0025] The output unit 120 outputs the tracking result after the tracking result integration processing by the integration unit 108. The tracking result output from the output unit 120 is applied to, for example, the autofocus function of the imaging device 110. For example, in the imaging device 110, several distance measurement points are sampled from the distance measurement points corresponding to the area of the tracking result, and the sampled distance measurement points are used for, for example, phase difference AF. Thereby, in the imaging device 110, it is possible to continuously perform subject tracking while performing AF while tracking a specific subject as the object to be tracked. In addition, the tracking result output from the output unit 120 can also be used during the automatic exposure adjustment of the imaging device 110.

[0026] With the above-described configuration, even when occlusion or the like occurs due to a similar object that is difficult to distinguish from the object to be tracked during tracking of the object to be tracked, the image processing apparatus 100 according to the present embodiment can discover the object to be tracked by searching using the features of the related object associated with the object to be tracked. Therefore, according to the image processing apparatus 100 of the present embodiment, it is possible to suppress the occurrence of false tracking and improve the tracking accuracy. As an example, when tracking a specific horse as the object to be tracked in a horse racing scene, the jockey riding on the horse is associated as a related object with the specific horse that is the object to be tracked. Then, when the specific horse crosses another horse or the like, occlusion is determined by changes in the feature of the image area of the jockey, and further, by searching for the original jockey after the occlusion, it becomes possible to track the specific horse.

[0027] FIG. 2 is a flowchart showing the flow of image processing in the image processing apparatus 100 according to the present embodiment. The processing according to the flowchart of FIG. 2 is assumed to be realized by the CPU 131 executing the image processing program according to the present embodiment. Here, the following description will be made on the assumption that both the tracking unit 102 and the object detection unit 103 are configured by a multi-layer neural network. In the following flowchart, the symbol S represents a processing step (processing process). Note that only the outline of the processing is described in the flowchart of FIG. 2, and the detailed processing will be described later.

[0028] First, as the process of S200, the image acquisition unit 101 acquires, for each frame, an image (moving image) in time series order output from the imaging device 110 by starting shooting. When the image acquisition unit 101 acquires the time-series images for each frame from the imaging device 110, as the process of S201, it crops or enlarges / reduces each of those images to convert them to a predetermined resolution.

[0029] As the process of S205, the tracking unit 102 performs a tracking result calculation process of detecting and tracking a specific tracking target object from the images for each frame whose resolution has been converted by the image acquisition unit 101, and outputs the tracking result.

[0030] Separate from the tracking process by the tracking unit 102, as the process of S206, the object detection unit 103 performs a process of detecting candidate objects different from the tracking target object from the images for each frame whose resolution has been converted by the image acquisition unit 101, and outputs the detection result of the candidate objects.

[0031] Then, as the process of S207, the linking unit 104 links, as related objects, candidate objects whose relationship with the tracking target object satisfies a predetermined condition to the tracking target object based on the tracking result by the tracking unit 102 and the detection result of the candidate objects by the object detection unit 103.

[0032] Subsequently, as the process of S208, the feature acquisition unit 105 performs a related object feature calculation process for calculating features from the image region of the related object linked to the object to be tracked by the linking unit 104. Then, as the process of S209, the occlusion determination unit 106 performs an occlusion determination process for determining whether or not occlusion of the object to be tracked has occurred based on the temporal change of the features of the related object and the like.

[0033] Next, as the process of S210, the matching unit 107 determines whether or not the occlusion determination result by the occlusion determination unit 106 is a determination result that the object is occluded. If it is determined that there is occlusion, as the process of S211, the matching unit 107 performs a matching process based on the features of the related object. Although details will be described later, the matching unit 107 performs a related object feature matching process for searching for the region in which the related object linked to the object to be tracked before being determined to be occluded appears in the image after occlusion. That is, the matching unit 107 searches for the region of the related object linked to the object to be tracked before occlusion from the image after occlusion of the object to be tracked, and outputs the related object feature matching result. Note that when the related object is not searched from the image after occlusion, or when the object to be tracked is not occluded, the matching unit 107 outputs a related object feature matching result indicating that fact.

[0034] Subsequently, as the process of S212, the integration unit 108 performs a process of integrating the tracking results, taking into account both the matching result by the matching unit 107 and the tracking result by the tracking unit 102. Although details will be described later, when the object to be tracked is occluded, the integration unit 108 performs a tracking result integration process in which the region to be linked to the related object searched from the image after occlusion is set as the region of the object to be tracked in the image after occlusion. Also, for example, when the related object cannot be searched from the image after occlusion, or when the object to be tracked is not occluded, the integration unit 108 sets the re-tracking result by the tracking unit 102 as the result of the tracking result integration process.

[0035] After that, as the process of S213, the image processing apparatus 100 determines whether to end the tracking process. If the tracking process is not ended, the process returns to S200. On the other hand, if the tracking is ended, the process of the flowchart in FIG. 2 ends.

[0036] <Image conversion process> Next, the image conversion process performed by the image acquisition unit 101 in S201 will be described. Assume that the image acquired by the image acquisition unit 101 is a YUV image with a width of 6000 pixels and a height of 4000 pixels, for example. In the case of this embodiment, as described above, both the tracking unit 102 and the object detection unit 103 are configured by a multi-layer neural network. Therefore, the image acquisition unit 101 converts the image into a predetermined size so as to match the input format of the multi-layer neural network, which is the component for the tracking unit 102 to calculate the tracking result and the component for the object detection unit 103 to calculate the object result. In the case of this embodiment, the image after the image conversion that matches the input format of those multi-layer neural networks is a YUV image with a width of 600 pixels and a height of 400 pixels, which is one-tenth of the original size of the image. In the case of this embodiment, the image acquisition unit 101 scales the area after cropping a part of the acquired image to a size that matches the input size to the tracking unit 102 and the object detection unit 103. The area to be cropped is based on the position of the object detection result area that is closest to the position specified by the user, and is an area obtained by multiplying the area of the object detection result area by a constant.

[0037] Note that the cropping is not limited to the region based on the object detection result as described above. It may also be the cropping of the region near the position specified by the user, or the cropping of the cropping region based on the region where the tracking result was obtained in the past tracking process. Instead of cropping, the entire image may be used for calculating the tracking result or object detection. In particular, the image input to the tracking unit 102 may be an image obtained by cropping the region of the image acquired by the image acquisition unit 101 where the object to be tracked is considered to be shown. Note that the cropping of the region where the object to be tracked is shown may be the cropping based on the region where the tracking result was obtained in the past tracking process as described above, or the cropping of the region where the object to be tracked is most likely to be shown considering the tracking result and the object detection result, etc.

[0038] <Tracking result calculation process> Next, the tracking result calculation process performed by the tracking unit 102 in S205 will be described with reference to FIGS. 3(a) and 3(b). Here, an example in which tracking is performed by template feature extraction will be described. The tracking unit 102 first uses the image 300 acquired by the image acquisition unit 101 as an input image, and generates an intermediate feature amount 302 (feature amount 304 of the search image) by the feature extraction DNN 301 using a deep neural network (DNN). The feature extraction DNN 301 may share the feature extraction DNN and parameters as described in the following Reference 1, or may be different. In the present embodiment, the tracking unit 102 internally holds the template feature amount 303 and uses it for the tracking result calculation process.

[0039] Reference 1: High Performance Visual Tracking with Siames Region Proposal Network, Bo Li et al., 2018

[0040] Subsequently, the trailing part 102 performs a convolution operation 305 on the template feature amount 303 and the feature amount 304 of the search image, and inputs the result into the region proposal DNN 306. The region proposal DNN 306 outputs a similarity map 307 representing the position of the trailing candidate in the search image and a size map 308 representing the size. The region proposal DNN 306 may use the FCN (Fully-Convolutional Netrowk) described in Reference 1, or may have a configuration combined with a fully-connected layer. The region proposal DNN 306 is trained so that it can output the position and size of the trailing target by taking as input the map obtained by convolving the template feature amount 303 and the feature amount 304 of the search image. The trailing part 102 calculates the position and size of the subject from the similarity map 307 and the size map 308 representing the position of the trailing target object, and outputs them as the trailing result. As a method for such calculation, for example, a method can be cited in which the trailing result corresponding to the map point having the maximum value of the similarity map 307 is adopted. Additionally, as described in Reference 1, a method that gives priority to the similarity of the subject at the center of the input image using NMS (Non-maximum-suppression) or a cosine window may also be used. In the example of FIG. 3, an example in which a horse (zebra) is the trailing target is given, but the trailing target may be other animals, vehicles, or even people, etc.

[0041] <Object detection process> Next, the object detection process performed by the object detection unit 103 in S206 will be described with reference to FIG. 4(a). The input image to the multi-layer neural network of the object detection unit 103 is an image (input image 401) whose resolution has been converted by the image acquisition unit 101, and is a YUV image with a width of 600 pixels, a height of 400 pixels, and 3 channels. The input image 401 does not necessarily have to be the same as the image used in the tracking result calculation process of S205. For example, when detecting a person in the object detection process, the object detection unit 103 detects two people shown in the input image 401 and outputs them as detection results 403 and 404 respectively. The multi-layer neural network used in the object detection process is pre-trained in advance using a large number of training data (images and the width and height of objects) as described in the following Reference 2. In the description of this embodiment, an example is given in which the detection results 403 and 404 are detected as rectangular regions, but it is not limited to rectangular regions, and may be, for example, polygonal regions, or may be estimated as regions such as in a semantic segmentation task.

[0042] Reference 2: Objects as Points, Xingyi Zhou et al., 2019

[0043] In the case of this embodiment, an example is given in which the object detection unit 103 detects one type of object as a candidate object for a related object, but there may be multiple types (N types) of candidate objects for the related object. For example, in addition to a person, the object detection unit 103 can also detect, as a candidate object, a mask worn on a horse. In this case, the object detection unit 103 detects, for example, a person as the first candidate object and a horse mask as the second candidate object. When the object detection unit 103 detects N types of candidate objects, in the subsequent association unit 104, N types of related objects are associated with the objects to be tracked, and further, the occlusion determination unit 106 performs occlusion determination based on the characteristics of the N types of related objects. Also, the feature acquisition unit 105 and the matching unit 107 also perform the above-described processing for each of the N types of related objects respectively.

[0044] <Related object association process> Next, the related object linking process performed by the linking unit 104 in S207 will be described with reference to FIG. 4(b). In the example of FIG. 4(b), an example is given where a horse is tracked in the tracking result calculation process of S205 and a person is detected in the object detection process of S206. As the related object linking process in S207, the linking unit 104 sets a candidate object whose relationship with the tracking target object satisfies a predetermined condition as the related object of the tracking target object. In the example shown in FIG. 4(b), the linking unit 104 links the detection result 411 of the candidate object adjacent to the tracking result 413 of the tracking target object as the detection result of the related object.

[0045] The linking unit 104 receives the tracking result 413 of the tracking target object by the tracking unit 102 and the detection results 411 and 412 of the candidate object by the object detection unit 103. When a plurality of horses and people are shown in the input image 401 as in the horseback riding scene illustrated in FIG. 4(b), the linking unit 104 determines which of the detection results 411 and 412 of the candidate object (person) satisfies a predetermined condition with respect to the tracking result 413 of the tracking target object (horse). The linking unit 104 links the one (detection result 411 in this example) whose relative position and size with respect to the tracking result 413 of the tracking target object (horse) satisfy a predetermined condition among the detection results 411 and 412 of the person as the detection result of the related object to the tracking result 413.

[0046] For example, in the riding scene illustrated in FIG. 4(b), it is considered that the person is always riding on the horse. In the case of this riding scene, as the simplest example of a predetermined condition, a condition can be considered such that a frame that is above the center point of the tracking result 413 and within a certain distance from the center point of the frame of the tracking result 413 to the center point of the detection result is used as the detection result of the related object. Therefore, the linking unit 104 determines the detection result 411 that is a frame above the center point of the tracking result 413 and within a certain distance from the center point of the frame of the tracking result 413 to the center point of the detection result as the detection result of the related object, and links it to the tracking result 413. Also, in the case of a riding scene as shown in FIG. 4(b), generally, since the horse is larger than the person, it is highly likely that the detection result of a candidate object larger than the horse in the tracking result is not the person riding on the horse, and it is considered inappropriate to link it as a related object. Furthermore, the relative size of the person with respect to the horse is considered to fall within a certain range, and when the relative size is extremely small, it is highly likely that it is a small object other than the person, so in this case as well, it is considered inappropriate to link it as a related object. For this reason, it is desirable to include the relative size of the candidate object with respect to the object to be tracked in the predetermined condition. That is, in this case, when the relative size with respect to the tracking result 413 is not within the threshold range set as a predetermined condition, either a large size or conversely a small size, the detection result of that candidate object is excluded from the linking target.

[0047] In addition, in the case of an example where the object detection unit 103 detects N types of candidate objects, the linking unit 104 performs determination and linking of related objects with respect to the object to be tracked for each of those N types. For example, assume that in the object detection unit 103, a person is detected as the first candidate object and a horse mask is detected as the second candidate object. In this case, the linking unit 104 links the person related to the horse as the object to be tracked as the first related object and the horse mask of the object to be tracked as the second related object to the horse as the object to be tracked.

[0048] <Related Object Feature Calculation Process> Next, the related object feature calculation process performed by the feature acquisition unit 105 in S208 will be described with reference to FIG. 4(b). The feature acquisition unit 105 acquires features from the image regions of each candidate object (including related objects) detected by the object detection unit 103. The features acquired from the region of the related object are used in the occlusion determination process performed by the subsequent occlusion determination unit 106 in S209, and the features acquired from the regions of each candidate object including the related object are used in the related object feature matching process performed by the subsequent matching unit 107 in S211. The feature acquisition unit 105 acquires, for example, color histogram features in the regions of the detection result 411 of the related object and the detection result 412 of other candidate objects, or the output of a multi-layer neural network for feature calculation as features. In this embodiment, the detection results of the related object and each candidate object are rectangular regions, and an example of the feature calculation method is to calculate the color histogram of the rectangular region.

[0049] First, the feature acquisition unit 105 cuts out the rectangular regions of the detection result 411 of the related object and the detection result 412 of other candidate objects from the input image 401. Then, the feature acquisition unit 105 calculates the color histogram feature 415 from each of the cut-out rectangular region images 414.

[0050] In the case of an example where the object detection unit 103 detects N types of candidate objects, the feature acquisition unit 105 acquires the features of the candidate objects for each of those N types. For example, assume that the object detection unit 103 detects a person as the first candidate object and a horse mask as the second candidate object. In this case, the feature acquisition unit 105 acquires the feature of the rectangular region image of the first candidate object of the person as the first feature, and the feature of the rectangular region image of the second candidate object of the horse mask as the second feature.

[0051] <Occlusion determination process> Next, the occlusion determination process performed by the occlusion determination unit 106 in S209 will be described with reference to FIGS. 5(a) and 5(b). In the example of FIG. 5, it is assumed that for the input image 500 of the current frame, the feature acquisition unit 105 has acquired a feature 504 (hereinafter referred to as the current feature 504) from the rectangular region image 502 of the detection result 505 of the related object associated with the tracking result 507 by the association unit 104. Also, the feature 503 shown in FIG. 5(a) indicates a feature (hereinafter referred to as the past feature 503) acquired from the rectangular region image 501 of the related object that has already been acquired in the image of a past frame before the input image 500. It is assumed that this past feature 503 is held by the occlusion determination unit 106. The occlusion determination unit 106 determines whether occlusion has occurred in the tracking target object and the related object by comparing the current feature 504 acquired from the input image 500 with the past feature 503 acquired from the image of the past frame.

[0052] Here, when tracking the horse of the tracking target 508, the case where the horse of the tracking target 508 is occluded by another horse, that is, the case where occlusion occurs by another object that is difficult to distinguish from the tracking target object will be described as an example. As in this example, when the horse of the tracking target 508 is occluded by another horse, the rectangular region image of the detection result 505 of the related object of the tracking target 508 becomes like the rectangular region image 502 in FIG. 5(a). The current feature 504 obtained from the rectangular region image 502 has a difference due to the object on the front side (the head of another person in the rectangular region image 502) compared to the past feature 503 of the rectangular region image 501 acquired from the image of the previous past frame. For example, in the case of the color histogram feature cited in this embodiment, the color histogram feature also changes in the direction of increasing the white component by the amount that the occluding person on the front side (the white person) is included in the rectangular region image 502 of the related object.

[0053] Therefore, the occlusion determination unit 106 calculates the correlation or Bhattacharyya distance between the color histograms of the past feature 503 and the current feature 504 in order to determine the change in the color histogram feature. For example, when the Bhattacharyya distance becomes greater than or equal to a certain value, the occlusion determination unit 106 determines that the related object (the person in the detection result 505) of the tracking target 508 is occluded. In particular, in the case of a relationship such as a horse and a person in a riding state, if the rider on the tracking target horse is occluded, it is assumed that the tracking target horse is also occluded with a high probability. Therefore, when the occlusion determination unit 106 determines that the detection result 505 of the related object associated with the tracking result 507 is occluded, it also determines that the tracking result 507 is occluded at the same time. For example, even in a case where the colors and shapes of the horses are similar and difficult to distinguish, if the rider on the horse of the tracking target 508 is occluded, the occlusion determination unit 106 can determine that the tracking target 508 is also occluded. As a result, it is possible to correctly determine that the tracking target horse is occluded by another horse that is difficult to distinguish from the tracking target horse, and prevent the situation where the horse in front is erroneously tracked continuously in the tracking process. In the present embodiment, for the occlusion determination, in addition to the Bhattacharyya distance of the object detection results between the color histograms, a method of determining occlusion if the regions of the object detection results overlap by a predetermined ratio or more may be used.

[0054] <Matching Process and Integration Process of Tracking Results> Next, the related object feature matching process performed by the matching unit 107 in S211 and the tracking result integration process performed by the integration unit 108 in S212 will be described with reference to FIG. 5(b). It is assumed that FIG. 5(b) shows the input image 520 after the occlusion has been resolved, starting from a situation where occlusion has occurred as in the input image 500 illustrated in FIG. 5(a). However, in the case of the example in FIG. 5(b), due to the occurrence of occlusion as in FIG. 5(a), the correct tracking result 507 before occlusion has led to an incorrect tracking result 509 (referred to as the incorrect tracking result 509) in which a non-tracking target 515 has been erroneously tracked. In this case, in the linking unit 104, a detection result 510 that satisfies a predetermined condition with respect to the incorrect tracking result 509 is linked as a related object. That is, the detection result 511 that satisfies a predetermined condition with respect to the originally correct tracking target 508 should have been linked to the tracking target 508 as a related object.

[0055] Therefore, the matching unit 107 performs a related object feature matching process for searching for the related object (detection result 505) that was linked to the tracking result 507 before the occlusion occurred, from the input image 520 after the occlusion has been resolved in the occlusion determination unit 106. That is, in the present embodiment, the related object feature matching process in the matching unit 107 is performed in order to correctly correct or re-track the incorrect tracking result 509 in the subsequent integration unit 108.

[0056] The matching unit 107 receives the features 512 and 513 respectively acquired by the feature acquisition unit 105 from the detection results 510 and 511 of each candidate object detected by the object detection unit 103 from the input image 520. Also, at this time, the matching unit 107 holds the feature of the detection result (hereinafter referred to as the pre - occlusion feature 514) that was associated with the tracking target 508 until immediately before being determined as occlusion in the occlusion determination unit 106. Then, the matching unit 107 calculates the similarity between the feature 512 acquired from the detection result 510, the feature 513 acquired from the detection result 511, and the pre - occlusion feature 514. For example, when the color histogram feature is used as the image feature, the matching unit 107 determines that the higher the similarity, the closer the distance using the Bhattacharyya distance. For example, the matching unit 107 determines whether the Bhattacharyya distance between the feature of the detection result of interest and the pre - occlusion feature 514 is smaller than the Bhattacharyya distance between the feature of other detection results and the pre - occlusion feature 514 and within a certain distance. And when the Bhattacharyya distance obtained for the feature of the detection result of interest is smaller than the Bhattacharyya distance obtained for the feature of other detection results and within a certain distance, the matching unit 107 determines that the detection result of interest is a related object. In the case of FIG. 5, the matching unit 107 determines that the object of the detection result 511 with the feature 512 is a related object because the Bhattacharyya distance between the pre - occlusion feature 514 and the feature 512 is smaller than the Bhattacharyya distance between the pre - occlusion feature 514 and the feature 513 and within a certain distance. Then, the matching unit 107 determines that the related object (person) associated before being determined as occlusion appears after occlusion, and ends the matching.

[0057] The integration unit 108 performs a process of integrating the tracking result, taking into account both the matching result by the matching unit 107 and the tracking result by the tracking unit 102. Here, it is explained assuming that in the matching unit 107, as illustrated in FIG. 5(b), the feature 512 of the detection result 511 is determined to be a related object because it is sufficiently similar to the pre - occlusion feature 514.

[0058] In this case, the integration unit 108 determines that the tracking result 509, which was considered to be the tracking target associated with the detection result 510 in the example of FIG. 5(b), is a non-tracking target 515 different from the correct tracking target that was being tracked before occlusion. Then, the integration unit 108 either newly searches for a tracking target assuming that the tracking has failed, or determines the image (the image after image conversion) to be input to the next tracking result calculation process based on the area where the obtained tracking result 509 has been moved near the detection result 511. Here, when the matching unit 107 performs a matching process using the reference area of the image after image conversion and the detection result 511 is determined to be the original related object, it is highly likely that the horse near the detection result 511 is the correct tracking target. Therefore, the integration unit 108 crops the area near the detection result 511 and adjusts the tracking unit 102 so that a horse is inferred with a high likelihood in the area near the detection result 511 of the cropped area. As this adjustment method, it is conceivable to apply a window function such as a Cosine-Window centered on the center point of the area of the detection result 511 to the similarity map 307 output by the region proposal DNN 306 of the tracking unit 102. By doing so, it is possible to make the similarity of the tracking target 508 closer to the detection result 511 higher than that of the non-tracking target 515 on the similarity map 307. If the related object cannot be found by the related object feature matching process in the matching unit 107, the integration unit 108 determines that the related object before the occlusion determination has not yet appeared and determines that the matching has failed, and uses the re-tracking result by the tracking unit 102 as the output of the tracking result integration process.

[0059] The method described in the foregoing embodiments is merely an example. In the tracking result calculation process of S205, in addition to the multi-layer neural network, the method is not limited to the above as long as it is a method for extracting features included in the subject such as color histograms and edge densities. Further, in the related object feature calculation process of S208, in addition to the color histogram described above, features obtained by inputting the cut-out image of the related object into the multi-layer neural network may be used. When the multi-layer neural network outputs a feature amount having a size of WxHxC, the feature amount corresponding to the tracking target and the feature amount associated with another detection result are obtained from each part of WxHxC. Then, the multi-layer neural network is trained so that the index of the feature distance (such as L1·L2 norm) is proportional to the similarity of the target for each of them.

[0060] <Effect of the First Embodiment> According to the first embodiment, during the tracking of the tracking target object, the occlusion determination of the tracking target object can be performed using the features of the related object of the tracking target object, and further, by searching for the related object after occlusion, the correct tracking of the tracking target object can be realized. For example, when a horse is the object to be tracked and there are multiple horses, it may be difficult to track them because they look similar to each other. Furthermore, in tracking that requires the autofocus function and real-time performance of a digital camera, the calculation time available for processing the imaging image for which the tracking result is to be calculated is limited. Therefore, if only a single tracking method is used, it is even more difficult to distinguish between similar objects. In contrast, according to the present embodiment, a related object of the object to be tracked (for example, a rider on a horse) is linked to the object to be tracked, and the occlusion of the object to be tracked can be determined using the characteristics of the related object. Particularly when photographing horse races, although the horses are similar in color and shape, the riders on the horses wear easily distinguishable uniforms and the like. Therefore, even if it is difficult to determine whether the original object to be tracked is in front or behind when the horses cross each other, it is easy to determine whether it is in front or behind based on the color characteristics of the rider, and occlusion can be determined. Furthermore, even if the tracking result has shifted to an incorrect object to be tracked when the occlusion ends, it is possible to find out where the original object to be tracked is using the characteristics of a more distinguishable related object (rider). Note that not limited to horses and riders, when a motorcycle is the object to be tracked, the driver (rider) can be used as the related object, or when a track athlete is the object to be tracked and the bib is used as the related object, occlusion determination of the object to be tracked and persistent tracking of the original object to be tracked can be realized.

[0061] <Second Embodiment> In the first embodiment, when performing occlusion determination and related object feature matching, the features of the related object were used as they were for processing. However, for example, when a rider is occluded when riders on horses are very similar to each other, there is a possibility of incorrect tracking due to matching with an incorrect related object (rider). Therefore, in the second embodiment, an example of determining whether occlusion determination or related object feature matching is effective from the features of the related object will be described.

[0062] FIG. 6 is a diagram showing the functional configuration of the image processing apparatus 600 according to the second embodiment. Note that the hardware configuration of the information processing apparatus to which the image processing apparatus 600 according to the second embodiment can be applied is the same as that shown in FIG. 1(a) described above, and the illustration and description thereof are omitted. The image processing apparatus 600 according to the second embodiment includes a use determination unit 601 in addition to the functional units shown in FIG. 1(b) described above. The imaging device 110, the image acquisition unit 101, the tracking unit 102, the object detection unit 103, the association unit 104, the feature acquisition unit 105, the occlusion determination unit 106, the matching unit 107, the integration unit 108, and the output unit 120 have the same configurations as described above, and thus their descriptions are omitted.

[0063] The use determination unit 601 receives the features of the related object acquired by the feature acquisition unit 105 and the detection results of each candidate object by the object detection unit 103, and determines whether to perform the occlusion determination process by the occlusion determination unit 106 and the related object feature matching process by the matching unit 107. The use determination unit 601 determines whether to perform the occlusion determination process by the occlusion determination unit 106 and the related object feature matching process by the matching unit 107 based on whether the features of the related object acquired by the feature acquisition unit 105 satisfy a predetermined criterion. The details of the use determination based on the predetermined criterion will be described later.

[0064] Then, the use determination unit 601 sends the determination result as to whether to perform the occlusion determination process and the related object feature matching process to the occlusion determination unit 106 and the matching unit 107. When the occlusion determination unit 106 and the matching unit 107 receive the determination result from the use determination unit 601 that they should perform the occlusion determination process and the related object feature matching process, they perform the same occlusion determination process and related object feature matching process as those in the first embodiment described above.

[0065] FIG. 7 is a flowchart showing the flow of image processing in the image processing apparatus 600 according to the second embodiment. The processing according to the flowchart of FIG. 7 is also realized by the CPU 131 shown in FIG. 1(a) executing the image processing program according to the present embodiment. Also in the second embodiment, as in the first embodiment, the trailing part 102 and the object detection part 103 are both configured by a multi-layer neural network for explanation. S200 to S213 in FIG. 7 are the same processing steps as S200 to S213 in FIG. 2, and their explanations are omitted.

[0066] In the case of the flowchart of FIG. 7, after the related object feature calculation process of S208, the use determination unit 601 performs a related object feature use determination process as the process of S701, and further determines whether the shielding determination can be effectively used as the process of the next S702. The use determination unit 601 receives, as the process of S701, the features of the related object acquired by the feature acquisition unit 105 and the detection result by the object detection unit 103, and determines whether the features of the related object can be effectively used in each of the shielding determination process and the related object feature matching process. Furthermore, the use determination unit 601 determines, as the process of S702, whether a determination result is obtained that the related object feature can be effectively used in the shielding determination in the determination in S701. When a determination result that it can be effectively used is obtained in the use determination unit 601, the shielding determination unit 106 performs the same processes of S209 and S210 as described above.

[0067] Also, when it is determined as shielding in S210, the use determination unit 601 determines, as the process of S703, whether a determination result is obtained that the related object feature can be effectively used in the related object feature matching as a result of the determination in S701. That is, when it is determined as shielding in the shielding determination process of S209, the use determination unit 601 refers to the result of the feature use determination process in S701 and determines whether the features of the related object can be effectively used in the related object feature matching. When a determination result that it can be effectively used is obtained in the use determination unit 601, the matching unit 107 performs the same process of S211 as described above.

[0068] On the other hand, when a determination result is obtained that the related object feature cannot be effectively used for occlusion determination in S702 or that the related object feature cannot be effectively used for related object feature matching in S703, the integration unit 108 outputs the tracking result of S205 in the process of S212. That is, in this case, the occlusion determination process and the related object feature matching process are not performed, and the tracking result of S205 is output from the integration unit 108.

[0069] <Related Object Feature Usage Determination Process> Next, the related object feature usage determination process performed by the usage determination unit 601 in S701, S702, and S703 will be described with reference to FIGS. 8(a) to 8(c). In the example of FIG. 8(a), the usage determination unit 601 performs a usage determination using either one or both of the detection results 801 and 802 of the related object and the candidate object (person) that are the targets of the feature usage determination, and the features 803 and 804 calculated from those detection results. For example, when the features 803 and 804 of the related object and the candidate object are color histogram features and the Bhattacharyya distance between these features has a difference greater than or equal to a threshold value, the usage determination unit 601 determines that it is in a state effective for the occlusion determination process and the matching process in the feature usage determination. When using the color histogram feature in this way, the usage determination unit 601 uses whether the Bhattacharyya distance between the features has a difference less than the threshold value as a predetermined criterion when determining whether to perform the occlusion determination process and the matching process. As a result, as in the input image 810 of FIG. 8(b), when not only the tracking target but also the colors and shapes of the detection results 807 and 808 of the related object and the candidate object are similar, it is possible to suppress the occlusion determination and the related object feature matching from being performed, and reduce the risk of false matching.

[0070] That is, for example, when considering an input image 800 showing a horse and a rider as shown in FIG. 8(a), in a situation such as a racecourse, the color of the rider (such as the uniform and helmet) often varies from rider to rider as in the input image 800. In this case, related object feature matching processing using features that emphasize color, such as color histogram features, can be performed. On the other hand, for example, in a situation like the input image 810 shown in FIG. 8(b) where the colors of the riders are the same, since the differences in features that emphasize color become similar, the possibility of false matching increases during related object feature matching. Therefore, when there is not enough difference between the features of the detection results of the related object and the candidate object, the usage determination unit 601 outputs a usage determination result indicating that occlusion determination and related object feature matching are not performed, thereby suppressing the occurrence of false matching in related object feature matching.

[0071] In this embodiment, a color histogram feature that emphasizes color features and matching using Bhattacharyya distance using the same are used. However, by performing usage determination according to the features obtained in related object feature calculation, the effect of improving the tracking accuracy can be obtained more effectively. Also, when feature extraction by a multi-layer neural network is performed in related object feature calculation, feature usage determination can be performed according to the similarity calculation method between the features. Also, when the feature distance (L1·L2 norm) described in the first embodiment is proportional to the similarity, the effect of suppressing false matching can also be obtained by performing related object feature matching when the feature distance is separated by a certain amount or more and not performing it otherwise.

[0072] 8(c), the use determination unit 601 may set a search range 824 within a certain distance from the tracking result in the feature use determination in S701, and perform feature use determination based only on features obtained from detection results 822 and 823 having their centers within this search range 824. When detection results 821, 822, and 823 of a related object and multiple candidate objects exist, as in the input image 820 of FIG. 8(c), it becomes possible to make a determination by excluding the detection result 821 that is clearly far from the tracking target and is not related. This makes it possible to use only the most plausible related object for determination, even when related objects and candidate objects are crowded together.

[0073] <Effects of the second embodiment> As described above, according to the second embodiment, similar to the first embodiment, more accurate tracking can be achieved by using the features of related objects of the tracking target, and moreover, accurate tracking can be achieved by suppressing erroneous matching in related object feature matching.

[0074] Although the first and second embodiments have been described above as an example in which there is one tracking target object, there may be multiple tracking target objects. In this case, the above-described related object linking, occlusion determination, matching processing, tracking result integration processing, use determination processing, etc. may be performed for each of the multiple tracking target objects.

[0075] The present invention can also be realized by supplying a program that realizes one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more of the functions. The above-described embodiments are merely examples of specific implementations of the present invention, and the technical scope of the present invention should not be construed as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

[0076] The disclosure of this embodiment includes the following configurations, methods, and programs. (Configuration 1) Detection means for detecting, from an image, a first region to be tracked and a second region associated with the first region; Determination means for determining whether the first region is blocked based on the second region; Tracking means for tracking the first region after it is determined by the determination means that the first region is blocked, based on the determination result of the blocking of the first region and the second region; An image processing apparatus characterized by comprising the above. (Configuration 2) The image processing apparatus according to Configuration 1, further comprising association means for determining, from among third regions different from the first region detected from the image by the detection means, the second region associated with the first region. (Configuration 3) The image processing apparatus according to Configuration 1 or 2, wherein the second region associated with the first region is a region in which at least one of the relative position between the first region and the second region and the relative size of the second region with respect to the first region satisfies a predetermined condition. (Configuration 4) Feature acquisition means for acquiring features of the second region and features of the third region; Usage determination means for determining whether the features of the second region and the features of the third region acquired by the feature acquisition means satisfy a predetermined criterion; The image processing apparatus according to Configuration 2, further comprising the above, wherein the determination means determines whether the first region is blocked when it is determined by the usage determination means that the features satisfy a predetermined criterion. (Configuration 5) The image processing apparatus according to any one of Configurations 1 to 4, wherein the determination means determines that the first region is blocked when the second region is not detected by the detection means. (Configuration 6) It has feature acquisition means for acquiring features of the second region, The determination means determines whether the first region is blocked based on the features of the second region acquired by the feature acquisition means, and is the image processing apparatus according to any one of Configurations 1 to 4. (Configuration 7) The image processing apparatus according to Configuration 6, wherein the determination means determines that the first region is blocked when a temporal change in the features of the second region acquired by the feature acquisition means exceeds a predetermined threshold. (Configuration 8) Feature acquisition means for acquiring features of the second region, The image processing apparatus according to Configuration 5, further comprising matching means for performing matching between the features acquired from the second region and the features acquired from the second region before it is determined that the first region is blocked when it is determined by the determination means that the first region is blocked. (Configuration 9) The image processing apparatus according to Configuration 8, wherein the tracking means tracks the first region associated with the second region when the matching result by the matching means satisfies a predetermined criterion. (Configuration 10) The image processing apparatus according to Configuration 9, wherein the tracking means tracks the first region detected by the detection means from an image region based on the position of the second region as the first region associated with the second region. (Configuration 11) The detection means detects a third region different from the first region from the image, The feature acquisition means acquires features of the second region and features of the third region, The image processing apparatus according to Configuration 8, wherein the matching means performs the matching when the features of the second region and the features of the third region acquired by the feature acquisition means satisfy a predetermined criterion. (Configuration 12) The image processing apparatus according to configuration 4 or 11, wherein when the predetermined standard is satisfied, the difference between the feature of the second area and the feature of the third area acquired by the feature acquisition means is equal to or greater than a predetermined threshold value. (Method 1) A detection step of detecting, from an image, a first area to be tracked and a second area associated with the first area; A determination step of determining whether the first area is blocked based on the second area; A tracking step of tracking the first area after it is determined by the determination step that the first area is blocked, based on the determination result of the blocking of the first area and the second area; An image processing method, characterized by comprising the above. (Program 1) A program for causing a computer to function as the image processing apparatus according to any one of configurations 1 to 12.

Explanation of Signs

[0077] 100: Image processing apparatus, 101: Image acquisition unit, 102: Tracking unit, 103: Object detection unit, 104: Linking unit, 105: Feature acquisition unit, 106: Occlusion determination unit, 107: Matching unit, 108: Integration unit, 120: Output unit

Claims

1. Detection means for detecting, from an image, a first region to be tracked and a second region associated with the first region; Determination means for determining whether the first region is blocked based on the second region; Tracking means for tracking the first region after it is determined by the determination means that the first region is blocked, based on the determination result of the blocking of the first region and the second region; An image processing apparatus characterized by comprising the above.

2. The image processing apparatus according to claim 1, further comprising association means for determining the second region associated with the first region from among third regions different from the first region detected from the image by the detection means.

3. The image processing apparatus according to claim 1, wherein the second region associated with the first region is a region in which at least one of the relative position between the first region and the second region and the relative size of the second region with respect to the first region satisfies a predetermined condition.

4. Feature acquisition means for acquiring features of the second region and features of the third region; Use determination means for determining whether the features of the second region and the features of the third region acquired by the feature acquisition means satisfy a predetermined criterion; comprising The determination means determines whether the first region is blocked when it is determined by the use determination means that the features satisfy a predetermined criterion. The image processing apparatus according to claim 2.

5. The image processing apparatus according to claim 1, wherein the determination means determines that the first region is blocked when the second region is not detected by the detection means.

6. Feature acquisition means for acquiring features of the second region; The determination means determines whether the first region is blocked based on the features of the second region acquired by the feature acquisition means. The image processing apparatus according to claim 1.

7. The image processing apparatus according to claim 6, wherein the determination means determines that the first region is blocked when a temporal change in the features of the second region acquired by the feature acquisition means exceeds a predetermined threshold.

8. Feature acquisition means for acquiring features of the second region; When it is determined by the determination means that the first area is blocked, the image processing apparatus according to claim 5, further comprising matching means for performing matching between the feature obtained from the second area and the feature obtained from the second area before it is determined that the first area is blocked.

9. The image processing apparatus according to claim 8, wherein the tracking means tracks the first area associated with the second area when the matching result by the matching means satisfies a predetermined criterion.

10. The image processing apparatus according to claim 9, wherein the tracking means tracks, as the first area associated with the second area, the first area detected by the detection means from an image area based on the position of the second area.

11. The detection means detects a third area different from the first area from the image, The feature acquisition means acquires the feature of the second area and the feature of the third area, The image processing apparatus according to claim 8, wherein the matching means performs the matching when the feature of the second area and the feature of the third area acquired by the feature acquisition means satisfy a predetermined criterion.

12. The case where the predetermined criterion is satisfied is the case where the difference between the feature of the second area and the feature of the third area acquired by the feature acquisition means is equal to or greater than a predetermined threshold value, according to claim 4 or 11. Image processing apparatus described.

13. A detection step of detecting, from an image, a first area to be tracked and a second area associated with the first area; A determination step of determining whether the first area is blocked based on the second area; A tracking step of tracking the first area after it is determined by the determination step that the first area is blocked, based on the determination result of the shielding of the first area and the second area; An image processing method, characterized by comprising:

14. A computer, Detection means for detecting, from an image, a first area to be tracked and a second area associated with the first area; Determination means for determining whether the first area is blocked based on the second area; Tracking means for tracking the first area after it has been determined by the determination means that the first area is blocked, based on the determination result of the blocking of the first area and the second area; A program for causing an image processing apparatus to function as such.

Images