Information processing apparatus, information processing method, and computer-readable storage medium

Abstract

The present application achieves an information processing apparatus, an information processing method, and a computer-readable storage medium that can accurately recognize information of an object from a captured image regardless of a capturing condition. A first circumscribed polygon region calculating section (32) calculates a first circumscribed polygon region that is a convex hull circumscribed region containing a feature region. An inertia principal axis calculating section (33) calculates an inertia principal axis of the feature region. A region dividing section (34) divides the feature region using the inertia principal axis. A region selecting section (36) selects either the first circumscribed polygon region or either of two second circumscribed polygon regions that are convex hull circumscribed regions containing the divided feature region, based on an area ratio of the first circumscribed polygon region to the two second circumscribed polygon regions. An object recognizing section (37) performs object detection processing on the selected first circumscribed polygon region or either of the two second circumscribed polygon regions.

Description

Technical Field

[0001] The present disclosure relates to an information processing device, an information processing method, and a computer-readable storage medium storing a control program for processing a captured image. Background Art

[0002] In shooting devices such as digital cameras, it is known to perform predetermined information processing (data processing) on ​​the captured images, thereby detecting objects such as human bodies from the captured images and displaying the detected objects or a frame surrounding the objects on a display device such as a display unit.

[0003] In such a camera, it is required to identify an object and its shadow in a captured image. Patent Document 1 discloses a technique for processing a captured image based on a predetermined reference brightness value to identify an object, such as a human body, and its shadow.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Laid-Open No. 2008-245063 Summary of the Invention

[0007] Problems to be solved by the invention

[0008] However, the conventional techniques described above may not accurately identify an object in a captured image, depending on the shooting conditions. Specifically, in conventional techniques, in situations such as low ambient brightness or backlighting, the brightness values ​​of an object and its shadow in the captured image may be close to each other, making it impossible to separate the object and the shadow, and thus, the object may not be accurately identified in the captured image.

[0009] The present disclosure has been made in view of the above-mentioned problems, and an object of the present disclosure is to provide an information processing device, an information processing method, and a control program that can accurately recognize an object from a captured image regardless of the capturing conditions.

[0010] Technical means to solve the problem

[0011] In order to solve the above-mentioned problems, the present disclosure adopts the following structure.

[0012] An information processing device according to one aspect of the present disclosure processes a captured image, the information processing device comprising: a feature area detection unit for detecting a feature area having specified features from an input captured image; a first circumscribed polygonal area calculation unit for calculating a first circumscribed polygonal area as a convex hull circumscribed area containing the feature area; a principal axis of inertia calculation unit for calculating the principal axis of inertia of the feature area; an area segmentation unit for segmenting the feature area using the calculated principal axis of inertia; a second circumscribed polygonal area calculation unit for calculating two second circumscribed polygonal areas as convex hull circumscribed areas, each containing the segmented feature area; an area selection unit for selecting the first circumscribed polygonal area and any one of the two second circumscribed polygonal areas based on an area ratio of the two second circumscribed polygonal areas to the first circumscribed polygonal area; and an object recognition unit for performing object detection processing on the first circumscribed polygonal area or the two second circumscribed polygonal areas selected by the area selection unit.

[0013] Moreover, an information processing method according to one aspect of the present disclosure processes a captured image, and the information processing method includes: a feature area detection process for detecting a feature area having specified features from an input captured image; a first circumscribed polygonal area calculation process for calculating a first circumscribed polygonal area as a convex hull circumscribed area containing the feature area; a principal axis of inertia calculation process for calculating the principal axis of inertia of the feature area; an area segmentation process for segmenting the feature area using the calculated principal axis of inertia; a second circumscribed polygonal area calculation process for calculating two second circumscribed polygonal areas as convex hull circumscribed areas that respectively contain the segmented feature area; an area selection process for selecting the first circumscribed polygonal area and any one of the two second circumscribed polygonal areas based on an area ratio of the two second circumscribed polygonal areas to the first circumscribed polygonal area; and an object recognition process for performing object detection processing on the first circumscribed polygonal area or the two second circumscribed polygonal areas selected by the area selection process.

[0014] Moreover, one aspect of the present disclosure is a computer-readable storage medium that stores a control program for enabling a computer to function as an information processing device, wherein the computer-readable storage medium stores a control program for enabling a computer to function as the feature area detection unit, the first circumscribed polygon area calculation unit, the principal axis of inertia calculation unit, the area segmentation unit, the second circumscribed polygon area calculation unit, the area selection unit, and the object recognition unit.

[0015] Effects of the Invention

[0016] According to the present disclosure, it is possible to provide an information processing device, an information processing method, and a control program that can accurately recognize an object from a captured image regardless of the capturing conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a block diagram showing a configuration example of an imaging device according to one embodiment of the present disclosure.

[0018] Figure 2 This is a flowchart showing an example of the operation of the imaging device.

[0019] Figure 3 It is a diagram for explaining a specific operation example of the imaging device.

[0020] Figure 4 It is a diagram illustrating another specific operation example of the imaging device.

[0021] Figure 5 This is a diagram illustrating an example of the effect of the imaging device.

[0022] Figure 6 It is a diagram for explaining the problem in the comparative example.

[0023] Figure 7 It is a diagram illustrating another specific operation example of the imaging device.

[0024] [Explanation of Symbols]

[0025] 1: Shooting device

[0026] 3: Control Department

[0027] 31: Feature area detection unit

[0028] 32: First circumscribed polygon area calculation unit

[0029] 33: Principal axis of inertia calculation unit

[0030] 34: Region Segmentation

[0031] 35: Second circumscribed polygon area calculation unit

[0032] 36: Regional Selection Department

[0033] 37: Object Recognition Unit DETAILED DESCRIPTION

[0034] [Implementation Method]

[0035] §1 Application

[0036] First, use Figure 1 An example of a scenario to which the present disclosure is applicable will be described. Figure 1This is a block diagram illustrating an example configuration of a camera device according to one embodiment of the present disclosure. The following description illustrates the application of the present disclosure to a camera device such as a digital camera. However, the present disclosure is not limited to any information processing device that processes captured images. The present disclosure may also be applied to portable devices such as tablets and smartphones, or to information processing terminals such as personal computers that independently include a camera unit for outputting captured images.

[0037] like Figure 1 As shown, the imaging device 1 of this embodiment includes an imaging unit 2 and a control unit 3 that processes the captured image from the imaging unit 2. The control unit 3 includes a feature region detection unit 31, a first circumscribed polygon region calculation unit 32, a principal axis of inertia calculation unit 33, a region segmentation unit 34, a second circumscribed polygon region calculation unit 35, a region selection unit 36, and an object recognition unit 37.

[0038] The characteristic region detection unit 31 detects characteristic regions having predetermined characteristics from the input captured image. The first circumscribed polygonal region calculation unit 32 calculates a first circumscribed polygonal region as a convex hull circumscribed region that includes the characteristic region. The principal axis of inertia calculation unit 33 calculates the principal axis of inertia of the characteristic region. The convex hull circumscribed region is a convex region in the captured image that is demarcated by a straight line (circumscribed line) connecting two adjacent pixels located at the outermost edge of the characteristic region, so as to include all pixels contained in the characteristic region.

[0039] The region segmentation unit 34 uses the calculated principal axis of inertia to segment the feature region into two. The second circumscribed polygonal region calculation unit 35 calculates two second circumscribed polygonal regions as convex hull circumscribed regions, each containing the segmented feature region. The region selection unit 36 ​​selects either the first circumscribed polygonal region or the two second circumscribed polygonal regions based on the area ratio of the two second circumscribed polygonal regions to the first circumscribed polygonal region. The object recognition unit 37 performs object detection processing on the selected first circumscribed polygonal region or the two second circumscribed polygonal regions.

[0040] In this manner, the control unit 3 calculates a first circumscribed polygonal region (convex hull circumscribed region) containing the feature region from the input captured image. Furthermore, the control unit 3 calculates the principal axis of inertia of the feature region and uses the calculated principal axis of inertia to segment the feature region. Furthermore, the control unit 3 calculates two second circumscribed polygonal regions (convex hull circumscribed regions) each containing the segmented feature region. Furthermore, the control unit 3 selects a convex hull circumscribed region as the target of object detection processing from the two second circumscribed polygonal regions after segmentation or the first circumscribed polygonal region before segmentation, and performs object detection processing on the selected convex hull circumscribed region, thereby identifying an object such as a human body from the input captured image.

[0041] Therefore, according to this embodiment, the camera 1 detects a feature region having predetermined characteristics, calculates the principal axis of inertia of the detected feature region, and uses the principal axis of inertia to segment the detected feature region. Furthermore, the convex hull circumscribed region that includes the corresponding feature region is used in the first circumscribed polygonal region that includes the feature region before segmentation and the second circumscribed polygonal region that includes the two feature regions after segmentation. This allows the region selection unit to select the first circumscribed polygonal region or the two second circumscribed polygonal regions appropriately and accurately, enabling the object detection process in the object recognition unit to be performed with high precision. As a result, the camera 1 can accurately identify objects in captured images regardless of the shooting conditions.

[0042] §2 Structure Example

[0043] <Regarding the Configuration of the Image Capturing Device 1>

[0044] like Figure 1 As shown, the imaging device 1 includes an imaging unit 2, a control unit 3, a display unit 4, and a storage unit 5. The imaging device 1 displays an image captured by the imaging unit 2 on the display unit 4 or stores it in the storage unit 5. The imaging device 1 performs predetermined (data) processing on the captured image through the control unit 3, thereby recognizing a predetermined object such as a human body contained in the captured image, and displays the recognition result on the display unit 4 or stores it in the storage unit 5.

[0045] The imaging unit 2 includes an imaging element (not shown) and captures a moving image or a still image in accordance with a user's operation instruction on the imaging device 1. The imaging unit 2 outputs the captured moving image or still image to the control unit 3.

[0046] The display unit 4 includes a liquid crystal display panel or a light emitting diode, and displays predetermined information. The display unit 4 may be configured to function as an input unit for receiving instructions from the user by being integrally configured with a touch panel, for example.

[0047] The storage unit 5 stores various data used by the control unit 3. Furthermore, the storage unit 5 may store various software that, when executed by the computer, causes the computer to function as the control unit 3. Furthermore, the storage unit 5 stores threshold values ​​used in the area selection process described later in the area selection unit 36.

[0048] The control unit 3 is a computing device that has the function of comprehensively controlling each component of the imaging device 1. For example, the control unit 3 can control each component of the imaging device 1 by having one or more processors (e.g., a central processing unit (CPU)) execute programs stored in one or more memories (e.g., random access memory (RAM) or read-only memory (ROM)). Furthermore, as described above, the control unit 3 includes multiple functional blocks, each of which performs predetermined data processing.

[0049] Here, the structure of the control unit 3 is described. The control unit 3 includes a characteristic region detection unit 31 that detects characteristic regions having predetermined characteristics from an input captured image; a first circumscribed polygon region calculation unit 32 that calculates the first circumscribed polygon region; and a principal axis of inertia calculation unit 33 that calculates the principal axis of inertia of the characteristic region.

[0050] The control unit 3 includes: an area segmentation unit 34, which uses the inertia principal axis calculated by the inertia principal axis calculation unit 33 to segment the feature area detected by the feature area detection unit 31; and a second circumscribed polygonal area calculation unit 35, which calculates two second circumscribed polygonal areas, and the two second circumscribed polygonal areas respectively contain the segmented feature areas from the area segmentation unit 34.

[0051] The control unit 3 includes: an area selection unit 36, which selects the first circumscribed polygonal area calculated by the first circumscribed polygonal area calculation unit 32 and any one of the two second circumscribed polygonal areas calculated by the second circumscribed polygonal area calculation unit 35; and an object recognition unit 37, which recognizes objects from the first circumscribed polygonal area or the two second circumscribed polygonal areas selected by the area selection unit 36.

[0052] §3 Action Examples

[0053] <Operation Example of Image Capture Device 1>

[0054] Also refer to Figures 2 to 4 An operation example of the imaging device 1 according to this embodiment will be described in detail. Figure 2 This is a flowchart showing an example of the operation of the imaging device. Figure 3 It is a diagram for explaining a specific operation example of the imaging device. Figure 4 It is a diagram illustrating another specific operation example of the imaging device.

[0055] Figure 2 In step S1 , the characteristic region detecting unit 31 extracts a characteristic region having predetermined characteristics from the captured image input from the capturing unit 2 using, for example, a difference method.

[0056] Next, in step S2 , the first circumscribed polygon area calculation unit 32 creates a bounding box that includes the characteristic area detected by the characteristic area detection unit 31 .

[0057] Next, in step S3, the principal axis of inertia calculation unit 33 calculates the center of gravity of the feature region and the center of the bounding box. Furthermore, the principal axis of inertia calculation unit 33 calculates the first principal axis of inertia based on the center of gravity of the feature region, and further calculates the deflection angle θ of the first principal axis of inertia. As described later, the deflection angle θ refers to, for example, the angle of the principal axis of inertia relative to one of the horizontal axis (X-axis) and the vertical axis (Y-axis) in the captured image, for example, the X-axis.

[0058] Next, in step S4 , the principal axis of inertia calculation unit 33 adds 90 degrees to the deflection angle θ obtained in step S3 to correct the deflection angle θ and obtain a second principal axis of inertia having an inclination 90 degrees different from the first principal axis of inertia obtained in step S3 .

[0059] Next, in step S5 , the principal axis of inertia calculation unit 33 calculates a principal axis of inertia more suitable for the captured image, based on the center of the calculated bounding box, from among the first and second principal axes of inertia respectively obtained in steps S3 and S4 .

[0060] Next, in step S6 , the first circumscribed polygonal area calculation unit 32 calculates the convex hull circumscribed area of ​​the entire characteristic area extracted by the characteristic area detection unit 31 .

[0061] Next, in step S7, the region segmentation unit 34 divides the characteristic region extracted in step S1 into two using the principal axis of inertia calculated in step S5. Subsequently, the second circumscribed polygonal region calculation unit 35 creates convex hull circumscribed regions for each of the two divided characteristic regions.

[0062] Next, in step S8 , the region selection unit 36 ​​determines whether the area ratio of the two convex hull circumscribed regions after segmentation created in step S7 to the convex hull circumscribed region before segmentation obtained in step S6 is equal to or smaller than a preset threshold.

[0063] If the region selection unit 36 ​​determines that the area ratio is equal to or smaller than the threshold value (Yes in S8 ), the region selection unit 36 ​​selects two divided convex hull circumscribed regions (step S9 ) and outputs them to the object recognition unit 37 .

[0064] On the other hand, if the region selection unit 36 ​​determines that the area ratio exceeds the threshold (No in S8 ), the region selection unit 36 ​​selects the convex hull circumscribed region before segmentation obtained in step S6 (step S10 ) and outputs it to the object recognition unit 37 .

[0065] Next, in step Sll, the object recognition section 37 performs object detection processing on the two divided convex hull regions or the pre-division convex hull region selected by the region selection section 36.

[0066] In addition, instead of the description of step S10, the region selection section 36 can select a pre-division feature region or a bounding box including the pre-division feature region, instead of the pre-division convex hull region.

[0067] Here, using Figure 3 and Figure 4 a more specific example of the operation of the imaging device 1 of the present embodiment will be described.

[0068] As shown in Figure 3 , the feature region detection section 31 detects a feature region 101 of, for example, a moving object pixel including a human body and a moving object pixel of a shadow thereof in the input captured image. The feature region detection section 31 detects at least one of, for example, a region in which activity is present, a region having a pixel value (luminance value of a pixel) within a prescribed range, and a region surrounded by an edge, as the feature region 101 from the input captured image.

[0069] That is, the feature region detection section 31 detects a feature such as "activity is present in a region", "a region has a pixel value within a prescribed range", or "a region is surrounded by an edge" as a prescribed feature. Also, the pixel value within the prescribed range refers to, for example, a range of pixel values that a prescribed object such as a human body can take. Also, the luminance and the like of the surroundings of an object changes depending on the time, and thus the pixel value within the prescribed range can also change depending on the time.

[0070] Also, the feature region detection section 31 detects the feature region 101 including a prescribed object, that is, a moving object, using, for example, a difference method. Specifically, the feature region detection section 31 determines the presence or absence of activity using a background difference method or an inter-frame difference method to detect a feature region 101 of a moving object pixel including a human body and a moving object pixel of a shadow.

[0071] In the background difference method, for example, a pixel in which the pixel value difference (absolute value) from a prescribed background image is equal to or greater than a prescribed value in a captured image is detected as a pixel in which activity is present (a moving object pixel).

[0072] In the inter-frame difference method, for example, a pixel in which a pixel value difference between a current captured image (current frame) and a past captured image (past frame) is equal to or greater than a predetermined value is detected as a pixel in which activity exists (moving object pixel). In the inter-frame difference method, for example, the past frame is a frame that is a predetermined number of frames before the current frame, and the predetermined number is one or more. The predetermined number can also be determined in accordance with a frame rate of information processing (information processing speed) in the control section 3 or a frame rate of the capturing process (capturing processing speed) performed by the capturing section 2, or the like.

[0073] Next, the first circumscribed polygon region calculation section 32 calculates, for example, a bounding box having a rectangular shape that circumscribes the feature region 101. Specifically, the first circumscribed polygon region calculation section 32 calculates a bounding box G of moving object pixels of a human body and a moving object pixel of a shadow in a region that contains the feature region 101. The bounding box G has a rectangular shape that contains the feature region by a straight line parallel to the horizontal axis and a straight line parallel to the vertical axis of the captured image.

[0074] Next, the inertia principal axis calculation section 33 calculates an inertia principal axis of the feature region detected by the feature region detection section 31. Specifically, the inertia principal axis calculation section 33 calculates a center of gravity GC of the feature region 101 detected in the captured image, and calculates a center Gl of the calculated bounding box G. Furthermore, the inertia principal axis calculation section 33 calculates a first inertia principal axis Kl of the feature region 101 that passes through the center of gravity GC.

[0075] Furthermore, the inertia principal axis calculation section 33 calculates an angle of inclination Θ of the first inertia principal axis Kl with respect to the X axis, and further calculates a second inertia principal axis K2 that intersects the X axis at an angle in which the angle of inclination Θ is added by 90 degrees (angle of inclination (Θ + 90)), that is, that is orthogonal to the first inertia principal axis Kl.

[0076] Furthermore, the inertia principal axis calculation section 33 determines one of the first inertia principal axis Kl and the second inertia principal axis K2 of the feature region 101 that is used to divide the convex hull circumscribed region calculated in step S6, on the basis of a positional relationship of the center of gravity GC of the feature region 101 with respect to the center Gl of the bounding box G.

[0077] Specifically, when the center of gravity GC is located in the upper right portion (first quadrant) or the lower left portion (third quadrant) when the bounding box G is divided into four by the X axis that passes through the center Gl of the bounding box G and the Y axis that is orthogonal to the X axis, the inertia principal axis calculation section 33 selects the first inertia principal axis Kl. Furthermore, when the center of gravity GC is located in the lower right portion (fourth quadrant) or the upper left portion (second quadrant), the inertia principal axis calculation section 33 selects the second inertia principal axis K2. That is, in the case of the feature region 101 in FIG. 3, the first inertia principal axis Kl is selected. Figure 3In the example shown, since the center of gravity GC is located in the lower right portion, the principal axis of inertia calculation unit 33 selects the second principal axis of inertia K2 as the principal axis of inertia for dividing the convex hull circumscribed region.

[0078] In this manner, the principal axis of inertia calculation unit 33 determines the principal axis of inertia with which to divide the convex hull circumscribed region (first circumscribed polygonal region) from among the two principal axes of inertia located within the circumscribed polygonal region for calculation of the principal axis of inertia, based on the positional relationship between the center G1 of the bounding box G in the captured image and the centroid GC of the feature region. Consequently, the imaging device 1 of this embodiment can more appropriately divide the first circumscribed polygonal region, thereby enabling high-precision recognition of objects contained within the first circumscribed polygonal region.

[0079] Alternatively, the principal axis of inertia calculation unit 33 may determine the principal axis of inertia based on the positional relationship between the centroid of the feature region detected by the feature region detection unit and the centroid of the convex hull circumscribed region calculated by the first circumscribed polygon region calculation unit 32. Furthermore, the method for calculating the principal axis of inertia in the principal axis of inertia calculation unit 33 is not limited to the two aforementioned methods; a known calculation method may be employed.

[0080] like Figure 4 As shown, the first circumscribed polygon area calculation unit 32 calculates a characteristic area 101 ( Figure 3 ), the feature region 101 of the entirety of the moving object pixels includes the moving object pixels of the human body and the moving object pixels of its shadow. The convex hull circumscribed region T is the smallest convex polygon (convex polyhedron) that encompasses all pixels contained in the feature region 101. A convex polygon herein refers to a polygon containing only vertices with interior angles less than 180°. More specifically, the convex hull circumscribed region T includes a shape where a straight line connects the outermost adjacent pixels of the pixels contained in the feature region 101.

[0081] Next, the region segmentation unit 34 segments the feature region using the second principal axis of inertia K2 calculated by the principal axis of inertia calculation unit 33. The second circumscribed polygonal region calculation unit 35 calculates a convex hull circumscribed region T1, which includes the feature region containing the moving object pixels of a human body, and a convex hull circumscribed region T2, which includes the feature region containing the moving object pixels of a shadow.

[0082] Furthermore, if the region selection unit 36 ​​determines that the area ratio (i.e., the pixel count ratio) obtained by dividing the total area of ​​the two convex hull circumscribed regions T1 and T2 after segmentation by the area of ​​the convex hull circumscribed region T before segmentation is less than or equal to the threshold, the region selection unit 36 ​​selects the two convex hull circumscribed regions T1 and T2 and outputs them to the object recognition unit 37. Furthermore, if the region selection unit 36 ​​determines that the area ratio exceeds the threshold, the region selection unit 36 ​​selects the convex hull circumscribed region T before segmentation and outputs it to the object recognition unit 37. Furthermore, if there is any overlap between the convex hull circumscribed regions T1 and T2 in the total area, the area of ​​the overlap is subtracted from the sum of the areas of the convex hull circumscribed regions T1 and T2.

[0083] In this manner, the region selection unit 36 ​​compares the ratio of the total area of ​​the two convex hull circumscribed regions T1 and T2 created after segmentation to the area of ​​the convex hull circumscribed region T before segmentation with a predetermined threshold value, thereby selecting a convex hull circumscribed region for object detection in the object recognition unit 37. Consequently, the camera 1 of this embodiment can, for example, recognize a human body with high accuracy. Specifically, when a human body is included in a feature region, segmentation of the feature region can determine a segmented region (convex hull circumscribed region T1) that is closer to the region of the moving object pixel containing the human body.

[0084] Next, the object recognition unit 37 performs object detection on the convex hull circumscribed region from the region selection unit 36, thereby identifying whether the object contained in the convex hull circumscribed region is a predetermined object such as a human body. Specifically, the object recognition unit 37 performs object detection processing by combining image features such as the Histogram of Oriented Gradient (HoG) or Haar-like features with boosting. The object recognition unit 37 displays the recognition results on the display unit 4. This allows the display unit 4 to display the object recognition results along with the image captured by the imaging unit 2.

[0085] In addition to the above description, the object recognition unit 37 may also use a learned model generated through deep learning, such as a region-based convolutional neural network (R-CNN), a fast region-based convolutional neural network (Fast R-CNN), You Only Look Once (YOLO), or a single-shot multibox detector (SSD). Furthermore, objects recognized by the object recognition unit 37 include, in addition to humans, non-human animals, vehicles such as cars, and buildings such as houses.

[0086] As described above, in the imaging device 1 of this embodiment, the characteristic region detection unit 31 detects characteristic regions from the image captured by the imaging unit 2. The first circumscribed polygonal region calculation unit 32 calculates a first circumscribed polygonal region as a convex hull circumscribing region that includes the characteristic region, and a bounding box that includes the characteristic region. The principal axis of inertia calculation unit 33 calculates the principal axis of inertia of the characteristic region, and the region segmentation unit 34 uses the calculated principal axis of inertia to segment the characteristic region into two. The second circumscribed polygonal region calculation unit 35 calculates two second circumscribed polygonal regions as convex hull circumscribing regions that each include the segmented characteristic region. The region selection unit 36 ​​selects either the first circumscribed polygonal region or the two second circumscribed polygonal regions based on the area ratio of the two second circumscribed polygonal regions to the first circumscribed polygonal region. The object recognition unit 37 performs object detection processing on the selected first circumscribed polygonal region or the two second circumscribed polygonal regions. This allows the region selection unit 36 ​​to select the first circumscribed polygonal region or the two second circumscribed polygonal regions appropriately and accurately. As a result, the object recognition unit 37 can perform the object detection process with high accuracy. Therefore, the imaging device 1 of this embodiment can accurately recognize an object from a captured image regardless of the imaging conditions.

[0087] Moreover, in this embodiment, the first circumscribed polygonal area calculation unit 32 further calculates the bounding box containing the feature area, and the principal axis of inertia calculation unit 33 calculates the principal axis of inertia of the feature area based on the center of gravity of the feature area and the center of the bounding box. Therefore, the calculation accuracy of the principal axis of inertia can be easily improved, and even the recognition accuracy of the object can be easily improved.

[0088] Further, in the description, the structure in which the first outer polygonal region calculating section 32 calculates the bounding box and the convex hull outer region is described, but the present embodiment is not limited thereto, and for example, the structure in which the first outer polygonal region calculating section 32 calculates only the convex hull outer region, and the bounding box is calculated by a separately provided bounding box calculating section can be adopted.

[0089] Here, the specific effects when the convex hull outer region is used as the first outer polygonal region in the imaging device 1 of the present embodiment are described using Figure 5 and Figure 6 . Figure 5 is a diagram illustrating an example of the effects in the imaging device. Figure 6 is a diagram illustrating a problem in a comparative example.

[0090] As illustrated in Figure 5 , in the case where, for example, the shadow of the human body exists at an angle other than 90 degrees, for example, at an angle smaller than 90 degrees, with respect to the human body, in the captured image from the imaging section 2, the first outer polygonal region calculating section 32 calculates the convex hull outer region Tt that contains the feature region of the entire moving object pixels including the moving object pixels of the human body and the moving object pixels of the shadow. The region dividing section 34 divides the feature region using the inertia principal axis K3, and the second outer polygonal region calculating section 35 calculates the convex hull outer region Ttl that contains the moving object pixels of the human body and the convex hull outer region Tt2 that contains the moving object pixels of the shadow. Further, in the captured image, the region that is subtracted from the convex hull outer region Tt by the convex hull outer regions Ttl and Tt2 becomes the convex hull outer region Tt3 that corresponds to the space between the moving object pixels of the human body and the moving object pixels of the shadow.

[0091] Thus, in the imaging device 1 of the present embodiment, the feature region is divided, and the area ratio of the convex hull outer region Tt before the division to the convex hull outer regions Ttl and Tt2 after the division is compared with the threshold value, whereby even in the case where the shadow of the human body exists at an angle other than 90 degrees, for example, at an angle smaller than 90 degrees, with respect to the human body, it is possible to calculate the convex hull outer region Ttl that is closer to the region containing the moving object pixels of the human body, and it is possible to calculate the convex hull outer region Tt2 that is closer to the region containing the moving object pixels of the shadow. Further, in the present embodiment, it is possible to set the convex hull outer region Tt3 that is closer to the space between the moving object pixels of the human body and the moving object pixels of the shadow. As a result, in the imaging device 1 of the present embodiment, it is possible to accurately recognize the human body and the shadow contained in the captured image.

[0092] Meanwhile, the verification results of a comparative example are shown below. In this comparative example, unlike the present embodiment, a bounding box is used as the first circumscribed polygonal area before segmentation or the second circumscribed polygonal area after segmentation, rather than a convex hull circumscribed area.

[0093] That is, Figure 6 As shown, in the comparative example, when the bounding box is divided using the principal axis of inertia, a bounding box circumscribed area HB1 containing moving object pixels of a human body and a bounding box circumscribed area HB2 containing moving object pixels of a shadow are newly calculated.

[0094] Moreover, in the circumscribed area HB2, as Figure 6 As shown by circles S1 and S2 in the figure, the area containing the moving object pixels is larger than the actual shadow. That is, in the comparative example, the circumscribed area HB2 has an area significantly larger than the actual shadow area, which adversely affects the area ratio calculation in the area selection unit 36. Consequently, the area to be output to the object recognition unit 37 cannot be correctly selected, making it difficult for the object recognition unit 37 to accurately recognize the human body and shadow contained in the captured image. That is, compared to the comparative example, the present embodiment, which uses the convex hull circumscribed area for area ratio calculation, can accurately recognize the human body and shadow even when the shadow is at an angle other than 90 degrees relative to the human body.

[0095] In addition, in the above description, the case where a human body is identified as a predetermined object is described as an example, but the imaging device 1 of this embodiment is not limited to this. Figure 7 As shown, the present invention is applicable to a captured image including, for example, a vehicle 202 in addition to the human body 201 .

[0096] Specifically, in the imaging device 1 of this embodiment, for the human body 201, since the area ratio is below the threshold, separate convex hull circumscribed regions are created for the moving object pixels containing the human body 201 and for the moving object pixels containing the shadow of the human body 201. Therefore, the display unit 4 displays, for example, a frame 203 containing the human body 201 and a frame 204 containing the shadow of the human body 201 in the display image P1.

[0097] Furthermore, for vehicle 202, since the area ratio exceeds the threshold, the convex hull circumscribed region containing the moving object pixels of vehicle 202 and the convex hull circumscribed region containing the moving object pixels of the shadow of vehicle 202 are not divided. Therefore, on display unit 4, for example, a frame 205 containing both vehicle 202 and its shadow is displayed.

[0098] [Example of implementation using software]

[0099] The functional blocks of the imaging device 1 (particularly the control unit 3 ) may be realized by a logic circuit (hardware) formed on an integrated circuit (IC chip) or the like, or by software.

[0100] In the latter case, the control unit 3 includes a computer that executes commands of software, i.e., a program, that implements each function. The computer may include, for example, one or more processors and a computer-readable recording medium storing the program. The computer then reads and executes the program from the recording medium via the processor, thereby achieving the objectives of this disclosure.

[0101] As the processor, for example, a central processing unit (CPU) can be used. As the recording medium, a "non-transitory tangible medium" can be used, such as a read-only memory (ROM), a magnetic disk, a card, a semiconductor memory, a programmable logic circuit, etc. Furthermore, a random access memory (RAM) for developing the program can also be included.

[0102] Furthermore, the program may be provided to the computer via any transmission medium capable of transmitting the program (communication network or broadcast waves, etc.). In addition, one aspect of the present invention may be implemented in the form of a data signal embedded in a carrier wave, embodying the program via electronic transmission.

[0103] 〔Summarize〕

[0104] An information processing device according to one aspect of the present disclosure processes a captured image, the information processing device comprising: a feature area detection unit for detecting a feature area having specified features from an input captured image; a first circumscribed polygonal area calculation unit for calculating a first circumscribed polygonal area as a convex hull circumscribed area containing the feature area; a principal axis of inertia calculation unit for calculating the principal axis of inertia of the feature area; an area segmentation unit for segmenting the feature area using the calculated principal axis of inertia; a second circumscribed polygonal area calculation unit for calculating two second circumscribed polygonal areas as convex hull circumscribed areas, each containing the segmented feature area; an area selection unit for selecting the first circumscribed polygonal area and any one of the two second circumscribed polygonal areas based on an area ratio of the two second circumscribed polygonal areas to the first circumscribed polygonal area; and an object recognition unit for performing object detection processing on the first circumscribed polygonal area or the two second circumscribed polygonal areas selected by the area selection unit.

[0105] According to this configuration, a feature region having predetermined characteristics is detected, the principal axis of inertia of the detected feature region is calculated, and the detected feature region is segmented using the principal axis of inertia. Furthermore, the convex hull circumscribed region that includes the corresponding feature region is used in the first circumscribed polygonal region that includes the feature region before segmentation and the second circumscribed polygonal region that includes the two feature regions after segmentation. This allows the region selection unit to select the first circumscribed polygonal region or the two second circumscribed polygonal regions appropriately and accurately, enabling the object detection process in the object recognition unit to be performed with high precision. As a result, objects can be accurately recognized from captured images regardless of the shooting conditions.

[0106] In the information processing device of the one aspect, the first circumscribed polygonal area calculation unit may further calculate a bounding box including the feature area, and the principal axis of inertia calculation unit may calculate the principal axis of inertia based on a positional relationship between the center of gravity of the feature area and the center of the bounding box or the center of gravity of the convex hull circumscribed area.

[0107] According to the above configuration, the principal axis of inertia of the characteristic region can be calculated with high accuracy. As a result, an object can be recognized more accurately from a captured image regardless of the capturing conditions.

[0108] In the information processing device according to the above aspect, the object recognition unit may recognize a human body by performing object detection processing.

[0109] According to the above configuration, the object recognition unit can recognize a human body with high accuracy.

[0110] In the information processing device according to the aspect, the characteristic region detecting unit may detect at least one of a region where motion exists, a region having pixel values ​​within a predetermined range, and a region surrounded by an edge as the characteristic region.

[0111] According to the above configuration, the characteristic region can be accurately detected regardless of the shooting conditions, and the object can be recognized more accurately from the captured image.

[0112] Moreover, an information processing method according to one aspect of the present disclosure processes a captured image, and the information processing method includes: a feature area detection process for detecting a feature area having specified features from an input captured image; a first circumscribed polygonal area calculation process for calculating a first circumscribed polygonal area as a convex hull circumscribed area containing the feature area; a principal axis of inertia calculation process for calculating the principal axis of inertia of the feature area; an area segmentation process for segmenting the feature area using the calculated principal axis of inertia; a second circumscribed polygonal area calculation process for calculating two second circumscribed polygonal areas as convex hull circumscribed areas that respectively contain the segmented feature area; an area selection process for selecting the first circumscribed polygonal area and any one of the two second circumscribed polygonal areas based on an area ratio of the two second circumscribed polygonal areas to the first circumscribed polygonal area; and an object recognition process for performing object detection processing on the first circumscribed polygonal area or the two second circumscribed polygonal areas selected by the area selection process.

[0113] According to this configuration, a feature region having predetermined characteristics is detected, the principal axis of inertia of the detected feature region is calculated, and the detected feature region is segmented using the principal axis of inertia. Furthermore, the convex hull circumscribed region that includes the corresponding feature region is used in the first circumscribed polygonal region that includes the feature region before segmentation and the second circumscribed polygonal region that includes the two feature regions after segmentation. This allows for appropriate and precise selection of the first circumscribed polygonal region or the two second circumscribed polygonal regions in the region selection step, enabling high-precision object detection in the object recognition step. As a result, objects can be accurately recognized from captured images regardless of the capture conditions.

[0114] Moreover, a control program in one aspect of the present disclosure is used to enable a computer to function as an information processing device, wherein the control program is used to enable the computer to function as the feature area detection unit, the first circumscribed polygon area calculation unit, the principal axis of inertia calculation unit, the area segmentation unit, the second circumscribed polygon area calculation unit, the area selection unit, and the object recognition unit.

[0115] According to the above configuration, an object can be accurately recognized from a captured image regardless of the capturing conditions.

[0116] The present disclosure is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims. Embodiments obtained by appropriately combining the technical components disclosed in the embodiments are also included in the technical scope of the present disclosure.

Claims

1. An information processing device for processing captured images, the information processing device comprising: a characteristic region detecting unit for detecting a characteristic region having a predetermined characteristic from an input captured image; a first circumscribed polygonal area calculation unit, which calculates a first circumscribed polygonal area as a convex hull circumscribed area including the feature area; a principal axis of inertia calculating unit for calculating the principal axis of inertia of the characteristic region; a region segmentation unit that segments the characteristic region using the calculated principal axis of inertia; A second circumscribed polygonal area calculation unit calculates two second circumscribed polygonal areas as convex hull circumscribed areas, each of which includes the segmented feature area; a region selection unit that selects the first circumscribed polygonal region and any one of the two second circumscribed polygonal regions based on an area ratio of the two second circumscribed polygonal regions to the first circumscribed polygonal region; as well as The object recognition unit performs object detection processing on the first circumscribed polygonal area or the two second circumscribed polygonal areas selected by the area selection unit.

2. The information processing device according to claim 1, wherein the first circumscribed polygon area calculation unit further calculates a bounding box including the feature area. The principal axis of inertia calculation unit calculates the principal axis of inertia based on a positional relationship between the centroid of the characteristic region and the center of the bounding box or the centroid of the convex hull circumscribed region. 3 . The information processing device according to claim 1 , wherein the object recognition section recognizes a human body by performing object detection processing. 4 . The information processing device according to claim 1 , wherein the characteristic region detecting unit detects at least one of a region where movement occurs, a region having pixel values ​​within a predetermined range, and a region surrounded by an edge as the characteristic region.

5. An information processing method for processing a captured image, the information processing method comprising: A feature region detection step of detecting a feature region having specified features from an input captured image; a first circumscribed polygonal region calculating step of calculating a first circumscribed polygonal region as a convex hull circumscribed region including the characteristic region; a principal axis of inertia calculation step of calculating the principal axis of inertia of the characteristic area; a region segmentation step of segmenting the characteristic region using the calculated principal axis of inertia; A second circumscribed polygonal region calculation step of calculating two second circumscribed polygonal regions as convex hull circumscribed regions, each of which includes the segmented feature region; a region selection step of selecting the first circumscribed polygonal region and any one of the two second circumscribed polygonal regions based on an area ratio of the two second circumscribed polygonal regions to the first circumscribed polygonal region; as well as The object recognition step performs object detection processing on the first circumscribed polygonal area or the two second circumscribed polygonal areas selected in the area selection step.

6. A computer-readable storage medium storing a control program for causing a computer to function as the information processing apparatus according to claim 1, wherein: The control program is used to cause a computer to function as the characteristic region detection unit, the first circumscribed polygon region calculation unit, the principal axis of inertia calculation unit, the region division unit, the second circumscribed polygon region calculation unit, the region selection unit, and the object recognition unit.

Images