Information processing device, information processing method, and information processing program

Abstract

To assist imaging of a specific object.SOLUTION: An information processing device 10 according to this embodiment includes a specific object region detecting unit 20B, an unexpected appearing region detecting unit 20C, and a determining unit 20D. The specific object region detecting unit 20B detects, from a picked-up image, a specific object region of a specific object. The unexpected appearing region detecting unit 20C detects, from the picked-up image, an unexpected appearing region of the specific object. The determining unit 20D determines a non-unexpected appearing region of the specific object on the basis of the specific object region and the unexpected appearing region.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] An embodiment of the present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] A technique for detecting a subject included in a photographed image has been disclosed. For example, a technique for detecting parts of the subject included in the photographed image and determining whether or not there are any undetected parts among the multiple parts that make up the subject has been disclosed.

[0003] However, conventional techniques simply determine whether or not there are undetected parts, and it is difficult to assist in photographing so that the entire specific target area, such as a specific part of the subject, is captured in the image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2020 / 217812 Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention has been made in view of the above, and has an object to provide an information processing device, an information processing method, and an information processing program that can assist in photographing a specific target. [Means for solving the problem]

[0006] According to an embodiment, an information processing apparatus includes a specific target region detection unit, a reflection region detection unit, and a determination unit. The specific target region detection unit detects a specific target region of a specific target from a captured image. The reflection region detection unit detects a reflection region of the specific target from the captured image. The determination unit The department A non-image area of ​​the identified target is determined based on the identified target area and the image area. The specific target area is an area that includes the entire specific target and includes an area where the specific target is hidden by an obstruction that appears in the captured image. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram showing a configuration of an information processing system. [Figure 2A] FIG. 10 is an explanatory diagram of detection of a specific target region. [Figure 2B] FIG. 10 is an explanatory diagram of detection of a specific target region. [Figure 2C] FIG. 10 is an explanatory diagram of detection of a specific target region. [Figure 3A] FIG. 4 is an explanatory diagram of detection of an inclusion area. [Figure 3B] FIG. 4 is an explanatory diagram of detection of an inclusion area. [Figure 3C] FIG. 4 is an explanatory diagram of detection of an inclusion area. [Figure 4A] FIG. 10 is an explanatory diagram of determining a hidden area. [Figure 4B] FIG. [Figure 5] FIG. 4 is a schematic diagram of an example of a display screen. [Figure 6] 10 is a flowchart showing the flow of information processing. [Figure 7] Hardware configuration diagram. DETAILED DESCRIPTION OF THE INVENTION

[0008] An information processing device, an information processing method, and an information processing program will be described in detail below with reference to the accompanying drawings.

[0009] FIG. 1 is a block diagram showing an example of the configuration of an information processing system 1 according to this embodiment.

[0010] The information processing system 1 includes an information processing device 10.

[0011] The information processing device 10 is an information processing device for assisting in photographing a specific target. The specific target will be described in detail later.

[0012] The information processing device 10 includes an image capturing unit 12, a storage unit 14, a communication unit 16, a UI (user interface) unit 18, and a control unit 20. The image capturing unit 12, the storage unit 14, the communication unit 16, the UI unit 18, and the control unit 20 are communicatively connected via a bus 19 or the like.

[0013] The photographing unit 12 acquires photographed image data by photographing. In the following description, the photographed image data will be referred to as a photographed image. The storage unit 14 stores various types of information.

[0014] The communication unit 16 is a communication interface for communicating with an external information processing device of the information processing device 10. For example, the communication unit 16 communicates with an external information processing device or electronic device via a wired network such as Ethernet (registered trademark), a wireless network such as Wi-Fi (Wireless Fidelity) or Bluetooth (registered trademark), or the like.

[0015] The UI section 18 includes an output section 18A and an input section 18B.

[0016] The output unit 18A outputs various types of information. The output unit 18A is, for example, a display unit, a speaker, a projection device, etc. The input unit 18B accepts operation instructions from a user. The input unit 18B is, for example, a pointing device such as a mouse or a touchpad, a keyboard, etc. The UI unit 18 may be a touch panel in which the output unit 18A and the input unit 18B are integrally configured.

[0017] The control unit 20 executes information processing in the information processing device 10. The control unit 20 includes an acquisition unit 20A, a specific target region detection unit 20B, an image region detection unit 20C, a determination unit 20D, a specification unit 20E, and an output control unit 20F.

[0018] The acquisition unit 20A, the specific target area detection unit 20B, the image area detection unit 20C, the determination unit 20D, the identification unit 20E, and the output control unit 20F are realized, for example, by one or more processors. For example, each of the above units may be realized by having a processor such as a CPU (Central Processing Unit) execute a program, i.e., by software. Each of the above units may be realized by a processor such as a dedicated IC, i.e., by hardware. Each of the above units may be realized by a combination of software and hardware. When multiple processors are used, each processor may realize one of the units, or two or more of the units.

[0019] At least one of the above-described units included in the control unit 20 may be mounted on an external information processing device communicatively connected to the information processing device 10 via a network or the like. At least one of the various pieces of information stored in the storage unit 14 may be stored in an external storage device communicatively connected to the information processing device 10 via a network or the like. At least one of the photographing unit 12, the storage unit 14, and the UI unit 18 may be mounted on an external information processing device communicatively connected to the information processing device 10. In this case, a system including the externally mounted components and the information processing device 10 may be configured as the information processing system 1.

[0020] The acquisition unit 20A acquires a captured image to be processed. For example, the acquisition unit 20A acquires a captured image captured by the imaging unit 12 from the imaging unit 12. The acquisition unit 20A may acquire the captured image from the storage unit 14. Alternatively, the acquisition unit 20A may acquire the captured image from an external information processing device connected to the information processing device 10 via the communication unit 16 and a network.

[0021] The specific target region detection unit 20B detects a specific target region of a specific target from the captured image. The specific target region detection unit 20B detects a specific target region from the captured image acquired by the acquisition unit 20A. Detecting a specific target region means detecting the position, size, and range of the specific target region in the captured image.

[0022] A specific object refers to a specific element contained in the object.

[0023] The target is a subject for which photography support is required. The target may be either a living or non-living object. Living objects include, for example, people, plants, animals such as dogs, and cells. Non-living objects include structures such as bridges and buildings, moving objects such as automobiles, and non-moving objects. In this embodiment, an example will be described in which the target is a person.

[0024] The specific target is at least a partial element of the target. When the target is a person, the specific target may be, for example, a part of the person or the entire body of the person. Specifically, when the target is a person, the specific target may be any of the person's head, upper body, lower body, legs, or entire body. When the target is a bridge, the specific target may be, for example, the entire bridge or a pier that forms part of the bridge. The specific target may be determined in advance depending on the target to which the photography assistance is applied. The specific target may also be changeable as appropriate by a user's operation instruction on the input unit 18B. In this embodiment, a form in which the specific target is the entire body of a person will be described as an example.

[0025] The specific target area is an area that includes both the area of ​​the specific target that is captured in the captured image and the area that is not captured in the captured image. More specifically, the specific target area is an area that includes the entire specific target and is included in a virtual enlarged area that is an enlarged capture range of the captured image.

[0026] Assume that the target is a person and the specific target is the person's entire body. Also assume that a portion of the person's entire body is cut off from the captured image, and that a portion of the person's entire body is hidden by another object or the like in the captured image. In this case, the specific target region is a region that includes the entire body of the person, including both the cut-off region and the hidden region of the person's entire body.

[0027] The specific target region may be any region that includes the specific target, and its shape is not limited. For example, the specific target region may be a region within a frame line that follows the outline of the specific target, or a region within a frame line of a predetermined shape that surrounds the specific target. The shape of the frame line that surrounds the specific target is not limited. For example, the shape of the frame line representing the specific target region may be any of a polygonal shape, a rectangular shape, an ellipse, a circle, etc.

[0028] The specific target region is preferably a region surrounded by a frame that circumscribes at least a portion of the specific target. For example, the specific target region is preferably a region surrounded by a frame that at least a portion of which contacts the outline of a person included in the captured image.

[0029] In this embodiment, a specific target region is described as a rectangular region surrounded by a rectangular frame that surrounds the entire body of a person, which is an example of a specific target, and circumscribes at least a portion of the outline of the person included in the captured image. That is, in this embodiment, a specific target region is described as a rectangular region surrounded by a rectangular frame that represents the position, size, and range relative to the captured image.

[0030] 2A to 2C are explanatory diagrams of an example of detection of specific target region 32. FIG.

[0031] 2A is an explanatory diagram of an example in which the acquisition unit 20 A acquires a photographed image 30 A. The photographed image 30 A is an example of the photographed image 30.

[0032] For example, consider a situation in which the acquisition unit 20A acquires a captured image 30A in which the legs of a person, who is an example of a specific target P, are hidden by an obstruction C. In this case, the specific target region detection unit 20B detects, from the captured image 30A, a rectangular region that includes the region of the specific target P hidden by the obstruction C and that surrounds the entire body of the person, who is an example of a specific target P, as a specific target region 32A.

[0033] 2B is an explanatory diagram of an example in which the acquisition unit 20 A acquires a photographed image 30 B. The photographed image 30 B is an example of the photographed image 30.

[0034] For example, consider a situation in which the acquisition unit 20A acquires a photographed image 30B in which the legs of the whole body of a person, who is an example of a specific target P, are cut off from the photographed image 30. In this case, the specific target region detection unit 20B detects, from the photographed image 30B, as a specific target region 32B, a rectangular region that is included in a virtual enlarged region 35 obtained by enlarging the shooting range of the photographed image 30B, includes the cut-off region that is cut off from the photographed image 30B, and surrounds the whole body of the person, who is an example of a specific target P.

[0035] 2C is an explanatory diagram of an example in which the acquisition unit 20A acquires a photographed image 30C. The photographed image 30C is an example of the photographed image 30.

[0036] For example, consider a situation in which the acquisition unit 20A acquires a photographed image 30C in which the entire body of a person, who is an example of a specific target P, is captured in the photographed image 30C and is not hidden by an obstruction C or the like. In this case, the specific target region detection unit 20B detects a rectangular region surrounding the entire body of the person, who is an example of a specific target P, from the photographed image 30C as a specific target region 32C.

[0037] Through these processes, the specific target region detection unit 20B detects, from the photographed image 30, a specific target region 32 which is a rectangular region surrounded by a rectangular frame line that indicates the position, size, and range relative to the photographed image 30.

[0038] Returning to Figure 1, we continue the explanation.

[0039] The specific target region detection unit 20B detects a specific target region 32 from the captured image 30 using the first learning model.

[0040] The first learning model used to detect the specific target region 32 is a model that receives the captured image 30 as input and outputs the specific target region 32. For example, the first learning model may be a machine learning model that outputs a rectangular region surrounding the entire body of a person, which is an example of the specific target P, as the specific target region 32 that is the classification result of the captured image 30.

[0041] The first learning model used to detect the specific target region 32 may be learned in advance and stored in the storage unit 14. The learning model used to detect the specific target region 32 may be a model using a known machine learning method such as a convolution neural network (CNN), semantic segmentation, or a support vector machine (SVM).

[0042] For example, consider a learning model using the machine learning method described in Non-Patent Document 1. Non-Patent Document 1 discloses that a detector that outputs a desired rectangle is trained by performing training using training data consisting of multiple pairs of training images and correct rectangles. Furthermore, in a typical object detection process, when a rectangular region that includes an area that protrudes from the training image is detected as an output rectangle, the protruding area is clipped from the output rectangle and then output as an output rectangle representing the detection result. Therefore, in this embodiment, a first learning model that detects the specific target region 32 of this embodiment from the captured image 30 can be trained in advance by further training so as not to perform the clipping process.

[0043] Non-patent document 1: Ren, Shaoqing, Kaiming He, Ross Girshick, and Jian Sun. "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks." Advances in Neural Information Processing Systems. Vol. 28, 2015.

[0044] The specific target area detection unit 20B inputs the photographed image 30 acquired by the acquisition unit 20A into a first learning model used to detect the specific target area 32, and thereby detects the specific target area 32 as an output from the first learning model. That is, by inputting the photographed image 30 into the first learning model, the specific target area detection unit 20B detects information about a rectangular area representing the position, size, and range relative to the photographed image 30 as the specific target area 32.

[0045] The image area detection unit 20C detects the image area of ​​the specific object P from the photographed image 30. The image area detection unit 20C detects the image area from the photographed image 30 acquired by the acquisition unit 20A. Detecting the image area means detecting the position, size, and range of the image area relative to the photographed image 30.

[0046] The reflected area refers to an area of ​​the specific target P that is reflected in the photographed image 30. In detail, the reflected area refers to an area of ​​the specific target P that is not hidden by an obstruction C or the like in the photographed image 30, that is, an area that is reflected as an image in the photographed image 30.

[0047] The captured area may be an area that appears in the photographed image 30 of the specific target P, and its shape is not limited. For example, the captured area may be an area within a frame line that follows the outline of the part that appears in the photographed image 30 of the specific target P, or an area within a frame line of a predetermined shape that surrounds the part that appears in the photographed image 30 of the specific target P. The shape of the frame line that surrounds the part that appears in the photographed image 30 of the specific target P is not limited. For example, the shape of the frame line that represents the captured area may be any of a polygonal shape, a rectangle, an ellipse, a circle, etc.

[0048] Furthermore, the shape of the frame line representing the captured area may be the same as or different from the shape of the frame line representing the specific target area 32. In this embodiment, an example will be described in which the captured area is an area surrounded by a rectangular frame line that surrounds the area of ​​the specific target P that is captured in the captured image 30. That is, in this embodiment, an example will be described in which the captured area is represented by a rectangular area surrounded by a rectangular frame line that represents the position, size, and range relative to the captured image 30.

[0049] In this embodiment, a rectangular shape is used as an example of the shape of the frame lines representing the captured area and the specific target area 32. It goes without saying that at least one of the aspect ratio and size of the rectangular frame lines representing the captured area and the specific target area 32 may differ depending on the specific target P captured in the captured image 30.

[0050] 3A to 3C are explanatory diagrams of an example of detection of the projection area 34. FIG.

[0051] 3A is an explanatory diagram of an example in which the acquisition unit 20 A acquires a photographed image 30 A. The photographed image 30 A shown in FIG. 3A and FIG.

[0052] For example, consider a situation in which acquisition unit 20A acquires photographed image 30A. In this case, photographed area detection unit 20C detects, from photographed image 30A, a region of the entire body of a person, which is an example of specific target P, that is photographed in photographed image 30, i.e., a rectangular region that surrounds the region in photographed image 30A that is not hidden by obstruction C, as photographed area 34A.

[0053] 3B is an explanatory diagram of an example in which the acquisition unit 20A acquires a photographed image 30B. The photographed image 30B shown in FIG. 3B and FIG.

[0054] For example, consider a situation in which acquisition unit 20A acquires photographed image 30B. In this case, photographed area detection unit 20C detects, from photographed image 30B, a rectangular area surrounding an area of ​​the entire body of a person, which is an example of specific target P, that is photographed in photographed image 30, as photographed area 34B.

[0055] 3C is an explanatory diagram of an example in which the acquisition unit 20A acquires a photographed image 30C. The photographed images 30C shown in FIG. 3C and FIG.

[0056] For example, consider a situation in which acquisition unit 20A acquires photographed image 30C. In this case, specific target region detection unit 20B detects, from photographed image 30C, a rectangular region surrounding a region of the entire body of a person, which is an example of specific target P, that is reflected in photographed image 30, as reflected region 34C.

[0057] Through these processes, the image area detection unit 20C detects an image area 34 from the photographed image 30, which is a rectangular area surrounded by a rectangular frame line that indicates the position, size, and range relative to the photographed image 30.

[0058] Continuing the explanation, returning to Figure 1, the inclusion area detection unit 20C detects the inclusion area 34 from the captured image 30 using the second learning model.

[0059] The second learning model used to detect the captured image 34 is a model that receives the captured image 30 as input and outputs the captured image 34. For example, the second learning model may be a machine learning model that outputs a rectangular area surrounding the part of the specific target P that is captured in the captured image 30 as the captured image 34, which is the classification result, from the captured image 30.

[0060] The second learning model used to detect the image area 34 may be learned in advance and stored in the storage unit 14. The second learning model used to detect the image area 34 may be a model using a known machine learning method such as CNN, semantic segmentation, or SVM. Specifically, the second learning model may be a learning model using the machine learning method shown in Non-Patent Document 1, for example.

[0061] The captured image detection unit 20C inputs the captured image 30 acquired by the acquisition unit 20A into a second learning model used to detect the captured image 34, and thereby detects the captured image 34 as an output from the second learning model. That is, by inputting the captured image 30 into the second learning model, the captured image detection unit 20C detects, as the captured image 34, information about a rectangular area that represents the position, size, and range of the captured image 30.

[0062] Next, the determination unit 20D will be described.

[0063] The determination unit 20D determines the non-image area of ​​the identified target P based on the identified target area 32 and the image area .

[0064] The non-imaged region refers to a region of the specific target P that is not captured in the captured image 30. In detail, the non-imaged region is a region of the specific target P that includes at least one of a hidden region that is hidden in the captured image 30 by an obstruction C or the like, and a cut-off region that is cut off from the captured image 30.

[0065] The hidden region refers to a region of the specific target P that is not captured in the captured image 30 because it is hidden by another object such as an obstruction C in the captured image 30. In other words, the hidden region is a region of the specific target P that exists within the angle of view of the captured image 30 but is located in the blind spot of the imaging unit 12 due to another object such as the obstruction C.

[0066] For example, assume a scene in which the acquisition unit 20A acquires a captured image 30A (see FIGS. 2A and 3A). As shown in FIGS. 2A and 3A, the captured image 30A is a captured image 30 in which the legs of a person, who is an example of a specific target P, are hidden by an obstruction C. In this case, the area of ​​the specific target P that is hidden by the obstruction C in the captured image 30A corresponds to the hidden area. In the case of the captured image 30A shown in FIGS. 2A and 3A, the area of ​​the legs of the person, who is an example of a specific target P, that is hidden by the obstruction C corresponds to the hidden area.

[0067] The cut-off region means a region of the specific target P that is cut off from the photographed image 30. In other words, the cut-off region means a region of the specific target P that is outside the photographic angle of view of the photographed image 30. In other words, the cut-off region is a region of the specific target P that is outside the photographic angle of view of the photographed image 30.

[0068] For example, consider a scene in which the acquisition unit 20A acquires a photographed image 30B (see FIGS. 2B and 3B). As shown in FIGS. 2B and 3B, the photographed image 30B is a photographed image 30 in which the legs of a person, who is an example of a specific target P, are cut off from the photographed image 30. In this case, the area of ​​the specific target P that is cut off from the photographed image 30B corresponds to the cut-off area. In the case of the photographed image 30B shown in FIGS. 2B and 3B, the area of ​​part of the legs, which is the area of ​​the whole body of the person, who is an example of a specific target P, that is cut off from the photographed image 30B, corresponds to the cut-off area.

[0069] Based on the specific target area 32 and the captured area 34, the determination unit 20D determines a non-captured area that includes at least one of a hidden area of ​​the specific target P included in the photographed image 30 and a cut-off area of ​​the specific target P that is cut off from the photographed image 30. The determination unit 20D uses the photographed image 30, the specific target area 32 detected from the photographed image 30, and the captured area 34 detected from the photographed image 30 to determine a non-captured area that includes at least one of a hidden area and a cut-off area.

[0070] In detail, determination unit 20D compares identified target region 32 detected by identified target region detection unit 20B with captured region 34 detected by captured region detection unit 20C. If there is a non-overlapping region between identified target region 32 and captured region 34 in captured image 30, determination unit 20D determines that the captured image 30 includes a hidden region. Then, determination unit 20D determines that the non-overlapping region between identified target region 32 and captured region 34 in captured image 30 is a hidden region.

[0071] Fig. 4A is an explanatory diagram of an example of determining a hidden area 36. For example, consider a scene in which acquisition unit 20A acquires captured image 30A. Captured images 30A shown in Figs. 4A, 2A, and 3A are examples of the same captured image 30. Also consider a scene in which specific target area detection unit 20B identifies specific target area 32A from captured image 30A, and captured area detection unit 20C identifies captured area 34A from captured image 30A.

[0072] In this case, determination unit 20D compares identified target area 32A with captured area 34A and determines whether a non-overlapping area exists between identified target area 32A and captured area 34A within captured image 30A. That is, determination unit 20D determines whether there are differences between the position, size, and range represented by identified target area 32A and the position, size, and range represented by captured area 34A. If a non-overlapping area is included between identified target area 32A and captured area 34A, determination unit 20D determines that a hidden area exists and identifies the non-overlapping area as hidden area 36.

[0073] Furthermore, the determination unit 20D compares the captured image 30 with the specific target region 32 detected by the specific target region detection unit 20B. The determination unit 20D determines whether or not a non-overlapping region that does not overlap with the captured image 30 exists within the specific target region 32. If a non-overlapping region that does not overlap with the captured image 30 exists within the specific target region 32, the determination unit 20D determines that the captured image 30 includes a border region. Then, the determination unit 20D determines that the non-overlapping region in the specific target region 32 that does not overlap with the captured image 30 is a border region.

[0074] Fig. 4B is an explanatory diagram of an example of determining the edge region 38. For example, consider a scene in which the acquisition unit 20A acquires a captured image 30B. The captured images 30B shown in Figs. 4B, 2B, and 3B are examples of the same captured image 30. Also consider a scene in which the specific target region detection unit 20B identifies a specific target region 32B from the captured image 30B, and the captured region detection unit 20C identifies a captured region 34B from the captured image 30B.

[0075] In this case, the determination unit 20D compares the captured image 30B with the identified target region 32, and determines whether or not a non-overlapping region that does not overlap with the captured image 30B exists within the identified target region 32B. That is, the determination unit 20D determines whether or not at least a portion of the identified target region 32B exists outside the image frame 31 of the captured image 30B. If a non-overlapping region that does not overlap with the captured image 30B exists within the identified target region 32B, the determination unit 20D determines that a cutoff region 38 exists, and determines the non-overlapping region as the cutoff region 38. That is, the determination unit 20D determines that a region within the identified target region 32B that is located outside the image frame 31 of the captured image 30B is the cutoff region 38.

[0076] Through these processes, the determination unit 20D determines at least one of the hidden area 36 and the border area 38 as a non-image area 37 using the captured image 30, the specific target area 32, and the image area 34.

[0077] Returning to Figure 1, we continue the explanation.

[0078] The specifying unit 20E specifies a recommended photographing environment in which the non-photographic area 37 determined by the determining unit 20D appears in the photographed image 30.

[0079] In detail, the specifying unit 20E specifies at least one of the recommended photographing angle of view in which the edge region 38 appears and the recommended photographing direction in which the hidden region 36 appears as the recommended photographing environment.

[0080] The identification unit 20E determines whether the determination unit 20D has determined that the edge region 38 is a non-image region 37. If the edge region 38 is determined to be a non-image region 37, the identification unit 20E performs at least one of enlargement, transformation, and movement processing on the image frame 31 of the captured image 30 so that the image frame 31 becomes a shooting angle of view that includes the entire identified target region 32 that has been determined to include the edge region 38, and identifies the frame as the recommended shooting angle of view.

[0081] The specification unit 20E may specify a recommended shooting angle of view depending on the function of the imaging unit 12 that captures the captured image 30. For example, assume that the imaging unit 12 has a zoom function. In this case, the specification unit 20E may specify, as the recommended shooting angle of view, a frame obtained by enlarging the image frame 31 of the captured image 30 so as to obtain a shooting angle of view that includes the entire identified target area 32 that has been determined to include the edge area 38. The specification unit 20E may, for example, associate the identification information of the imaging unit 12 with function information that indicates the function of the imaging unit 12 and store the associated information in advance in the storage unit 14. Then, when specifying the recommended shooting angle of view, the specification unit 20E may read the function information that corresponds to the identification information of the target imaging unit 12 and specify the recommended shooting angle of view.

[0082] Furthermore, the identification unit 20E determines whether the determination unit 20D has determined that the hidden area 36 is a non-image area 37. If the hidden area 36 is determined, the identification unit 20E identifies the shooting direction in which the hidden area 36 included in the identified target P is captured in the captured image 30 as the recommended shooting direction.

[0083] For example, when the hidden area 36 is determined, the identification unit 20E identifies a direction shifted by a predetermined shooting angle from the current shooting direction as the recommended shooting direction. The predetermined shooting angle may be, for example, 30°, 60°, 90°, etc., but is not limited to these values. The predetermined shooting angle may be changeable as appropriate by a user's operation instruction via the input unit 18B. Furthermore, the identification unit 20E may learn in advance a learning model that receives the captured image 30 and at least one of the hidden area 36 and the identified target area 32 as input, and outputs a recommended shooting direction, and identify the recommended shooting direction using the learning model.

[0084] Next, the output control unit 20F will be described.

[0085] The output control unit 20F outputs determination result information according to the determination result of the determination unit 20D to the output unit 18A. The output control unit 20F outputs at least one of a sound and an image representing the determination result information to the output unit 18A.

[0086] For example, the output control unit 20F outputs to the output unit 18A determination result information representing at least one of the non-image area 37 determined by the determination unit 20D, the specific target area 32 detected by the specific target area detection unit 20B, the image area 34 detected by the image area detection unit 20C, the recommended shooting environment identified by the identification unit 20E, and a warning indicating that the non-image area 37 is included in the captured image 30.

[0087] Specifically, the output control unit 20F displays a display screen including the determination result information on a display unit included in the output unit 18A.

[0088] 5 is a schematic diagram of an example of the display screen 40. The display screen 40 is a display screen that includes determination result information, and is an example of a display screen that is displayed on a display unit included in the output unit 18A.

[0089] For example, the output control unit 20F outputs the display screen 40 including the superimposed image 50 in which an image representing the determination result information is superimposed on the photographed image 30 to the output unit 18A.

[0090] In detail, the output control unit 20F creates a superimposed image 50 by superimposing on the captured image 30 at least one of an image 42 representing the specific target region 32, an image 44 representing the projected region 34, an image 46 representing the hidden region 36, and an image 48 representing the edge region 38. The output control unit 20F may further create at least one of an image 41 representing the recommended shooting angle of view and an image 43 representing the recommended shooting direction. The output control unit 20F then displays the superimposed image 50, which includes the superimposed image 50 and at least one of the image 41 representing the recommended shooting angle of view and the image 43 representing the recommended shooting direction, on a display unit included in the output unit 18A.

[0091] The display forms of the images 41, 43, 42, 44, 46, and 48 generated by the output control unit 20F are not limited to the example shown in Fig. 5. For example, as shown in Fig. 5, the output control unit 20F may use images representing the outer frames of the specific target area 32, the projection area 34, the hidden area 36, ​​and the border area 38 as images representing each of these areas. Furthermore, the output control unit 20F may use images in which the interior of each area is filled in as images representing each of these areas.

[0092] The output control unit 20F may generate a superimposed image 50 by further superimposing at least one of an image 41 representing the recommended shooting angle of view and an image 43 representing the recommended shooting direction on the captured image 30, and display the generated image on a display unit which is the output unit 18A.

[0093] In addition, when the specific target area 32 includes a non-image area 37, which is at least one of a hidden area 36 and a cut-off area 38, the output control unit 20F may display a display screen 40 on the display unit that is the output unit 18A, which further includes a warning image indicating that the non-image area 37 is included.

[0094] Furthermore, the output control unit 20F may output the sound representing the warning from a speaker included in the output unit 18A.

[0095] By outputting the determination result information to the output unit 18A, the output control unit 20F can provide information for adjusting the shooting environment so that the specific target region 32 appears in the captured image 30.

[0096] Next, an example of the flow of information processing executed by the information processing device 10 of this embodiment will be described.

[0097] FIG. 6 is a flowchart showing an example of the flow of information processing executed by the information processing device 10 of this embodiment.

[0098] The acquisition unit 20A acquires the photographed image 30 (step S100).

[0099] The specific target region detection unit 20B detects the specific target region 32 from the photographed image 30 acquired in step S100 (step S102). The inclusion region detection unit 20C detects the inclusion region 34 from the photographed image 30 acquired in step S100 (step S104).

[0100] Based on the specific target region 32 detected in step S102 and the captured region 34 detected in step S104, the determination unit 20D determines the non-captured region 37 of the specific target P (step S106). The determination unit 20D determines the non-captured region 37 to include at least one of the hidden region 36 and the edge region 38.

[0101] Next, the determination unit 20D determines whether or not the non-image area 37 was determined in step S106 (step S108). If the non-image area 37 could not be determined (step S108: No), the process proceeds to step S112, which will be described later. If the non-image area 37 was determined (step S108: Yes), the process proceeds to step S110.

[0102] In step S110, the identification unit 20E identifies a recommended shooting environment in which the non-image area 37 appears in the captured image 30, based on the captured image 30 acquired in step S100 and the non-image area 37 determined in step S106 (step S110).

[0103] The output control unit 20F outputs determination result information according to the determination result of step S106 to the output unit 18A (step S112). The output control unit 20F outputs determination result information indicating at least one of the non-image area 37 determined in step S108, the specific target area 32 detected in step S102, the image area 34 detected in step S104, the recommended shooting environment identified in step S110, and a warning indicating that the non-image area 37 is included in the captured image 30 to the output unit 18A. Then, this routine ends.

[0104] As described above, information processing device 10 of this embodiment includes identified target region detection unit 20B, captured region detection unit 20C, and determination unit 20D. Identified target region detection unit 20B detects identified target region 32 of identified target P from captured image 30. Captured region detection unit 20C detects captured region 34 of identified target P from captured image 30. Determination unit 20D determines non-captured region 37 of identified target P based on identified target region 32 and captured region 34.

[0105] Here, the prior art discloses a technique for detecting parts of a subject included in a photographed image 30 and determining whether or not there are any undetected parts among the multiple parts that make up the subject. However, the prior art only determines whether or not there are any undetected parts, and does not identify the area of ​​the undetected parts in the photographed image 30. For this reason, the prior art was unable to provide information that allows adjustment of the photographing environment so that the entire specific target area 32, such as a specific part of the subject, is captured in the photograph. In other words, the prior art has had difficulty in supporting the photographing of a specific target P.

[0106] On the other hand, in information processing device 10 of this embodiment, specific target area detection unit 20B detects specific target area 32, and captured area detection unit 20C detects captured area 34. Determination unit 20D determines non-captured area 37 of specific target P based on specific target area 32 and captured area 34 of specific target P detected from captured image 30.

[0107] In this way, the information processing device 10 of this embodiment does not simply detect the presence or absence of a specific part, but determines the non-image area 37 that is not captured in the photographed image 30 of the specific object P.

[0108] Therefore, by providing information representing the non-image area 37 determined by the determination unit 20D, the information processing device 10 of this embodiment can provide information for adjusting the shooting environment so that the non-image area 37 appears in the captured image 30. In other words, the information processing device 10 of this embodiment can provide information for adjusting the shooting environment so that the entire specific target P, including the non-image area 37, appears in the captured image 30.

[0109] Therefore, the information processing device 10 of this embodiment can assist in photographing the specific object P.

[0110] Furthermore, in the conventional technology, specific objects P are limited to people and automobiles that are made up of multiple parts, and it is difficult to apply the conventional technology to specific objects P that are difficult to represent by distinguishing between multiple parts. Furthermore, in the conventional technology, it is necessary to execute a detection process for each of the multiple parts, which can cause problems in terms of processing load and processing cost.

[0111] On the other hand, the information processing device 10 of this embodiment detects the specific target area 32 and the reflected area 34 of the specific target P from the captured image 30, regardless of whether the specific target P is composed of one or more parts, and determines the non-reflected area 37 of the specific target P based on the specific target area 32 and the reflected area 34.

[0112] Therefore, in addition to the above-mentioned effects, the information processing device 10 of this embodiment can reduce the processing load and the processing cost.

[0113] Furthermore, the information processing device 10 of this embodiment detects the specific target region 32 and the captured region 34 from the captured image 30, thereby determining the non-captured region 37 of the specific target P. Therefore, in addition to the above-described effects, the information processing device 10 of this embodiment can be applied to various specific targets P, not limited to people and automobiles.

[0114] The information processing device 10 of this embodiment also includes an output control unit 20F. The output control unit 20F outputs determination result information according to the determination result of the determination unit 20D to the output unit 18A.

[0115] Therefore, for example, by checking the determination result information output to the output unit 18A, the user can easily adjust the installation environment of the imaging unit 12, etc., so that the imaging environment, such as the imaging angle of view and imaging direction, becomes a desired imaging environment. In other words, by checking the determination result information, the user can easily adjust the imaging environment, such as the installation environment of the imaging unit 12, so that the imaging direction and imaging angle of view become such that the entire specific target P is captured.

[0116] Therefore, in addition to the above-mentioned effects, the information processing device 10 of this embodiment can reduce the workload of adjusting the imaging unit 12.

[0117] In the above embodiment, the photographing unit 12 is provided in the information processing device 10. However, as described above, the photographing unit 12 may be mounted on an external information processing device that is communicatively connected to the information processing device 10 via the communication unit 16.

[0118] Examples of external information processing devices equipped with the imaging unit 12 include, but are not limited to, surveillance cameras installed on buildings, flying objects such as drones, unmanned guided vehicles equipped with the imaging unit 12, and moving objects equipped with the imaging unit 12.

[0119] In the above embodiment, the output control unit 20F outputs the determination result information to the UI unit 18. However, the output control unit 20F may output the determination result information to an external information processing device via the communication unit 16.

[0120] For example, the output control unit 20F may output the determination result information to an information processing device equipped with the photographing unit 12 via the communication unit 16. In this case, the control unit of the information processing device may use the determination result information received from the information processing device 10 to control a driving unit such as a motor for adjusting the photographing direction and photographing angle of view of the photographing unit 12 so that the photographing direction and photographing angle of view are such that the entire specific target P is captured.

[0121] Next, an example of the hardware configuration of the information processing device 10 of the above embodiment will be described.

[0122] FIG. 7 is a diagram showing an example of the hardware configuration of the information processing device 10 according to the embodiment.

[0123] The information processing device 10 of the above embodiment has a hardware configuration that utilizes a conventional computer, in which a CPU (Central Processing Unit) 81, a ROM (Read Only Memory) 82, a RAM (Random Access Memory) 83, a communication I / F 84, etc. are interconnected by a bus 85.

[0124] The CPU 81 is a computing device that controls the information processing device 10 of the above embodiment. The ROM 82 stores programs and the like that realize various processes by the CPU 81. Although a CPU is used in the description here, a GPU (Graphics Processing Unit) may also be used as the computing device that controls the information processing device 10. The RAM 83 stores data necessary for various processes by the CPU 81. The communication I / F 84 is an interface that is connected to the UI unit 18, etc., and is used to send and receive data.

[0125] In the information processing device 10 of the above embodiment, the CPU 81 reads out a program from the ROM 82 onto the RAM 83 and executes it, thereby realizing each of the above functions on the computer.

[0126] The programs for executing the above processes executed by the information processing device 10 of the embodiment may be stored in an HDD (hard disk drive). Also, the programs for executing the above processes executed by the information processing device 10 of the embodiment may be provided by being pre-installed in the ROM 82.

[0127] Furthermore, the program for executing the above-described processes executed by the information processing device 10 of the above-described embodiment may be stored in an installable or executable file format on a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD (Digital Versatile Disk), or flexible disk (FD) and provided as a computer program product. Furthermore, the program for executing the above-described processes executed by the information processing device 10 of the above-described embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Furthermore, the program for executing the above-described processes executed by the information processing device 10 of the above-described embodiment may be provided or distributed via a network such as the Internet.

[0128] Although the embodiment of the present invention has been described above, this embodiment is presented as an example and is not intended to limit the scope of the invention. This novel embodiment can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. This embodiment and its modifications are included within the scope and spirit of the invention, and are also included in the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0129] 10. Information processing equipment 20B Specific target area detection unit 20C Image area detection section 20D Judgment section 20E Specific part 20F Output control section

Claims

1. a specific target region detection unit that detects a specific target region of a specific target from a captured image; a reflection area detection unit that detects a reflection area of ​​the specified object from the captured image; a determination unit that determines a non-image area of ​​the identified target based on the identified target area and the image area; Equipped with The specific target area is an area including a hidden area of ​​the specific target due to an obstruction that appears in the captured image and including the entire specific target. Information processing device.

2. The specific target area is a region including the entire specific target, the region being included in a virtual enlarged region obtained by enlarging the shooting range of the captured image; The information processing device according to claim 1 .

3. The determination unit determining the non-imaged area, which includes at least one of a hidden area of ​​the specific target included in the photographed image and a cut-out area of ​​the specific target that is cut off from the photographed image, based on the specific target area and the captured area; The information processing device according to claim 1 .

4. an identification unit that identifies a recommended photographing environment in which the non-photographic area is photographed in the photographed image; The information processing device according to claim 3 , comprising:

5. The identification unit specifying, as the recommended shooting environment, at least one of a recommended shooting angle of view in which the edge area is captured and a recommended shooting direction in which the hidden area is captured; The information processing device according to claim 4 .

6. an output control unit that outputs, to an output unit, determination result information corresponding to the determination result of the determination unit; The information processing device according to claim 1 , comprising:

7. The output control unit outputting at least one of a sound and an image representing the determination result information to the output unit; The information processing device according to claim 6 .

8. The output control unit outputting, to the output unit, the determination result information indicating at least one of the non-imaged area, the specific target area, the imaged area, the recommended photographing environment, and a warning indicating that the non-imaged area is included in the photographed image; The information processing device according to claim 6 .

9. The output control unit The information processing apparatus according to claim 6 , wherein a superimposed image obtained by superimposing an image representing the determination result information on the photographed image is output to the output unit.

10. detecting a specific target region of a specific target from the captured image; detecting a region of the specified object from the captured image; determining a non-image area of ​​the identified target based on the identified target area and the image area; Including, The specific target area is an area including a hidden area of ​​the specific target due to an obstruction that appears in the captured image and including the entire specific target. Information processing methods.

11. An information processing program to be executed by a computer, detecting a specific target region of a specific target from the captured image; detecting a region of the specified object from the captured image; determining a non-image area of ​​the identified target based on the identified target area and the image area; Including, The specific target area is an area including a hidden area of ​​the specific target due to an obstruction that appears in the captured image and including the entire specific target. Information processing program.

Images