Information processing device, information processing method, and computer program

Abstract

To provide an information processing device capable of controlling a view angle without missing a person to be tracked.SOLUTION: An information processing device comprises: detection means for detecting a prescribed object from an image captured by capturing means; setting means for setting a prescribed region and an exit determination region smaller than the prescribed region for the image; determination means for determining exit on the basis of whether the object has exited from the exit determination region; and control means for controlling a photographing range of the capturing means according to the results determined by the determination means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a computer program for tracking. [Background technology]

[0002] In recent years, there has been a growing need for remote cameras to automatically capture moving scenes such as lectures and sports scenes. One technology that enables this is to adjust the angle of view by pan-tilt-zoom (PTZ) operations to match the movement of the person being tracked, thereby keeping the person within the angle of view.

[0003] For example, in a lecture scenario, when a lecturer is explaining something on a board or using slides, you may want to fit both the lecturer and the subject of the explanation on the screen. In this case, if the angle of view is adjusted to match the lecturer's movements or changes in position, it can become difficult to see the notes on the board or the text on the slides in the background.

[0004] To prevent this, it is possible to fix the angle of view when the person being tracked enters a specific area, and when the person leaves the area, control the angle of view according to the composition settings (settings for where on the screen the target will be displayed and how large on the screen the target will be displayed). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-148646 Summary of the Invention [Problem to be solved by the invention]

[0006] On the other hand, Patent Document 1 describes a configuration in which an exit determination area of ​​a size different from the error range is set according to the error range of the position of the target's terminal device, and exit from the monitoring area is determined based on the overlapping relationship between the monitoring area and the exit determination area. However, if the exit determination of the target is performed over the entire shooting angle of view, the exit determination cannot be made unless the target completely leaves the angle of view. Therefore, if the angle of view is fixed when the tracking target enters a specific area and controlled according to the composition setting when the tracking target leaves the area, there is a problem in that the tracking target will disappear from the screen before the determination is completed, making it impossible to track.

[0007] An object of the present invention is to provide an information processing device that can control the angle of view without losing sight of the person to be tracked. [Means for solving the problem]

[0008] An information processing device according to one aspect of the present invention includes: a detection means for detecting an object from an image captured by the imaging means; specified a setting means for setting an exit determination area smaller than the predetermined area; and, The aforementioned Imaging means image A control means for controlling the range. death, the setting means is capable of setting a display position and a display size of the object in the image; When the object is present outside the exit determination area, the control means controls the imaging range so that the object is displayed at the display position and the display size set by the setting means. It is characterized by: [Effects of the Invention]

[0009] According to the present invention, it is possible to realize an information processing device that can control the angle of view without losing sight of the tracking target. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a functional block diagram of an automatic photography system according to a first embodiment of the present invention. [Figure 2] 1A and 1B are diagrams illustrating composition setting according to the first embodiment of the present invention. [Figure 3] 10A to 10D are diagrams illustrating an example in which the angle of view is controlled in accordance with the composition setting by moving and fixing the angle of view to a predetermined area when the tracking target person enters the predetermined area, and releasing the fixed angle of view when the tracking target person leaves the predetermined area. [Figure 4] 10A and 10B are diagrams illustrating an intrusion determination area and an exit determination area within an area setting, and composition setting according to the first embodiment of the present invention. [Figure 5] FIG. 2 is a diagram for explaining an example in which a tracking target person enters inside an intrusion determination area within an area setting in the first embodiment of the present invention. [Figure 6] FIG. 2 is a diagram showing that a tracking target person remains within a set area in the first embodiment of the present invention. [Figure 7] 10 is a diagram illustrating a state in which a tracking target person exits outside an exit determination area within an area setting in accordance with the first embodiment of the present invention. FIG. [Figure 8] 4 is a flowchart showing the processing procedure of the automatic photography system according to the first embodiment of the present invention. [Figure 9] 9 is a flowchart showing the processing procedure continued from FIG. 8. [Figure 10] FIG. 10 is a functional block diagram of an automatic photography system according to a second embodiment of the present invention. [Figure 11] 10A and 10B are diagrams illustrating how the sizes of an exit determination region and an intrusion determination region are determined when the face of a tracking target person is small in the second embodiment of the present invention. [Figure 12] 10A and 10B are diagrams illustrating how the sizes of an exit determination region and an intrusion determination region are determined when the face of the tracking target is large in the second embodiment of the present invention. [Figure 13] 10 is a flowchart showing the processing procedure of an automatic photography system according to a third embodiment of the present invention. [Figure 14] 14 is a flowchart showing the processing procedure continued from FIG. 13. [Figure 15] FIG. 10 is a functional block diagram of an automatic photography system according to a third embodiment of the present invention. [Figure 16]10A and 10B are diagrams illustrating a state of area inside / outside determination when a tracking target person moves through an area where area settings overlap according to a third embodiment of the present invention. [Figure 17] 10 is a flowchart showing the processing procedure of an automatic photography system according to a third embodiment of the present invention. [Figure 18] 18 is a flowchart showing the processing procedure continued from FIG. 17. [Figure 19] FIG. 10 is a functional block diagram of an automatic photography system according to a fourth embodiment of the present invention. [Figure 20] 10 is a flowchart showing the processing procedure of an automatic photography system according to a fourth embodiment of the present invention. [Figure 21] 21 is a flowchart showing the processing procedure continued from FIG. 20. [Figure 22] FIG. 10 is a diagram illustrating how, when it is determined that a tracking target person has left an area, the angle of view is controlled in accordance with the zoom magnification and composition settings that were recorded at the time of entry into the area, in accordance with the fourth embodiment of the present invention. [Figure 23] FIG. 2 is a diagram illustrating a hardware configuration of the imaging apparatus. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiments. In each drawing, the same members or elements are designated by the same reference numerals, and duplicate descriptions will be omitted or simplified.

[0012] (Embodiment 1) An example of the configuration of an automatic photography system A1000 according to the first embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a functional block diagram of the automatic photography system according to the first embodiment of the present invention.

[0013] Note that some of the functional blocks shown in FIG. 1 are realized by causing a CPU or the like serving as a computer (not shown) included in the automatic photography system to execute a computer program stored in a memory serving as a storage medium (not shown).

[0014] However, some or all of these functions may be implemented by hardware, which may be a dedicated circuit (ASIC) or a processor (reconfigurable processor, DSP).

[0015] Furthermore, the functional blocks shown in Fig. 1 do not have to be housed in the same housing, but may be configured as separate devices connected to each other via signal paths. The above explanation regarding Fig. 1 also applies to Figs. 10, 15, and 19.

[0016] The automatic photography system A1000 of this embodiment includes an image capturing device A1001, a PTZ driving device A1003, an imaging device A1020 including an information processing device A1004, and an external device A1014.

[0017] The imaging device A1020 and the external device A1014 are connected via a network. The image capture device A1020 and the external device A1014 may be connected via a video interface.

[0018] The imaging device A1020 of this embodiment performs tracking processing of a predetermined target based on the image acquired from the image acquisition device A1001 and the tracking settings acquired from the user input acquisition unit A1002. Then, the angle of view is controlled using the PTZ driving device A1003 so that the tracking target does not move out of the angle of view.

[0019] In this embodiment, the predetermined object to be tracked is a human body, and in particular a human face. Therefore, a human body or a human face is automatically tracked. However, the predetermined object to be tracked is not limited to a human body and may be an animal. Alternatively, the object may be a moving object such as an automobile, airplane, train, or ship, and the object in this embodiment includes these animals and other moving objects.

[0020] When controlling the angle of view, it is determined whether the tracking target is within a predetermined area (a set area) and the angle of view control is switched depending on whether the target is within the predetermined area or outside the predetermined area. The photographing results are displayed on the external device A1014. The external device A1014 is, for example, a PC terminal or the like including a display element such as a liquid crystal display.

[0021] The image capturing device A1001 is a device that captures images of the surroundings and generates captured images, and is configured with a camera or the like as an imaging means. The image capturing device A1001 outputs captured image information to the face detection unit A1006 and image output unit A1013 of the information processing device A1004.

[0022] The user input acquisition unit A1002 is a part that acquires tracking settings made by a user. The user input acquisition unit A1002 acquires settings from an external device A1014 that is configured by a GUI (Graphical User Interface) that runs on a web browser (not shown) or the like. The user input acquisition unit A1002 outputs the tracking settings input by the user to the tracking setting unit A1007.

[0023] The user input acquisition unit A1002 functions as a setting unit for setting a predetermined area and composition for the image. Here, the composition refers to the position and size at which the tracking target is displayed on the screen. That is, the user input acquisition unit A1002 can set the composition, such as the display position and display size of the target in the image.

[0024] The setting means is not limited to an example in which a user sets a predetermined area or composition for an image, but may set the predetermined area or composition according to the characteristics (shape, size, speed, acceleration) of a subject detected by image recognition, etc., or by initial setting. Furthermore, the setting means can set a predetermined area for an image, an exit determination area smaller than the predetermined area, or an intrusion determination area smaller than the exit determination area.

[0025] The PTZ driving device A1003 is a device that changes the angle of view, and is configured with driving devices such as motors for performing pan, tilt, and zoom control, and performs PTZ driving based on PTZ control values ​​input from the angle of view operation unit A1012. Note that the PTZ driving device A1003 in this embodiment only needs to be capable of at least one of the three driving operations of pan, tilt, and zoom.

[0026] The information processing device A1004 performs image analysis processing, determination processing, etc. The information processing device A1004 performs tracking processing based on the coordinates of a face detected using the input video and the composition settings input by the user (settings related to the position where the tracking target person is displayed on the screen, the display size, etc.).

[0027] Furthermore, if the person to be tracked is within the area (predetermined area) input by the user, the angle of view is moved to that area and then fixed, and when the person to be tracked leaves the predetermined area, the angle of view control is switched according to the position of the person to be tracked so that the angle of view is controlled based on the composition setting.

[0028] In other words, if it is determined that the object is outside the set specified area (not inside the specified area), the shooting range (angle of view) is controlled using the PTZ drive unit so that the object is in the composition set by the setting means (display position and display size on the screen, etc.).

[0029] The information processing device A1004 has an image acquisition unit A1005, a face detection unit A1006, a tracking setting unit A1007, a tracking setting recording unit A1008, a tracking processing unit A1009, an area inside / outside determination unit A1010, and a field of view operation calculation unit A1011. The information processing device A1004 also has a field of view operation unit A1012 and a video output unit A1013. The video acquisition unit A1005 outputs the acquired image to the face detection unit A1006 and the video output unit A1013.

[0030] The face detection unit A1006 performs face detection processing of human bodies in the video by performing image recognition based on the video information input via the video acquisition unit A1005. Note that the face detection processing may use any method that can detect faces, such as template matching or semantic region segmentation.

[0031] The template matching method and the semantic region segmentation method are well-known techniques, and therefore detailed description thereof will be omitted. The face detection unit A1006 functions as a detection unit that detects a predetermined object from an image captured by an imaging unit. As mentioned above, the detected object is not limited to a human body and its face, but may also be an animal, its face, or a moving object. The face detection unit A1006 outputs the coordinates of the face detected from each piece of video information to the tracking processing unit A1009.

[0032] When a tracking setting is input from the user input acquisition unit A1002, the tracking setting unit A1007 reflects the tracking setting and outputs it to the tracking setting recording unit A1008. In this embodiment, there are two types of tracking setting: area setting and composition setting, but other settings such as tracking sensitivity setting may also be set. Composition setting will be described with reference to FIGS. 2(A) and 2(B).

[0033] 2A and 2B are diagrams for explaining composition setting according to the first embodiment of the present invention. Composition setting refers to setting the position, size, etc. of the tracking target on the screen so that the tracking target continues to be captured at a specific position even if the tracking target moves within the screen, and so that the tracking target continues to be maintained at a specific size even if the size of the tracking target on the screen changes.

[0034] Figures 2(A) and (B) show examples of different composition settings, with D001 and D002 being image diagrams of the composition settings. P001 and P003 represent the position at which the tracking target is displayed, while P002 and P004 represent the size at which the tracking target is displayed. D001 represents a composition setting where the tracking target is at the center of the screen, and the size represents a composition setting where the upper half of the tracking target fits within the screen.

[0035] In D002, the composition is set so that the tracking target is in the upper right corner of the screen, and the size is set so that the entire body of the tracking target fits within the screen. Switching between area setting and angle of view control will be explained using Figure 3.

[0036] 3A to 3D are diagrams illustrating an example in which the angle of view is moved to a predetermined area and fixed there when the tracking target person enters the predetermined area, and the fixed angle of view is released when the tracking target person leaves the predetermined area, thereby controlling the angle of view in accordance with the composition setting.

[0037] Screens D101, D102, and D103 show examples of the angle of view when the tracking target person enters a predetermined area (an area previously set by the user) P103. Screen D101 shows a state in which the tracking target person has moved from position P101 to position P102, and position P102 is within the predetermined area (an area previously set by the user) P103.

[0038] At this time, the angle of view of screen D101 is changed to match a predetermined area P103, resulting in screen D102. While the tracking target person is inside area P103, the angle of view is fixed even if the tracking target person moves inside area P103. Screen D103 shows this situation. Even if the tracking target person moves from position P102 to position P104 inside this area P103, the shooting direction does not move and the shooting size does not change. In other words, the angle of view does not change.

[0039] In this way, when it is determined that the tracking target as an object is present within the set predetermined area (has entered the intrusion determination area), the photographing range (photographing angle of view) is controlled by the PTZ driving device so that the photographing range includes the above-mentioned predetermined area. Furthermore, in this case, the photographing range (photographing angle of view) is controlled by the PTZ driving device so that the photographing range is in a position and size corresponding to the above-mentioned predetermined area.

[0040] When the tracking target person moves out of area P103, the fixed angle of view is released and the angle of view is widened to capture the tracking target person. Screen D104 shows this situation. As the tracking target person moves from position P104 to position P105 outside area P103, the shooting angle of view is widened from the angle of view corresponding to area P103 and moves to match position P105, changing to the state shown in screen D104.

[0041] The tracking setting unit A1007 outputs the above-described area setting and composition setting as tracking settings to the area inside / outside determination unit A1010 and the angle of view operation calculation unit A1011. The tracking setting recording unit A1008 records the tracking setting input from the tracking setting unit A1007. The tracking setting recording unit A1008 also outputs the recorded tracking setting to the tracking setting unit A1007.

[0042] The tracking processing unit A1009 performs tracking processing based on the face information input from the face detection unit A1006. That is, if tracking processing has not been performed, it selects a tracking target and starts tracking processing, and if tracking processing has already been performed, it inputs the detected face information and performs tracking processing. Any selection method for selecting a tracking target may be used as long as it is a method that allows selection of one target from multiple pieces of face information.

[0043] For example, the face closest to the center of the screen may be set as the tracking target. Any tracking process may be used as long as it is possible to determine the position information of the selected tracking target in the previous frame and the position information of the tracking target in the current frame from the input face information. For example, tracking may be performed based on the position predicted from the movement history of the tracking target and the position of the detected face. The tracking process result is output to the area inside / outside determination unit A1010.

[0044] The area inside / outside determination unit A1010 determines whether the tracking target is inside or outside a specified area (set area) based on the tracking processing result input from the tracking processing unit A1009 and the area setting input from the tracking setting unit A1007.

[0045] The area inside / outside determination unit A1010 functions as a determination means for determining whether or not an object is present inside a predetermined area. The determination means also performs an exit determination based on whether or not the object has exited the exit determination area, and performs an intrusion determination based on whether or not the object has entered the intrusion determination area.

[0046] An example of input tracking settings will be described with reference to Fig. 4. Figs. 4(A) and (B) are diagrams illustrating an intrusion determination area and an exit determination area within the area setting according to the first embodiment of the present invention, and composition settings. P201 represents a predetermined area (a set area), P202 represents an intrusion determination area for determining an intrusion into the area of ​​P201, and P203 represents an exit determination area for determining an exit from the area of ​​P201.

[0047] If P202 and P203 are the same size, the tracking target will repeatedly be judged as entering and leaving if it remains at the boundary of the area. Therefore, the size of each of these areas must have the following relationship: P201>P203>P202. As long as this relationship is maintained, the size of each area can be any size.

[0048] D201 is a screen showing an example of composition settings. P204 shows the position where the tracking target is displayed, and P205 shows the size of the tracking target. Next, the determination process in the area inside / outside determination unit A1010 will be explained. First, it is confirmed whether the tracking target is already inside a predetermined area. If multiple predetermined areas have been set, it is confirmed whether the tracking target is inside any of the predetermined areas.

[0049] If the tracking target person is not yet inside the predetermined area, it is determined whether the person has entered the predetermined area from outside, and if not, it is determined whether the person has left the predetermined area. An example of determining whether the person has entered the predetermined area from outside will be described using Figure 5.

[0050] 5 is a diagram illustrating an example in which a tracking target person has entered an intrusion determination area within a predetermined area (area setting) in embodiment 1 of the present invention. D301 is a screen showing the shooting angle of view. P301 represents the predetermined area, and P302 represents the intrusion determination area.

[0051] Assume that when the tracking target person moves from position P303 to position P304, it is detected that the center position P305 of the face frame P306 has moved into the intrusion determination area P302 based on the face detection result of the tracking target person. In this case, the area inside / outside determination unit A1010 determines that the tracking target person has intruded into a predetermined area (set area) P301.

[0052] The determination result and area information are then output to the angle of view operation calculation unit A1011. That is, if the center of the face frame of the target object is inside the intrusion determination area, it is determined that an intrusion has occurred. Note that it may also be determined that an intrusion has occurred if the center of the face is inside the intrusion determination area, and in this embodiment, the center of the face includes the center of the face frame.

[0053] Next, the state of determination when the tracking target is present within a predetermined area will be described with reference to Fig. 6 and Fig. 7. Fig. 6 is a diagram showing that the tracking target remains within a predetermined area (set area) in embodiment 1 of the present invention, and Fig. 7 is a diagram showing that the tracking target has exited to the outside of the exit determination area within the predetermined area (set area) in embodiment 1 of the present invention.

[0054] D401 in Fig. 6 and D501 in Fig. 7 represent the angle of view set in accordance with the predetermined area P201 in Fig. 4. In Fig. 6, the position P401 of the tracking target person is inside the exit determination area P402, and therefore it is determined that the tracking target person is inside the predetermined area.

[0055] In FIG. 7, the tracking target person moves from position P501 to position P502, and as a result, the center position P503 of the face frame P504, which is the face detection result of the tracking target person P502, is detected to be outside the exit determination area P505.

[0056] In this case, it is determined that the tracking target person has exited the predetermined area. The determination result is then output to the angle of view operation calculation unit A1011. Note that it may also be determined that the person has exited if the center of the face is outside the exit determination area, and as described above, the center of the face includes the center of the face frame.

[0057] In this way, the area inside / outside determination unit A1010 makes an exit determination based on whether or not the target (tracking target) has exited an exit determination area that is smaller than the set predetermined area, and makes an intrusion determination based on whether or not the target has entered an intrusion determination area that is smaller than the exit determination area.

[0058] The angle of view operation calculation unit A1011 calculates the angle of view value to be controlled based on the determination result and area information input from the area inside / outside determination unit A1010 and the composition setting input from the tracking setting unit A1007.

[0059] If the input determination result indicates that the object continues to remain within the predetermined area, the same angle of view as the current one is output to the angle of view operation unit A1012 in order to fix (maintain) the current angle of view. If the input determination result indicates that the object has entered the predetermined area from outside the predetermined area, the angle of view value at which the invaded predetermined area becomes the angle of view is output to the angle of view operation unit A1012.

[0060] If it is determined that the person has left the specified area (left the exit determination area) or is outside the specified area, the angle of view value is calculated so that the current position and size of the person to be tracked within the screen matches the position and size specified in the composition settings, and this angle of view value is output to the angle of view operation unit A1012.

[0061] The angle of view operation unit A1012 calculates a PTZ control value from the angle of view value input from the angle of view operation calculation unit A1011. Any method for calculating the PTZ control value may be used as long as it results in the input angle of view value, and may include a method of directly specifying PTZ coordinate values ​​or a method of specifying the direction and speed of panning or tilting.

[0062] The calculated PTZ control value is output to the PTZ driving device A1003 to control the angle of view of the imaging device. Here, the angle of view operation calculation unit A1011 and angle of view operation unit A1012 function as control means for controlling the shooting range of the imaging means in accordance with the result determined by the area inside / outside determination unit A1010.

[0063] The video output unit A1013 outputs the video input via the video acquisition unit A1005 to an external device A1014 such as a PC terminal. The external device A1014 displays the video input from the video output unit A1013 on a monitor.

[0064] Next, a procedure for performing processing by the imaging device will be described with reference to the flowcharts of Fig. 8 and Fig. 9. Fig. 8 is a flowchart showing the processing procedure by the imaging device according to the first embodiment of the present invention, and Fig. 9 is a flowchart showing the processing procedure continued from Fig. 8.

[0065] The operations of the steps in the flowcharts of FIGS. 8 and 9 are performed by a CPU or the like serving as a computer within the imaging device executing a computer program stored in a memory.

[0066] 8 and 9 starts when the image capture device A1020 of the automatic photography system A1000 is activated by a user operation. First, in step S001, the video acquisition unit A1005 acquires video information from the video capture device A1001 and outputs it to the face detection unit A1006 and the video output unit A1013. Then, the process proceeds to step S002.

[0067] In step S002, the user input acquisition unit A1002 acquires area information and composition settings (settings indicating the position where the tracking target is to be displayed on the screen, the size at which it is displayed, etc.) set by the user, and automatic selection settings for the tracking target, and outputs these to the tracking setting unit A1007. The tracking setting unit A1007 outputs the input area information, composition settings, and automatic selection settings for the tracking target to the tracking setting recording unit A1008 and records them.

[0068] Then, the process proceeds to step S003. Here, step S002 functions as a setting step for setting a predetermined area in the image. Note that the setting step can set a predetermined area in the image, an exit determination area smaller than the predetermined area, or an intrusion determination area smaller than the exit determination area.

[0069] In step S003, the face detection unit A1006 performs image recognition based on the input video information to detect a face as a human body. The detected face information and video information are output to the tracking processing unit A1009. Then, the process proceeds to step S004. Here, step S003 functions as a detection step for detecting a predetermined object from an image captured by the imaging means.

[0070] In step S004, the tracking setting unit A1007 selects a tracking target from the face information as the input human body detection result and performs tracking processing. Also, it outputs the coordinate information of the tracking target person as the tracking processing result to the area inside / outside determination unit A1010. Then, the process proceeds to step S005.

[0071] In step S005, the tracking setting unit A1007 determines whether the tracking process of the human body (face) has been successful. If the answer is Yes in step S005, the process proceeds to step S006. If the answer is No in step S005, the process proceeds to step S012 in FIG. 9 via C in FIG. 8.

[0072] In step S006, the area inside / outside determination unit A1010 determines whether the tracking target has entered the inside of a predetermined area. This determination process is a process of first confirming whether the tracking target is already inside the predetermined area. Note that if multiple predetermined areas are set, it is confirmed whether the tracking target is inside any of the predetermined areas.

[0073] If the answer is Yes in step S006, that is, if the tracking target is present inside the predetermined area, proceed to step S007. If the answer is No in step S006, that is, if the tracking target is not present inside the predetermined area, proceed to step S008 in Fig. 9 via A in Fig. 8.

[0074] In step S007, the area inside / outside determination unit A1010 determines whether the center of the face frame (or face) of the tracking target person is outside the exit determination area. That is, in step S007, it is determined whether the tracking target person who is present inside the predetermined area in step S006 has exited the exit determination area within the predetermined area.

[0075] If the answer is Yes in step S007, that is, if the tracking target person has exited the exit determination area within the predetermined area, the process proceeds to step S010 in Fig. 9 via B in Fig. 8. If the answer is No in step S007, that is, if the tracking target person has not exited the exit determination area within the predetermined area but continues to remain within the predetermined area, the information is output to the angle of view operation calculation unit A1011, and the process proceeds to step S012 in Fig. 9 via C in Fig. 8.

[0076] 9, the area inside / outside determination unit A1010 determines whether the center of the face frame of the tracking target person is inside any of the intrusion determination areas, and outputs the determination result to the angle of view operation calculation unit A1011. That is, in step S008, it is determined whether the tracking target person, who was not inside the predetermined area in step S006, has intruded into an intrusion determination area within the predetermined area.

[0077] If the answer to step S008 is Yes, that is, if the tracking target has entered the intrusion determination area within the predetermined area, proceed to step S009. If the answer to step S008 is No, that is, if the tracking target has not entered the intrusion determination area within the predetermined area, proceed to step S010.

[0078] Here, steps S006 to S008 function as a determination step for determining whether or not an object exists within a predetermined area. In this determination step, an exit determination is made based on whether or not the object has exited an exit determination area that is smaller than the predetermined area, and an intrusion determination is made based on whether or not the object has intruded into an intrusion determination area that is smaller than the exit determination area.

[0079] In step S009, the view angle operation calculation unit A1011 calculates a view angle control value for changing the view angle to a predetermined area where the tracking target has entered, outputs the calculated value to the view angle operation unit A1012, and generates a PTZ control command. Then, the process proceeds to step S011.

[0080] Meanwhile, in step S010, the angle of view operation calculation unit A1011 calculates an angle of view control value for displaying the tracking target within the screen in accordance with the predicted position of the tracking target and the composition setting acquired from the tracking setting unit A1007, outputs the calculated value to the angle of view operation unit A1012, and generates a PTZ control command. Then, the process proceeds to step S011.

[0081] In step S011, the angle-of-view operation unit A1012 outputs a PTZ control command to the PTZ drive device A1003 based on the angle-of-view control value input from the angle-of-view operation calculation unit A1011. The PTZ drive device A1003 then drives the PTZ to change the shooting angle of view. Then, the process proceeds to step S012. Here, step S011 functions as a control step that controls the shooting range of the imaging means in accordance with the result of the determination made in the determination step.

[0082] In step S012, the video output unit A1013 outputs the input video information to the external device A1014. Then, the process proceeds to step S013.

[0083] In step S013, it is determined whether a user has performed a stop operation to turn off the image capture device A 1020. If No in step S013, the process returns to step S001 in Fig. 8 via D in Fig. 9, and if Yes in step S013, the automatic shooting process ends and the flow in Fig. 8 and Fig. 9 ends.

[0084] As described above, in the first embodiment, when tracking processing is performed based on face detection, an intrusion determination area and an exit determination area of ​​different sizes are provided inside a predetermined area, and it is determined that an intrusion has occurred when the center of the face frame (or face) of the tracking target person enters inside the intrusion determination area. Also, it is determined that an exit has occurred when the center of the face frame (or face) of the tracking target person leaves the exit determination area. This makes it possible to control the angle of view while continuing tracking without losing sight of the face when an exit determination is made.

[0085] (Embodiment 2) An example of the configuration of an automatic photography system B1000 according to a second embodiment of the present invention will be described with reference to Fig. 10. Fig. 10 is a functional block diagram of the automatic photography system according to the second embodiment of the present invention. In the second embodiment, the sizes of the exit determination area and the intrusion determination area are dynamically changed according to the size of the face of the tracking target person to prevent the face of the tracking target person from being lost from the screen when an exit determination is made depending on the size of the face of the tracking target person within the screen.

[0086] The automatic photography system B1000 performs tracking processing based on the image acquired via the image acquisition unit A1005 and the tracking settings acquired from the user input acquisition unit A1002, and controls the angle of view using the PTZ drive device A1003 so that the tracking target does not move outside the angle of view.

[0087] At this time, it is determined whether the tracking target is within a predetermined area (a set area), and the angle of view control is switched depending on whether the target is within the predetermined area or outside the predetermined area. The shooting results are then displayed on the external device A1014. The automatic photography system B1000 has an image capturing device A1001, a PTZ driving device A1003, an imaging device B1020 including an information processing device B1004, and the external device A1014.

[0088] The imaging device and the external device A 1014 are connected via a network, or may be connected via a video interface.

[0089] The information processing device B1004 performs image analysis processing, determination processing, etc. The information processing device B1004 performs tracking processing based on the coordinates of the face detected using the input video and the set composition setting. Furthermore, when the tracking target is inside a predetermined area (a set area), the angle of view is changed to the predetermined area and then fixed, and when the tracking target leaves the predetermined area, the angle of view control is switched according to the position of the tracking target.

[0090] The information processing device B1004 has an image acquisition unit A1005, a face detection unit A1006, a tracking setting unit A1007, a tracking setting recording unit A1008, a tracking processing unit A1009, an area inside / outside determination unit B1010, and a field of view operation calculation unit A1011. The information processing device B1004 also has a field of view operation unit A1012 and a video output unit A1013.

[0091] The area inside / outside determination unit B1010 determines whether the tracking target is inside or outside a predetermined area based on the tracking processing result input from the tracking processing unit A1009 and the area setting input from the tracking setting unit.

[0092] In order to prevent the face of the tracking target from being lost from the screen when an exit determination is made due to the size of the tracking target on the screen, in this embodiment, the sizes of the exit determination area and the intrusion determination area are dynamically changed according to the size of the face of the tracking target. This will be explained with reference to Figs. 11 and 12.

[0093] Fig. 11 is a diagram showing how the sizes of the exit determination area and the intrusion determination area are determined when the face of the tracking target person is small in embodiment 2 of the present invention. Also, Fig. 12 is a diagram showing how the sizes of the exit determination area and the intrusion determination area are determined when the face of the tracking target person is large in embodiment 2 of the present invention.

[0094] P601 represents a predetermined area (a set area), P604 represents the tracking target, P605 represents a face frame as a result of face detection of the tracking target, and P606 represents the center position of the face frame of the tracking target. Also, P602 represents an intrusion determination area, and P603 represents an exit determination area. Here, to make it easier to understand how to determine the size of the intrusion determination area and exit determination area, an example is shown in which the tracking target is at the edge of the screen, but the size of the area can be determined even if the tracking target is not in this position.

[0095] In this embodiment, as shown in FIG. 11, the exit determination area P603 is set so that the difference in size between it and a predetermined area (set area) P601 is greater than, for example, half the length of a side of the face frame P605 of the tracking target person P604.

[0096] The intrusion determination area P602 is set to be smaller than the exit determination area P603. On the other hand, even when the size of the tracking target person is large as shown in Fig. 12, the size of the exit determination area is dynamically set in a similar manner. In Fig. 12, P701 represents a predetermined area (a set area), P704 represents the tracking target person, P705 represents the face frame of the tracking target person, and P706 represents the center position of the face frame of the tracking target person.

[0097] As shown in FIG. 11, the exit determination area P703 is set so that the difference in size between it and a predetermined area (set area) P701 is greater than, for example, half the length of a side of the face frame P705 of the tracking target person P704.

[0098] The intrusion determination area P702 is set smaller than the exit determination area P703. Other determination processes in the area inside / outside determination unit B1010 are the same as those in the area inside / outside determination unit A1010. Moreover, since the blocks other than the area inside / outside determination unit B1010 are the same as those in the first embodiment, a description thereof will be omitted.

[0099] Next, the procedure for processing the imaging device will be described with reference to the flowcharts of Fig. 13 and Fig. 14. Fig. 13 is a flowchart showing the processing procedure of the imaging device according to the second embodiment of the present invention, and Fig. 14 is a flowchart showing the processing procedure continued from Fig. 13. Note that the operation of each step in the flowcharts of Fig. 13 and Fig. 14 is performed by a CPU or the like serving as a computer in the imaging device executing a computer program stored in memory.

[0100] 13 and 14 starts when the image capture device B1020 of the automatic photography system B1000 is activated by a user operation. First, in step S001, the video capture unit A1005 acquires video information from the video capture device A1001 and outputs it to the face detection unit A1006 and the video output unit A1013. Then, the process proceeds to step S002.

[0101] In step S002, the user input acquisition unit A1002 acquires area information and composition settings (settings indicating the position where the tracking target is displayed on the screen, the display size, etc.) set by the user, and automatic tracking target selection settings, and outputs these to the tracking setting unit A1007. The tracking setting unit A1007 outputs the input area information, composition settings, and automatic tracking target selection settings to the tracking setting recording unit A1008 and records them. Then, the process proceeds to step S003.

[0102] In step S003, the face detection unit A1006 performs image recognition based on the input video information to detect faces as human body detection. The detected face information and video information are output to the tracking processing unit A1009. Then, the process proceeds to step S004.

[0103] In step S004, the tracking setting unit A1007 selects a tracking target from the face information as the input human body detection result and performs tracking processing. Also, it outputs the coordinate information of the tracking target person as the tracking processing result to the area inside / outside determination unit B1010. Then, the process proceeds to step S005.

[0104] In step S005, the tracking setting unit A1007 determines whether the tracking process of the human body (face) has been successful. If the answer is Yes in step S005, the process proceeds to step S114. If the answer is No in step S005, the process proceeds to step S012 in FIG. 14 via C in FIG. 13.

[0105] In step S114, the area inside / outside determination unit B1010 dynamically determines the sizes of the intrusion determination area and exit determination area of ​​the set area according to the size of the face of the tracking target person. Then, the process proceeds to step S006. That is, in this embodiment, the sizes of the intrusion determination area and exit determination area are changed according to the size of the target within the shooting range. Note that the size of at least one of the intrusion determination area and exit determination area may be changed according to the size of the target within the shooting range.

[0106] In step S006, the area inside / outside determination unit B1010 determines whether the tracking target has entered the predetermined area. If the answer is Yes in step S006, that is, if the tracking target is present inside the predetermined area, the process proceeds to step S007. If the answer is No in step S006, that is, if the tracking target is not present inside the predetermined area, the process proceeds to step S007 in FIG. 14 via A in FIG. 13.

[0107] In step S007, the area inside / outside determination unit B1010 determines whether the center of the face frame of the tracking target person is outside the exit determination area. That is, in step S007, it is determined whether the tracking target person who is present inside the predetermined area in step S006 has exited the exit determination area within the predetermined area.

[0108] If the answer is Yes in step S007, that is, if the tracking target person has exited the exit determination area within the predetermined area, the process proceeds to step S010 in Fig. 14 via B in Fig. 13. If the answer is No in step S007, that is, if the tracking target person has not exited the exit determination area within the predetermined area but continues to remain within the predetermined area, that information is output to the angle of view operation calculation unit A1011, and the process proceeds to step S012 in Fig. 14 via C in Fig. 13.

[0109] 14, the area inside / outside determination unit B1010 determines whether the center of the face frame of the tracking target person is inside any of the intrusion determination areas, and outputs the determination result to the field of view operation calculation unit A1011. That is, in step S008, it is determined whether the tracking target person, who was not inside the predetermined area in step S006, has intruded into the intrusion determination area within the predetermined area.

[0110] If the answer to step S008 is Yes, that is, if the tracking target has entered the intrusion determination area within the predetermined area, proceed to step S009. If the answer to step S008 is No, that is, if the tracking target has not entered the intrusion determination area within the predetermined area, proceed to step S010.

[0111] In step S009, the view angle operation calculation unit A1011 calculates a view angle control value for changing the view angle to a predetermined area where the tracking target has entered, outputs the calculated value to the view angle operation unit A1012, and generates a PTZ control command. Then, the process proceeds to step S011.

[0112] Meanwhile, in step S010, the angle of view operation calculation unit A1011 calculates an angle of view control value for displaying the tracking target within the screen in accordance with the predicted position of the tracking target and the composition setting acquired from the tracking setting unit A1007, outputs the calculated value to the angle of view operation unit A1012, and generates a PTZ control command. Then, the process proceeds to step S011.

[0113] In step S011, the angle of view operation unit A1012 outputs a PTZ control command to the PTZ drive device A1003 based on the angle of view control value input from the angle of view operation calculation unit A1011. The PTZ drive device A1003 also changes the shooting angle of view by PTZ driving. Then, the process proceeds to step S012.

[0114] In step S012, the video output unit A1013 outputs the input video information to the external device A1014. Then, the process proceeds to step S013.

[0115] In step S013, it is determined whether a user has performed a stop operation to turn off the imaging device. If the result of step S013 is No, the process returns to step S001 in Fig. 13 via D in Fig. 14, and if the result of step S013 is Yes, the automatic photography process is terminated and the flow in Fig. 13 and Fig. 14 is terminated.

[0116] As described above, in the second embodiment, when tracking processing is performed based on face detection, the intrusion determination area and exit determination area are dynamically resized according to the size of the face of the tracking target person located inside the specified area.

[0117] Furthermore, when the center of the face frame (or face) of the tracking target enters the inside of the entry determination area, it is determined that the target has entered, and when the center of the face frame (or face) of the tracking target exits the outside of the exit determination area, it is determined that the target has exited. This makes it possible to control the angle of view while continuing tracking without losing sight of the face when an exit is determined.

[0118] (Embodiment 3) Next, an automatic photography system C1000 according to a third embodiment of the present invention will be described with reference to Fig. 15. Fig. 15 is a functional block diagram of the automatic photography system C1000 according to the third embodiment of the present invention. In the third embodiment, a plurality of predetermined areas are set, and processing is performed to seamlessly switch fixed angle of view control when the tracking target person moves through an area where the plurality of predetermined areas overlap.

[0119] The automatic photography system C1000 of this embodiment includes an image capturing device A1001, a PTZ driving device A1003, an imaging device C1020 including an information processing device C1004, and an external device A1014.

[0120] The imaging device and the external device A1014 are connected via a network. The imaging device and the external device A1014 may be connected via a video interface.

[0121] The imaging device of this embodiment performs tracking processing based on an image acquired from an image acquisition device A1001 and a tracking setting acquired from a user input acquisition unit A1002. Then, the angle of view is controlled using a PTZ driving device A1003 so that the tracking target does not move out of the angle of view.

[0122] When controlling the angle of view, the system determines whether the tracking target is within a predetermined area, and switches the angle of view control depending on whether the tracking target is within the predetermined area or outside the predetermined area.Then, the image capturing results are displayed on the external device A1014.

[0123] The information processing device C1004 performs tracking processing based on the coordinates of the face detected using the input video and the composition setting input by the user. Also, if the tracking target is within the area input by the user, the angle of view is moved to that area and then fixed, and when the tracking target leaves the predetermined area, the angle of view control is performed based on the composition setting, and the angle of view control is switched according to the position of the tracking target.

[0124] The information processing device C1004 has an image acquisition unit A1005, a face detection unit A1006, a tracking setting unit A1007, a tracking setting recording unit A1008, a tracking processing unit A1009, an area inside / outside determination unit C1010, and a field of view operation calculation unit A1011. Furthermore, the information processing device C1004 has a field of view operation unit A1012 and a video output unit A1013.

[0125] The area inside / outside determination unit C1010 determines whether the tracking target is inside or outside a predetermined area based on the tracking processing result input from the tracking processing unit A1009 and the area setting input from the tracking setting unit.

[0126] In the third embodiment, in order to seamlessly switch fixed angle of view control when the tracking target moves through an area where multiple predetermined areas overlap, when the tracking target leaves a predetermined area, it is determined whether the tracking target is inside an intrusion determination area of ​​another predetermined area. This will be explained with reference to Fig. 16. Fig. 16 is a diagram for explaining the area inside / outside determination when the tracking target moves through an area where multiple predetermined areas overlap, according to the third embodiment of the present invention.

[0127] A first predetermined area P801 and a second predetermined area P804 are set to overlap each other. The predetermined areas P801 and P804 have intrusion determination areas P802 and P805, respectively, and exit determination areas P803 and P806, respectively. When the tracking target moves from position P807 to position P808, it is determined that the tracking target is within the first predetermined area P801.

[0128] Next, when the tracking target person P809 moves from position P808 to position P809, the center position P811 of the face frame P810 of the tracking target person P809 is outside the exit determination area P803 of the set first predetermined area P801. Therefore, it is determined that the tracking target person P809 has exited the first predetermined area P801.

[0129] However, since the central position P811 is located inside the intrusion determination area P805 of the second predetermined area P804, it is determined that the object has intruded into the second predetermined area P804. In this manner, in this embodiment, when multiple predetermined areas are set, it is possible to seamlessly switch the angle of view control by determining whether the object has intruded into another predetermined area after determining that the object has exited the predetermined area.

[0130] The other determination operations are the same as those of the area inside / outside determination unit B 1010, and therefore descriptions thereof will be omitted. Furthermore, the functions of the other blocks are the same as those of the first and second embodiments, and therefore descriptions thereof will be omitted.

[0131] Next, the procedure for processing the imaging device will be described with reference to the flowcharts of Fig. 17 and Fig. 18. Fig. 17 is a flowchart showing the processing procedure of the imaging device according to the third embodiment of the present invention, and Fig. 18 is a flowchart showing the processing procedure continued from Fig. 17. Note that the operation of each step in the flowcharts of Fig. 17 and Fig. 18 is performed by a CPU or the like serving as a computer in the imaging device executing a computer program stored in memory.

[0132] 17 and 18 starts when the image capture device of the automatic photography system C1000 is activated by a user operation. First, in step S001, the video acquisition unit A1005 acquires video information from the video capture device A1001 and outputs it to the face detection unit A1006 and the video output unit A1013. Then, the process proceeds to step S002.

[0133] In step S002, the user input acquisition unit A1002 acquires area information and composition settings (settings indicating the position where the tracking target is displayed on the screen, the display size, etc.) set by the user, and automatic tracking target selection settings, and outputs these to the tracking setting unit A1007. The tracking setting unit A1007 outputs the input area information, composition settings, and automatic tracking target selection settings to the tracking setting recording unit A1008 and records them. Then, the process proceeds to step S003.

[0134] In step S003, the face detection unit A1006 performs image recognition based on the input video information to detect faces as human body detection, and outputs the detected face information and video information to the tracking processing unit A1009. Then, the process proceeds to step S004.

[0135] In step S004, the tracking setting unit A 1007 selects a tracking target from the input face information and performs tracking processing. Also, as a tracking processing result, it outputs the coordinate information of the tracking target person to the area inside / outside determination unit C 1008. Then, the process proceeds to step S005.

[0136] In step S005, the tracking setting unit A1007 determines whether the tracking process of the human body (face) has been successful. If the answer is Yes in step S005, the process proceeds to step S114. If the answer is No in step S005, the process proceeds to step S012 in FIG. 18 via C in FIG. 17.

[0137] In step S114, the area inside / outside determination unit C1008 dynamically determines the size of the intrusion determination area and the exit determination area of ​​the set area according to the size of the face of the tracking target person. This process is the same as in embodiment 2. Note that this step may not be necessary in this embodiment. Then, the process proceeds to step S006.

[0138] In step S006, the area inside / outside determination unit C1008 determines whether the tracking target has entered the predetermined area. This determination process is a process of first confirming whether the tracking target is already inside the predetermined area.

[0139] If multiple predetermined areas are set, it is confirmed whether the tracking target is present inside any of the predetermined areas. If the answer is Yes in step S006, that is, if the tracking target is present inside a predetermined area, the process proceeds to step S007. If the answer is No in step S006, that is, if the tracking target is not present inside a predetermined area, the process proceeds to step S008 in FIG. 18 via A in FIG. 17.

[0140] In step S007, the area inside / outside determination unit C1008 determines whether the center of the face frame (or face) of the tracking target person is outside the exit determination area. That is, in step S007, it is determined whether the tracking target person who is present inside the predetermined area in step S006 has exited the exit determination area within the predetermined area.

[0141] If the answer is Yes in step S007, that is, if the tracking target person has exited the exit determination area within the predetermined area, the process proceeds to step S008 in Fig. 18 via A in Fig. 17. If the answer is No in step S007, that is, if the tracking target person has not exited the exit determination area within the predetermined area but continues to remain within the predetermined area, that information is output to the angle of view operation calculation unit A1011, and the process proceeds to step S012 in Fig. 18 via C in Fig. 17.

[0142] 18, the area inside / outside determination unit C1008 determines whether the center of the face frame (or face) of the tracking target person is inside the intrusion determination area of ​​any area, and outputs the determination result to the field of view operation calculation unit A1011. That is, in step S008, it is determined whether the tracking target person, who was not inside the predetermined area in step S006, has intruded into the intrusion determination area within the predetermined area.

[0143] If the answer to step S008 is Yes, that is, if the tracking target has entered the intrusion determination area within the predetermined area, proceed to step S009. If the answer to step S008 is No, that is, if the tracking target has not entered the intrusion determination area within the predetermined area, proceed to step S010.

[0144] In step S009, the view angle operation calculation unit A1011 calculates a view angle control value for changing the view angle to a predetermined area where the tracking target has entered, outputs the calculated value to the view angle operation unit A1012, and generates a PTZ control command. Then, the process proceeds to step S011.

[0145] Meanwhile, in step S010, the angle of view operation calculation unit A1011 calculates an angle of view control value for displaying the tracking target within the screen in accordance with the predicted position of the tracking target and the composition setting acquired from the tracking setting unit A1007, outputs the calculated value to the angle of view operation unit A1012, and generates a PTZ control command. Then, the process proceeds to step S011.

[0146] In step S011, the angle of view operation unit A1012 outputs a PTZ control command to the PTZ drive device A1003 based on the angle of view control value input from the angle of view operation calculation unit A1011. The PTZ drive device A1003 also changes the shooting angle of view by driving the PTZ. Then, the process proceeds to step S012.

[0147] In step S012, the video output unit A1013 outputs the input video information to the external device A1014. Then, the process proceeds to step S013.

[0148] In step S013, it is determined whether a user has performed a stop operation to turn off the imaging device. If the result of step S013 is No, the process returns to step S001 in Fig. 17 via D in Fig. 18, and if the result of step S013 is Yes, the automatic photography process is terminated and the flow in Fig. 17 and Fig. 18 is terminated.

[0149] As described above, in the third embodiment, when a tracking target moves through an area where multiple predetermined areas overlap, it is possible to seamlessly switch the angle of view control by determining that the target has entered another overlapping predetermined area immediately after exiting one predetermined area.

[0150] (Embodiment 4) An automatic photography system D1000 according to a fourth embodiment of the present invention will be described with reference to Fig. 19. Fig. 19 is a functional block diagram of the automatic photography system according to the fourth embodiment of the present invention. In the fourth embodiment, control processing is performed to keep the zoom magnification substantially unchanged before the tracking target person enters a predetermined area and after the tracking target person leaves the predetermined area.

[0151] The automatic photography system D1000 of this embodiment has an image capturing device D1020 including an image capturing device A1001, a PTZ driving device A1003, and an information processing device D1004, and an external device A1014.

[0152] The imaging device and the external device A 1014 are connected via a network, or may be connected via a video interface.

[0153] The imaging device of this embodiment performs tracking processing based on an image acquired from an image acquisition device A1001 and a tracking setting acquired from a user input acquisition unit A1002. Then, the angle of view is controlled using a PTZ driving device A1003 so that the tracking target does not move out of the angle of view.

[0154] When controlling the angle of view, the system determines whether the tracking target is within a predetermined area, and switches the angle of view control depending on whether the tracking target is within the predetermined area or outside the predetermined area.Then, the image capturing results are displayed on the external device A1014.

[0155] The information processing device D1004 performs image analysis processing, determination processing, etc. The information processing device D1004 performs tracking processing based on the coordinates of a face detected using the input video and the composition settings input by the user.

[0156] Furthermore, if the person to be tracked is within the area (predetermined area) input by the user, the angle of view is changed to that area and then fixed, and when the person to be tracked leaves the predetermined area, the angle of view control is switched according to the position of the person to be tracked so that the angle of view is controlled based on the composition setting.

[0157] The information processing device D1004 has an image acquisition unit A1005, a face detection unit A1006, a tracking setting unit A1007, a tracking setting recording unit A1008, a tracking processing unit A1009, an area inside / outside determination unit D1010, a zoom magnification recording unit D1015, and a zoom magnification setting unit D1016. Furthermore, the information processing device C1004 has a field of view operation calculation unit D1011, a field of view operation unit A1012, and a video output unit A1013.

[0158] The area inside / outside determination unit D1010 determines whether the tracking target is inside or outside a predetermined area based on the tracking processing result input from the tracking processing unit A1009 and the area setting and composition setting input from the tracking setting unit. Note that among the composition settings, it is possible to turn off adjustment of the size at which the tracking target is displayed on the screen.

[0159] This is used when you want to shoot while maintaining the zoom magnification at the start of shooting, etc. In other words, if the tracking target person enters a predetermined area and is controlled to a fixed angle of view to maintain the angle of view, and then leaves the predetermined area and the angle of view is controlled based on the composition setting, the zoom magnification will change before entering the predetermined area and after leaving the predetermined area.

[0160] In this embodiment, in order to control the zoom magnification so that it does not change before entering a specified area and after leaving the specified area, the zoom magnification of the shooting angle of view is recorded when the tracking target person enters the specified area.

[0161] 5, when the tracking target person moves from P303 to P304 and enters a predetermined area, the zoom magnification of D301, which is the shooting angle of view before entering the predetermined area, is recorded in zoom magnification recording unit D1015. Other area determination operations are the same as those of area inside / outside determination unit B1010, and therefore description thereof will be omitted.

[0162] The zoom magnification recording unit D1015, which serves as a zoom magnification setting means, records the zoom magnification input from the area inside / outside determination unit D1010 and outputs it to the zoom magnification setting unit D1016. The zoom magnification setting unit D1016 sets the zoom magnification of the image capture device A1001 to approximately the same zoom magnification as the zoom magnification before the object entered the specified area. The zoom magnification setting unit D1016 outputs the set zoom magnification to the angle of view operation calculation unit D1011.

[0163] The angle of view operation calculation unit D1011 calculates the angle of view value to be controlled based on the determination result and area information input from the area inside / outside determination unit D1010, the composition setting input from the tracking setting unit A1007, and the zoom magnification input from the zoom magnification setting unit D1016.

[0164] Assume that when it is determined that the tracking target person has left a predetermined area, the automatic adjustment of the size at which the tracking target person is displayed on the screen in the composition setting is turned off. In this case, the angle of view control value is calculated based on the zoom magnification read from the zoom magnification recording unit D1015 and the position setting for displaying the human body on the screen input from the tracking setting unit A1007.

[0165] On the other hand, if automatic size adjustment is not turned off, the angle of view control value is calculated based on the composition setting input from the tracking setting unit A1007. The calculation of the angle of view control value other than this is the same as that of the angle of view operation calculation unit A1011, so its description will be omitted. Furthermore, the functions of the other blocks are also the same as those of the first embodiment, so their description will be omitted.

[0166] Fig. 20 is a flowchart showing the processing procedure of the imaging device according to the fourth embodiment of the present invention, and Fig. 21 is a flowchart showing the processing procedure continued from Fig. 20. Note that the operation of each step in the flowcharts of Fig. 20 and Fig. 21 is performed by a CPU or the like serving as a computer in the imaging device executing a computer program stored in memory.

[0167] 20 and 21 begins when the image capture device D1020 of the automatic photography system D1000 is activated by a user operation. First, in step S001, the video acquisition unit A1005 acquires video information from the video capture device A1001 and outputs it to the face detection unit A1006 and the video output unit A1013. Then, the process proceeds to step S002.

[0168] In step S002, the user input acquisition unit A1002 acquires area information and composition settings (settings related to the position where the tracking target is displayed on the screen, the display size, etc.) set by the user, and automatic selection settings for the tracking target, and outputs them to the tracking setting unit A1007. The tracking setting unit A1007 outputs the input area information, composition settings, and automatic selection settings for the tracking target to the tracking setting recording unit A1008 and records them. Then, the process proceeds to step S003.

[0169] In step S003, the face detection unit A1006 performs image recognition based on the input video information to detect faces as human body detection. The detected face information and video information are output to the tracking processing unit A1009. Then, the process proceeds to step S004.

[0170] In step S004, the tracking setting unit A1007 selects a tracking target from the input face information and performs tracking processing. Also, as a tracking processing result, it outputs the coordinate information of the tracking target person to the area inside / outside determination unit D1008. Then, the process proceeds to step S005.

[0171] In step S005, the tracking setting unit A1007 determines whether the tracking process of the human body (face) has been successful. If the answer is Yes in step S005, the process proceeds to step S114. If the answer is No in step S005, the process proceeds to step S012 in FIG. 21 via C in FIG. 20.

[0172] In step S114, the area inside / outside determination unit D1008 dynamically determines the size of the intrusion determination area and the exit determination area of ​​the set area according to the size of the face of the tracking target person. This process is the same as in embodiment 2. Note that this step may not be necessary in this embodiment. Then, the process proceeds to step S006.

[0173] In step S006, the area inside / outside determination unit D1008 determines whether the tracking target has entered the inside of a predetermined area. This determination process is a process of first confirming whether the tracking target is already inside the predetermined area. Note that if multiple predetermined areas are set, it is confirmed whether the tracking target is inside any of the predetermined areas.

[0174] If the answer is Yes in step S006, that is, if the tracking target is present inside the predetermined area, proceed to step S007. If the answer is No in step S006, that is, if the tracking target is not present inside the predetermined area, proceed to step S008 in Fig. 21 via A in Fig. 20.

[0175] In step S007, the area inside / outside determination unit D1008 determines whether the center of the face frame (or face) of the tracking target person is outside the exit determination area. That is, in step S007, it is determined whether the tracking target person who is present inside the predetermined area in step S006 has exited the exit determination area within the predetermined area.

[0176] If the answer is Yes in step S007, that is, if the tracking target person has exited the exit determination area within the predetermined area, the process proceeds to step S008 in Fig. 21 via A in Fig. 20. If the answer is No in step S007, that is, if the tracking target person has not exited the exit determination area within the predetermined area but continues to remain within the predetermined area, the process outputs that information to the angle of view operation calculation unit D1011 and proceeds to step S012 in Fig. 21 via C in Fig. 20.

[0177] In step S008 of FIG. 21, the area inside / outside determination unit D1008 determines whether the center of the face frame (or face) of the tracking target person is inside the intrusion determination area of ​​a predetermined area, and outputs the determination result to the view angle operation calculation unit D1011.

[0178] That is, in step S008, it is determined whether the tracking target person who was not present inside the predetermined area in step S006 has entered an intrusion determination area within the predetermined area, or whether the tracking target person who exited the exit determination area in step S007 has again entered the intrusion determination area within the predetermined area.

[0179] If the answer to step S008 is Yes, that is, if the tracking target has entered the intrusion determination area within the predetermined area, proceed to step S315. If the answer to step S008 is No, that is, if the tracking target has not entered the intrusion determination area within the predetermined area, proceed to step S316.

[0180] In step S315, the area inside / outside determination unit D1008 outputs the current zoom magnification to the zoom magnification recording unit D1015 and records it. That is, the size of the shooting range before the size of the shooting range was changed is stored. Then, the process proceeds to step S009.

[0181] On the other hand, in step S316, the angle of view operation calculation unit D1011 determines whether the adjustment setting for the size of the tracking target person displayed on the screen in the composition setting is off. If the answer is No in step S316, proceed to step S010. If the answer is Yes in step S316, proceed to step S317.

[0182] In step S317, the angle of view operation calculation unit D1011 sets the angle of view (control value) based on the predicted position of the person to be tracked, the position setting for displaying the person to be tracked within the screen obtained from the tracking setting unit A1007, and the zoom magnification obtained from the zoom magnification recording unit D1015.

[0183] That is, when the subject leaves the predetermined area, the size of the shooting range (zoom magnification) is controlled to be the same as that stored in step S315 at the time of the predetermined entry. Then, the calculated angle of view control value is output to the angle of view operation unit A1012, and a PTZ control command is generated. Then, the process proceeds to step S011.

[0184] 22 is a diagram illustrating that when it is determined that the tracking target person has left a predetermined area, the angle of view is controlled in accordance with the zoom magnification and composition setting recorded at the time of intrusion into the predetermined area according to the fourth embodiment of the present invention. In the example shown in Fig. 22, when the position P901 of the tracking target person is present within the intrusion determination area P906, the angle of view is controlled to be the angle of view corresponding to the predetermined area (the set area).

[0185] However, when the person to be tracked moves to position P902 and the center P903 of the face frame P904 goes outside the exit determination area 905, the angle of view is controlled to D902 based on the zoom magnification and composition setting stored in the zoom magnification recording unit D1015 at the time of entry into the specified area.

[0186] Meanwhile, in step S009, the view angle operation calculation unit D1011 calculates a view angle control value for changing the view angle to a predetermined area where the tracking target has entered, outputs the calculated value to the view angle operation unit D1012, and generates a PTZ control command. Then, the process proceeds to step S011.

[0187] In step S010, the angle of view operation calculation unit D1011 calculates an angle of view control value for displaying the tracking target within the screen in accordance with the composition setting acquired from the tracking setting unit A1007, outputs the calculated value to the angle of view operation unit A1012, and generates a PTZ control command. Then, the process proceeds to step S011.

[0188] In step S011, the angle of view operation unit A1012 outputs a PTZ control command to the PTZ drive device A1003 based on the angle of view control value input from the angle of view operation calculation unit D1011. The PTZ drive device A1003 also changes the shooting angle of view by PTZ driving. Then, the process proceeds to step S012. In step S012, the video output unit A1013 outputs the input video information to the external device A1014. Then, the process proceeds to step S013.

[0189] In step S013, it is determined whether a user has performed a stop operation to turn off the imaging device. If No in step S013, the process returns to step S001 in Fig. 20 via D in Fig. 21, and if Yes in step S013, the automatic photography process is terminated and the flow in Fig. 20 and Fig. 21 is terminated.

[0190] Here, the zoom magnification recorded (stored) in step S315 is read out in step S317 next time and used to control the photographing range (zoom magnification) in step S011.

[0191] That is, when the tracking target person enters the specified area, the imaging range of the imaging means is changed, and the size of the imaging range before the angle of view is changed is stored, and when the tracking target person leaves the specified area, the imaging range is controlled so that it becomes the size of the stored imaging range.

[0192] As described above, in the fourth embodiment, when the size setting for displaying the tracking target person on the screen as a composition setting is turned off, the zoom magnification when the tracking target person enters a predetermined area is recorded.

[0193] Then, when it is determined that the tracking target has left the predetermined area, the angle of view is controlled in accordance with the recorded zoom magnification and the setting of the position at which the tracking target is to be displayed on the screen as a composition setting, thereby enabling the angle of view to be controlled so that the zoom magnification remains substantially the same before entering the predetermined area and after leaving the predetermined area.

[0194] That is, in the fourth embodiment, when an object enters a predetermined area, the imaging range of the imaging means is changed, and the size of the imaging range (zoom magnification) before the change (before the object entered) is stored. Then, when it is determined that the object has left the predetermined area, the zoom magnification of the imaging means is set to the same zoom magnification as the size of the imaging range (zoom magnification) that has been stored. Therefore, even if an object frequently enters and leaves the predetermined area, tracking can be performed with a natural imaging range size.

[0195] Steps S006 to S008, S316, etc. function as determination steps for determining whether or not the object has exited from the inside of a predetermined area. In these determination steps, the exit determination is made based on whether or not the object has exited from an exit determination area that is smaller than the predetermined area.

[0196] Furthermore, step S317 functions as a zoom magnification setting step for setting the zoom magnification of the imaging means to the same zoom magnification as the zoom magnification before the object entered the predetermined area when it is determined that the object has left the predetermined area. Note that, in the above, the "same zoom magnification" is not limited to the exact same zoom magnification, but also includes approximately the same zoom magnification.

[0197] In the fourth embodiment, when an intrusion into a predetermined area is detected, the intrusion is detected using an intrusion determination area, and when an exit to the outside of the predetermined area is detected, the exit is detected using an exit determination area that is larger than the intrusion determination area. However, the fourth embodiment can be applied even when the intrusion determination area and the exit determination area are the same size, and is not limited to the sizes of the intrusion determination area and the exit determination area.

[0198] In the configuration of the fourth embodiment, when it is determined that the person to be tracked has left a predetermined area and angle of view control is performed in accordance with the composition setting and zoom magnification, PT control may be performed in accordance with the composition setting while returning to the recorded zoom magnification.

[0199] In addition, in the configurations of the first to fourth embodiments, when the tracking target is lost while the tracking target is entering a predetermined area, the tracking target may be searched for by moving to a pre-specified angle of view. In addition, in the configurations of the first to fourth embodiments, when the tracking target is about to fall below the minimum detection size while the tracking target is entering a predetermined area, the fixed angle of view may be released and the tracking target may be zoomed in so as not to lose sight of the tracking target.

[0200] (Embodiment 5) Next, the hardware configuration of the imaging device will be described with reference to Fig. 23. An example of the hardware configuration of the imaging device will be described using the block diagram of Fig. 23. As shown in Fig. 23, the imaging device can be configured to include a CPU 2300, a main storage device 2301, an auxiliary storage device 2302, a PTZ driving device A1003, and an image capturing device A1001. Note that the configuration shown in Fig. 23 is merely one example of a configuration applicable to the imaging device, and can be modified / altered as appropriate.

[0201] The CPU 2300 serving as the information processing device A1004 executes processing using computer programs and data stored in the main storage device 212. As a result, the CPU 211 controls the operation of the entire imaging device, and also executes or controls each of the above-mentioned processes executed by the imaging device.

[0202] For example, the CPU 2300 executes processing using computer programs and data stored in the main memory device 2301, thereby realizing the functions of each functional unit of the information processing device A1004 shown in Figures 1, 10, 15, and 19.

[0203] The main memory device 212 is a memory device such as a RAM (Random Access Memory). The main memory device 2301 stores computer programs and data loaded from the auxiliary memory device 2302. It also has an area for storing captured images acquired by the video acquisition device A1001 and various data received from the external device A1014 via the network I / F 2303.

[0204] Furthermore, the main memory device 2301 has a work area that is used when the CPU 2300 executes various processes. In this way, the main memory device 2301 can provide various areas as needed.

[0205] The auxiliary storage device 2302 is a large-capacity information storage device such as a hard disk drive (HDD), a read-only memory (ROM), or a solid-state drive (SSD).

[0206] The auxiliary storage device 2302 stores an OS (operating system) and computer programs and data for causing the CPU 2300 to execute or control the various processes described above as those performed by the imaging device. The auxiliary storage device 2302 also stores data (e.g., the imaging parameters described above) received from the external device A 1014 via the network I / F 2303.

[0207] Computer programs and data stored in the auxiliary storage device 2302 are loaded into the main storage device 2301 as appropriate under the control of the CPU 2300 and are then processed by the CPU 2300 .

[0208] As described above, the PTZ driving device A1003 has the function of changing the angle of view of the imaging device and is configured with a driving device such as a motor for performing at least one of pan, tilt, and zoom control. As described above, the image capturing device A1001 is a part that captures the surroundings and generates captured images, and is configured with a camera, etc. The network I / F 2303 is an interface used by the imaging device to perform data communication with an external device A1014 via a network.

[0209] Although the present invention has been described in detail above based on the preferred embodiments, the present invention is not limited to the above embodiments, and various modifications are possible based on the spirit of the present invention, and these modifications are not excluded from the scope of the present invention. The above embodiments include the following combinations.

[0210] (Configuration 1) An information processing device characterized by having a detection means for detecting a predetermined object from an image captured by an imaging means, a setting means for setting a predetermined area and an exit determination area smaller than the predetermined area in the image, a determination means for making an exit determination based on whether the object has exited the exit determination area, and a control means for controlling the shooting range of the imaging means in accordance with the result determined by the determination means.

[0211] (Configuration 2) The information processing device according to Configuration 1, characterized in that the setting means sets an intrusion determination area that is smaller than the exit determination area, and the determination means makes an intrusion determination based on whether the object has intruded into the intrusion determination area.

[0212] (Configuration 3) The information processing device according to Configuration 2, wherein the control means controls the photographing range so that the predetermined area is included in the photographing range when the determination means determines that the object has entered the intrusion determination area.

[0213] (Configuration 4) The information processing device according to Configuration 3, wherein the control means controls the photographing range so that the photographing range is in a position and size corresponding to the predetermined area when the determination means determines that the object has entered the intrusion determination area.

[0214] (Configuration 5) An information processing device described in any one of configurations 1 to 4, characterized in that the setting means is capable of setting the display position and display size of the object in the image, and when the determination means determines that the object has exited the exit determination area, the control means controls the shooting range so that the object is at the display position and display size set by the setting means.

[0215] (Configuration 6) The information processing device according to any one of configurations 2 to 4, wherein the determination means determines that an intrusion has occurred when the center of the face of the target object is inside the intrusion determination area.

[0216] (Configuration 7) The information processing device according to any one of configurations 1 to 6, wherein the determination means determines that the target has exited when the center of the target's face is outside the exit determination area.

[0217] (Configuration 8) The information processing device according to any one of Configurations 2 to 4 or Configuration 6, characterized in that the determination means changes the size of at least one of the intrusion determination area and the exit determination area depending on the size of the object within the shooting range.

[0218] (Configuration 9) An information processing device described in any one of configurations 1 to 8, characterized in that, when multiple predetermined areas are set, the determination means determines whether the object has entered another predetermined area after determining that the object has left the predetermined area.

[0219] (Configuration 10) An information processing device comprising: a detection means for detecting a predetermined object from an image captured by an imaging means; a setting means for setting a predetermined area for the image; a judgment means for judging whether the object is present within the predetermined area; and a control means for controlling the imaging range of the imaging means in accordance with the result of the judgment by the judgment means, wherein the judgment means makes an exit judgment based on whether the object has exited an exit judgment area smaller than the predetermined area, and makes an intrusion judgment based on whether the object has intruded into an intrusion judgment area smaller than the exit judgment area.

[0220] (Configuration 11) The information processing device described in Configuration 10, characterized in that the control means controls the shooting range so that the specified area is included in the shooting range when the determination means determines that the object is present within the specified area.

[0221] (Configuration 12) The information processing device described in Configuration 11, characterized in that the control means controls the shooting range so that the shooting range is in a position and size corresponding to the specified area when the determination means determines that the object is present within the specified area.

[0222] (Configuration 13) An information processing device described in any one of configurations 10 to 12, characterized in that the setting means is capable of setting the display position and display size of the object in the image, and when the determination means determines that the object does not exist within the specified area, the control means controls the shooting range so that the object is at the display position and display size set by the setting means.

[0223] (Configuration 14) The information processing device according to any one of configurations 10 to 13, wherein the determination means determines that an intrusion has occurred when the center of the face of the target object is inside the intrusion determination area.

[0224] (Configuration 15) The information processing device according to any one of configurations 10 to 14, wherein the determination means determines that the target has exited when the center of the target's face is outside the exit determination area.

[0225] (Configuration 16) The information processing device described in any one of configurations 10 to 15, characterized in that the determination means changes the size of at least one of the intrusion determination area and the exit determination area depending on the size of the object within the shooting range.

[0226] (Configuration 17) The information processing device described in any one of configurations 10 to 16, characterized in that, when multiple predetermined areas are set, the determination means determines whether the object has entered another predetermined area after determining that the object has left the predetermined area.

[0227] (Method 1) An information processing method comprising: a detection step of detecting a predetermined object from an image captured by an imaging means; a setting step of setting a predetermined area and an exit judgment area smaller than the predetermined area in the image; a determination step of making an exit judgment based on whether the object has exited the exit judgment area; and a control step of controlling the shooting range of the imaging means in accordance with the result determined by the determination step.

[0228] (Method 2) An information processing method comprising a detection step of detecting a predetermined object from an image captured by an imaging means, a setting step of setting a predetermined area for the image, a determination step of determining whether the object is present within the predetermined area, and a control step of controlling the shooting range of the imaging means in accordance with the result determined by the determination step, wherein the determination step makes an exit determination based on whether the object has exited an exit determination area that is smaller than the predetermined area, and makes an intrusion determination based on whether the object has intruded into an intrusion determination area that is smaller than the exit determination area.

[0229] (Program) A computer program for controlling each means of the information processing device according to any one of configurations 1 to 17 by a computer.

[0230] In order to realize part or all of the control in the above-described embodiments, a computer program that realizes the functions of the above-described embodiments may be supplied to an information processing device or the like via a network or various storage media. Then, a computer (or a CPU, MPU, or the like) in the information processing device or the like may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of symbols]

[0231] A1000: Automatic photography system A1001: Image acquisition device A1002: User input acquisition section A1003:PTZ drive unit A1004: Information processing device A1005: Video acquisition unit A1006: Face detection unit A1007: Tracking setting section A1008: Tracking setting recorder A1009: Tracking processing unit A1010: Area outside / outside determination section A1011: Angle of view operation calculation unit A1012: Viewing angle control unit A1013:Video output section A1014: External device

Claims

1. a detection means for detecting an object from an image captured by the imaging means; a setting means for setting a predetermined area and an exit determination area smaller than the predetermined area; a control unit for controlling the imaging range of the imaging unit, the setting means is capable of setting a display position and a display size of the object in the image; The information processing device is characterized in that the control means controls the imaging range so that the object is at the display position and the display size set by the setting means when the object is outside the exit determination area.

2. An information processing device as described in Claim 1, further comprising a determination means for determining whether the object is located outside the exit determination area.

3. the setting means sets an entry determination area smaller than the exit determination area, 3. The information processing apparatus according to claim 2, wherein the determining means determines whether the object is present inside the intrusion determination area.

4. 4. The information processing apparatus according to claim 3, wherein the control means controls the imaging means so that an image is captured within a predetermined imaging range when the object is present inside the intrusion determination area.

5. a detection means for detecting an object from an image captured by the imaging means; a setting means for setting a predetermined area, an exit determination area smaller than the predetermined area, and an intrusion determination area smaller than the exit determination area; and a control means for controlling the imaging direction of the imaging means so as to track the object based on its position when the object is outside the exit determination area, and for controlling the imaging means so as to capture an image within a predetermined imaging range without tracking the object when the object is inside the intrusion determination area.

6. When the object is inside the exit determination area and outside the entry determination area, The information processing device according to claim 5, characterized in that the control means, when the tracking is being performed, continues to control the tracking, and when the tracking is not being performed, continues to control the imaging means to capture an image in the predetermined imaging range without performing the tracking.

7. An information processing device as described in Claim 5, further comprising a determination means for determining whether the object is present outside the exit determination area and whether the object is present inside the intrusion determination area.

8. An information processing device as described in claim 4 or 5, characterized in that the specified imaging range is an imaging range based on the specified area.

9. An information processing device as described in claim 4 or 5, characterized in that the specified imaging range is an imaging range whose position and size correspond to the specified area.

10. the setting means is capable of setting a display position and a display size of the object in the image; The information processing device according to claim 5, characterized in that the control means controls the imaging direction so that the object is at the display position and the display size set by the setting means when the object is outside the exit determination area.

11. 8. The information processing apparatus according to claim 3, wherein the determining means determines that the object is present inside the intrusion determination area when the center of the face of the object is present inside the intrusion determination area.

12. 8. The information processing apparatus according to claim 2, wherein the determining means determines that the object is present outside the exit determination area when the center of the face of the object is present outside the exit determination area.

13. An information processing device as described in claim 1 or 5, characterized in that the size of the exit judgment area is changed depending on the size of the object within the imaging range.

14. An information processing device as described in claim 3 or 5, characterized in that the size of the intrusion determination area is changed depending on the size of the object within the imaging range.

15. a detection step of detecting an object from an image captured by the imaging means; a setting step of setting a predetermined area and an exit determination area smaller than the predetermined area; a control step of controlling an imaging range of the imaging means, A display position and a display size of the object in the image are set; The information processing method is characterized in that the control step controls the imaging range so that the object is at the set display position and display size when the object is outside the exit judgment area.

16. a detection step of detecting an object from an image captured by the imaging means; a setting step of setting a predetermined area, an exit determination area smaller than the predetermined area, and an intrusion determination area smaller than the exit determination area; and a control step of controlling the imaging direction of the imaging means so that, if the object is present outside the exit determination area, the imaging direction of the imaging means is controlled to track the object based on its position, and, if the object is present inside the intrusion determination area, controlling the imaging means so that the object is imaged within a predetermined imaging range without tracking.

17. A computer program for causing a computer to execute the information processing method according to claim 15 or 16.

Images