Information processing device, information processing method, and program

Abstract

The aim is to simplify the control of the imaging range so that the subject is positioned in the location desired by the user within the imaging range. [Solution] The system comprises an acquisition means for acquiring an image captured by an imaging means, a display control means for displaying the image acquired by the acquisition means on a display means, and a control means for controlling the imaging range of the imaging means. The display control means displays on the display means information of a first placement candidate in which the subject is placed at a first position within the imaging range, and information of a second placement candidate in which the subject is placed at a second position different from the first position within the imaging range. The control means controls the imaging range based on the information of the placement candidate selected by the user from the first and second placement candidates.

Description

[[Technical Field]]

[0001] The present invention relates to an information processing method. [[Background Art]]

[0002] Currently, in the video production market, an increasing number of systems control an imaging device connected to a network from a remote location. In such systems, there exist applications that allow changing the range captured by the imaging device (hereinafter referred to as the imaging range) by controlling the PTZ (Pan-Tilt-Zoom) of the imaging device while displaying the video captured by the imaging device. A user using such an application can change the imaging range using a device such as a joystick.

[0003] Furthermore, as a method for controlling the imaging range of an imaging device, there is a method of controlling PTZ based on the position and size of a subject. Patent Document 1 discloses that in order to capture a subject within a desired imaging range, the ratio of the entire region of the subject to a partial region of the subject (such as the head) is calculated, and the zoom magnification is controlled so that the calculated ratio remains constant in accordance with a zoom operation. [[Prior Art Literature]] [[Patent Literature]]

[0004] [[Patent Document 1]] Japanese Patent Application Laid-Open No. 2011-209620 [[Summary of the Invention]] [[Problem to be Solved by the Invention]]

[0005] In video production sites, there are cases where a cameraman is conscious of the composition within the angle of view and desires to control the imaging range so that the subject is positioned at a desired location within the imaging range. Here, the operation of controlling the imaging range so that the subject is positioned at a desired location within the imaging range while the user controls PTZ using a joystick has been complicated.

[0006] Therefore, the present invention aims to simplify the control of the imaging range so that the subject is located in a position desired by the user within the imaging range. [Means for solving the problem]

[0007] To solve the above problems, for example, the information processing apparatus of the present invention has the following configuration. That is, it has an acquisition means for acquiring an image captured by an imaging means, a display control means for displaying the image acquired by the acquisition means on a display means, and a control means for controlling the imaging range of the imaging means, wherein the display control means displays on the display means information of a first placement candidate in which the subject is placed at a first position in the imaging range, and information of a second placement candidate in which the subject is placed at a second position different from the first position in the imaging range, and the control means controls the imaging range based on the information of a placement candidate selected by the user from among the first and second placement candidates. [Effects of the Invention]

[0008] According to the present invention, it is possible to easily control the imaging range so that the subject is located in a position desired by the user within the imaging range. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example of the system. [Figure 2] This figure shows an example of the external appearance of an imaging device. [Figure 3] This diagram shows the functional block of the imaging device. [Figure 4] This diagram shows the functional blocks of an information processing device. [Figure 5] This flowchart shows the process for displaying placement options. [Figure 6] This figure shows an example of images and their possible placements that can be displayed on the screen. [Figure 7] This is a diagram to explain the displayed placement options. [Figure 8] This flowchart shows the process flow for controlling the imaging range. [Figure 9] This figure shows an example of images and their possible placements that can be displayed on the screen. [Figure 10] This diagram shows the hardware configuration of each device. [Modes for carrying out the invention]

[0010] Embodiments of the present invention will be described below with reference to the attached drawings. Note that the configurations shown in the following embodiments are merely examples and are not limited to those illustrated.

[0011] (Embodiment 1) Figure 1 shows the system configuration in this embodiment. The system in this embodiment includes an imaging device 100, an information processing device 200, a display 400, and a network 300.

[0012] The imaging device 100 and the information processing device 200 are interconnected via a network 300. The network 300 is implemented using multiple routers, switches, cables, etc., that comply with a communication standard such as ETHERNET®.

[0013] Network 300 may be implemented using the Internet, a wired LAN (Local Area Network), a wireless LAN (Wireless LAN), a WAN (Wide Area Network), or the like.

[0014] The image capturing apparatus 100 is an apparatus that captures an image, and functions as an image capturing unit capable of controlling an imaging range. The image capturing apparatus 100 transmits, via a network 300 to an external apparatus such as the information processing apparatus 200, image data of a captured image, information on an imaging date and time when the image was captured, identification information for identifying the image capturing apparatus 100, and information on an imaging range of the image capturing apparatus 110. The information processing apparatus 200 is, for example, a client apparatus such as a personal computer in which a program for realizing processing functions described later is installed. In the system according to the present embodiment, the number of the image capturing apparatus 100 is one, but a plurality of image capturing apparatuses may be provided. That is, a plurality of image capturing apparatuses 100 may be connected to the information processing apparatus 200 via the network 300. In this case, the information processing apparatus 200 determines, for example, by using the identification information associated with the transmitted image, which image capturing apparatus 100 among the plurality of image capturing apparatuses 100 captured the transmitted image.

[0015] The display 400 is configured of an LCD (Liquid Crystal Display) or the like, and displays images captured by the image capturing apparatus 100 and the like. The display 400 is connected to the information processing apparatus 200 via a display cable compliant with a communication standard such as HDMI (registered trademark) (High Definition Multimedia Interface). Note that the display 400 and the information processing apparatus 200 may be provided in a single housing.

[0016] Next, the imaging apparatus 100 according to the present embodiment will be described with reference to FIG. 2 and FIG. 3. FIG. 2 is an example of an external view of the imaging apparatus 100 according to the present embodiment. FIG. 3 is an example of functional blocks of the imaging apparatus 100 and the information processing apparatus 200 according to the present embodiment. Among the functional blocks of the imaging apparatus 100 shown in FIG. 3, each function of an image processing unit 112, a system control unit 113, a pan-tilt-zoom control unit 114, a storage unit 115, a communication unit 116, and the like is realized as follows. That is, it is realized when a CPU (Central Processing Unit) 1000 of the imaging apparatus 100 executes a computer program stored in a ROM (Read Only Memory) 1020 of the imaging apparatus 100 described later with reference to FIG. 10.

[0017] The direction in which the optical axis of the lens 101 faces is the imaging direction of the imaging apparatus 100, and the light flux that has passed through the lens 101 forms an image on the imaging element of the imaging unit 111 of the imaging apparatus 100. The lens driving unit 102 is configured by a driving system that drives the lens 101, and changes the focal length of the lens 101. The lens driving unit 102 is controlled by the pan-tilt-zoom control unit 114.

[0018] The pan driving unit 103 is configured by a mechanical driving system that performs a pan operation and a motor as a driving source, and is driven to control rotational driving for rotationally driving the imaging direction of the imaging apparatus 100 in the pan direction 105. The pan driving unit 103 is controlled by the pan-tilt-zoom control unit 114.

[0019] The tilt driving unit 104 is configured by a mechanical driving system that performs a tilt operation and a motor as a driving source, and is driven to control rotational driving for rotationally driving the imaging direction of the imaging apparatus 100 in the tilt direction 106. The tilt driving unit 104 is controlled by the pan-tilt-zoom control unit 114.

[0020] The imaging unit 111 is composed of an image sensor (not shown), such as a CCD (charge coupled device) sensor or a CMOS (complementary metal oxide semiconductor) sensor. The imaging unit 111 then converts the image of the subject formed through the lens 101 into an electrical signal using photoelectric conversion. The image processing unit 112 performs image processing, such as converting the electrical signal converted into a digital signal by the imaging unit 111 into a digital signal and compression encoding, to generate image data.

[0021] The pan-tilt-zoom control unit 114 controls the pan, tilt, and zoom of the imaging device 100 by controlling the pan drive unit 103, the tilt drive unit 104, and the lens drive unit 102 based on instructions transmitted from the system control unit 113.

[0022] The storage unit 115 stores (holds) information indicating the imaging range, for example. The communication unit 116 communicates with the information processing device 200 via the I / F 1040, which will be described later with reference to Figure 10. For example, the communication unit 116 transmits image data of the image captured by the imaging device 100 to the information processing device 200 via the network 300. The communication unit 116 also transmits information indicating the imaging range of the imaging device 100. Furthermore, the communication unit 116 receives control commands, which are commands for controlling the imaging device 100, transmitted from the information processing device 200, and transmits them to the system control unit 113.

[0023] The system control unit 113 controls the entire imaging device 100 according to the processing performed by the CPU 1000, as described later with reference to Figure 10, and performs the following processing, for example: The system control unit 113 analyzes the control commands for controlling the imaging device 100 transmitted from the information processing device 200 and performs processing according to the control commands. The system control unit 113 also instructs the pan-tilt-zoom control unit 114 to perform pan-tilt-zoom operations. Furthermore, when the system control unit 113 transmits the image data generated by the image processing unit 112 to the information processing device 200, it also adds information about the time the image data was captured to the image data.

[0024] In this embodiment, the imaging range is determined by the pan value, tilt value, and zoom value of the imaging device 100. The pan value is the angle of the imaging direction (optical axis) in the pan direction 105 of the imaging device 100, with one of the drive ends of the pan drive unit 103 set to 0°. The tilt value is the angle of the imaging direction (optical axis) in the tilt direction 106 of the imaging device 100, with one of the drive ends of the tilt drive unit 104 set to 0°. The zoom value of the imaging device 100 when an image is captured by the imaging device 100 is calculated from the focal length of the lens 101.

[0025] Here, the information processing of the information processing device 200 according to this embodiment will be described with reference to the functional blocks of the information processing device 200 shown in Figure 4. Each function of the information processing device 200 is implemented as follows, using the ROM 1020 and CPU 1000, which will be described later with reference to Figure 10. That is, each function shown in Figure 4 is implemented by the CPU 1000 of the information processing device 200 executing a computer program stored in the ROM 1020 of the information processing device 200.

[0026] The communication unit 201 acquires image data transmitted from the imaging device 100 via the I / F 1040, which will be described later with reference to Figure 10. The recording unit 201 can be implemented by a RAM (Random Access Memory) 1010 or an HDD (Hard Disk Drive) 1030, which will be described later with reference to Figure 10, and records information and data related to information processing by the information processing device 200. For example, the recording unit 201 records information regarding the position of a subject detected from the image. The recording unit 201 also records controllability information, which indicates which of the pan value, tilt value, and zoom value the imaging device 100 can control.

[0027] The detection unit 203 performs a detection process to detect subjects in the image acquired by the communication unit 201. In this embodiment, the detection unit 202 uses a matching pattern (dictionary) to perform processing such as pattern matching to detect a person as a subject in the image. In addition, when detecting a person from an image, multiple matching patterns may be used, such as a matching pattern for when the person is facing forward and a matching pattern for when the person is facing sideways. By performing a detection process using multiple matching patterns in this way, an improvement in detection accuracy can be expected.

[0028] Furthermore, matching patterns may be prepared for viewing a person from other angles, such as from an oblique direction or from above. Also, when detecting a person from an image, it is not always necessary to prepare matching patterns (dictionaries) that show the features of the entire body; matching patterns may be prepared for parts of the person, such as the upper body, lower body, head, face, or feet.

[0029] In this embodiment, a person is detected as the subject detected from the image, but it is not limited to a person; for example, it could be another object such as a car. Also, in this embodiment, the detection unit 203 uses pattern matching processing as a method for detecting a specific object from an image, but other prior arts may be used to detect a specific object from an image.

[0030] The identification unit 204 identifies subjects included in the image as a reference for controlling the imaging range of the imaging device 100. The imaging range of the imaging device 100 is controlled based on the subjects (people) in the image identified by the identification unit 204. A detailed explanation of the process by which the identification unit 204 identifies subjects included in the image as a reference for controlling the imaging range will be given later.

[0031] The operation reception unit 205 receives operations performed by the user via an input device (not shown), such as a keyboard or mouse. The display control unit 206 displays the image acquired by the communication unit 201 on the display 400. In this embodiment, the display control unit 206 also displays multiple placement candidates on the display 400, indicating the possible placement of the subject within the imaging range. A detailed explanation of the placement candidates displayed by the display control unit 206 will be given later.

[0032] The range control unit 207 executes a process to control the imaging range of the imaging device 100. Specifically, the range control unit 207 generates a control command to change at least one of the pan value, tilt value, and zoom value of the imaging device 100 and transmits it to the imaging device 100 via the communication unit 201. The imaging device 100 controls the imaging range according to the transmitted control command. For example, if the operation reception unit 205 receives a user instruction to change the pan value of the imaging device 100 by a predetermined value in the pan direction 105, the range control unit 207 executes the following process: That is, the range control unit 207 generates a control command to change the pan value by a predetermined value in the pan direction 105 and transmits it to the imaging device 100 via the communication unit 201. The imaging device 100 changes the pan angle by a predetermined angle in the pan direction 105 by controlling the pan drive unit 103 according to the acquired control command.

[0033] Next, with reference to Figures 5 to 8, we will explain the process of generating multiple placement candidates and displaying them on the display 400. Figure 5 is a flowchart showing the flow of the process of generating multiple placement candidates and displaying them on the display 400.

[0034] In step S501, the display control unit 206 determines whether there is a subject identified by the identification unit 204 as a criterion for controlling the imaging range. If it is determined that there is a person identified by the identification unit 204 (Yes in S501), the process proceeds to S502. If it is determined that there is no identified person (No in S501), the process in S501 is repeated. The process of identifying a subject (hereinafter referred to as the reference subject) as a criterion for controlling the imaging range by the identification unit 204 is described below. For example, the detection unit 203 performs a detection process to detect a subject in the image, and the display control unit 206 displays a frame indicating the position of the detected subject superimposed on the image on the display 400. When the operation reception unit 205 receives a user operation to select at least one of the frames displayed superimposed on the image, the identification unit 204 identifies the subject corresponding to the frame selected according to the user operation as the reference subject. For example, if the detection unit 203 detects four people as subjects from the captured image, the display control unit 206 superimposes a rectangular frame around each of the four detected people onto the image and displays it on the display 400. Then, when the user input is received to select a frame corresponding to a particular person from among the frames superimposed on each of the four people, the identification unit 204 identifies the person corresponding to the selected frame as the reference subject.

[0035] Next, in S502, the display control unit 206 determines whether the number of reference subjects identified in S501 is 1. If it is determined that the number of identified reference subjects is 1 (Yes in S502), the process proceeds to S503. If it is determined that the number of identified reference subjects is not 1 (No in S502), the process proceeds to S504. In S503, the display control unit 206 displays information on a single-person placement candidate on the display 400. Information on placement candidates will be described later with reference to Figure 6. In S504, it determines whether the number of identified reference subjects is 2. If it is determined that the number of identified reference subjects is 2 (Yes in S504), the process proceeds to S505. If it is determined that the number of identified reference subjects is not 2 (No in S504), the process proceeds to S506. In S505, the display control unit 206 displays information on a two-person placement candidate on the display 400. In S506, the display control unit 206 displays on the display 400 a list of possible seating arrangements in which all individuals fit within the field of view.

[0036] Here, referring to Figures 6(a) and 6(b), the information of the placement candidates displayed on the display 400 by the display control unit 206 will be described. In this embodiment, the display control unit 206 causes the display 400 to display a display window 601 which includes an image display area 602, which is the area captured and displayed by the imaging device 100, and a candidate display area 605, which is the area where multiple placement candidates are displayed. The display control unit 206 may also display the image display area 602 and the candidate display area 605 separately on the display 400 in different windows. Furthermore, as shown in Figure 6(a), an image including a person 604 identified as a reference subject by the identification unit 204 is displayed in the image display area 206. At this time, the display control unit 206 displays a frame 603 superimposed on the image corresponding to the position of the person 604. Furthermore, a candidate display area 605 is displayed that includes at least information on a first placement candidate 606 in which the subject is placed at a first position within the imaging range of the imaging device 100, and information on a second placement candidate 608 in which the subject is placed at a second position different from the first position within the imaging range. As shown in Figure 6(a), the first placement candidate 606 is displayed in the candidate display area 605 as a figure in which the icon 607 representing the subject is placed on the left side within the rectangular area representing the imaging range. The second placement candidate 608 is displayed in the candidate display area 605 as a figure in which the icon 609 representing the subject is placed in the center within the rectangular area representing the imaging range. The third placement candidate 610 is displayed in the candidate display area 605 as a figure in which the icon 611 representing the subject is placed on the right side within the rectangular area representing the imaging range. In the example shown in Figure 6(a), three different placement candidates are displayed, but the number of placement candidates may be displayed as well.

[0037] Here, we will explain the case where the first placement candidate 606 is selected from among the multiple placement candidates in Figure 6(a). When the first placement candidate 606 is selected by user operation, the range control unit 207 performs the following processing. That is, the range control unit 207 performs processing to control the imaging range of the imaging device 100 so that the positional relationship of the subject with respect to the imaging range indicated by the first placement candidate 606 and the positional relationship of the person 604, which is the reference subject, with respect to the current imaging range of the imaging device 100 are the same. Specifically, the range control unit 207 identifies the centroid point of the icon 607 that indicates the subject with respect to the rectangular area indicating the imaging range in the first placement candidate 606. Furthermore, the range control unit 207 identifies the centroid point of the person 604, which is the reference subject, with respect to the current imaging range of the imaging device 100. Then, the range control unit 207 generates a control command to control the imaging range so that the relative positional relationship of the centroid point of the icon 607 with respect to the rectangular area of ​​the first placement candidate 606 and the relative positional relationship of the centroid point of the person 604 with respect to the current imaging range are the same. The control command generated at this time is transmitted to the imaging device 100 via the communication unit 201, and the imaging device 100 controls the imaging range by controlling at least one of the pan value, tilt value, and zoom value based on the transmitted control command. After the imaging range of the imaging device 100 is controlled in this way, the captured image is transmitted from the imaging device 100 to the information processing device 200. Figure 6(b) shows an example in which the image captured and transmitted after the imaging range of the imaging device 100 is controlled according to the selection of a placement candidate is displayed in the image display area 602 by the display control unit 206. At this time, as shown in Figure 6(b), an image is obtained in which the positional relationship of the subject with respect to the imaging range shown in the first placement candidate 606 is the same as the positional relationship of the subject (person 604) with respect to the actual imaging range. Also, as shown in Figure 6(b), the display control unit 206 makes the display mode of the first placement candidate 606 different from the display mode of the second placement candidate 608 and the third placement candidate 610, depending on the selection of the first placement candidate 606. This allows the user to be informed that the first placement candidate 606 has been selected.

[0038] Furthermore, while the example shown in Figure 6 described the case of a single-person placement candidate, here we will explain the case of placement candidates for other numbers of people with reference to Figure 7. Figure 7(a) shows examples of multiple placement candidates for one person, and since placement candidates 701 to 703 are the same as placement candidates 606, 608, and 610 shown in Figure 6, respectively, their explanation will be omitted. Figure 7(b) shows examples of multiple placement candidates for two people. Placement candidates 704 to 706 shown in Figure 7(b) are the information displayed in the candidate display area during processing in S505. Placement candidate 704 is a placement candidate in which the icon 707 corresponding to the first person identified in the image is placed on the left side of the imaging range, and the icon 708 corresponding to the second person identified in the image is placed on the right side of the imaging range. Placement candidate 705 is a placement candidate in which the icon 707 corresponding to the first person is placed so that it is zoomed in on the left side. Placement candidate 706 is a placement candidate in which the icon 708 corresponding to the second person is placed so that it is zoomed in on the right side. Furthermore, the arrangement candidate 709 shown in Figure 7(c) is an arrangement candidate that is displayed in the candidate display area by the display control unit 206 in S506 when it is determined in S504 of Figure 5 that there are not two reference subjects. At this time, it is an arrangement candidate in which the N people (N is an integer) identified by the identification unit 207 fit within the field of view. In this embodiment, when arrangement candidate 709 is selected, the range control unit 207 generates a control command to control the imaging range of the imaging device 100 so that the identified N people fit within the field of view and transmits it to the imaging device 100. Note that the arrangement candidates to be displayed are not limited to those in Figures 7(a) to (c), and other arrangement candidates may be added.

[0039] Here, referring to the flow shown in Figure 8, we will explain the process of controlling the imaging range associated with the user's selection of placement candidates. The process in the flow shown in Figure 8 is to be started or stopped, for example, according to instructions from the user. The process in the flowchart shown in Figure 10 is executed by the functional blocks of the information processing device 200 shown in Figure 4, which are realized by the CPU 1000 of the information processing device 200 executing a computer program stored in the ROM 1020 of the information processing device 200.

[0040] First, in S801, the range control unit 207 determines whether any of the placement candidates displayed in the candidate display area 605 have been selected. If it is determined that a placement candidate has been selected (Yes in S801), the process proceeds to S802. If it is determined that no placement candidate has been selected (No in S801), the process returns to S801. In S802, the range control unit 207 determines the amount by which to control the pan and tilt values ​​for controlling the imaging range, based on the information of the subject's position relative to the imaging range indicated by the selected placement candidate. Here, we will explain the case where the first placement candidate 606 is selected in the example shown in Figure 6(a). In this case, in S802, the range control unit 207 performs the following process. That is, it determines the pan and tilt values ​​for controlling the imaging range so that the relative positional relationship of the centroid point of the icon 607 with respect to the rectangular area of ​​the first placement candidate 606 and the relative positional relationship of the centroid point of the person 604 with respect to the actual imaging range are the same. Next, in S803, the range control unit 207 determines the zoom value for controlling the imaging range based on information about the size of the subject relative to the imaging range indicated by the selected placement candidate. Here, we assume that the first placement candidate 606 is selected. In this case, the range control unit 207 determines the zoom value of the imaging device 100 such that the relative size of the icon 606 to the rectangular area of ​​the first placement candidate 606 is the same as the relative size of the person 604 to the actual imaging range. In S804, the range control unit 207 generates a control command to control the imaging device 100 so that the pan value, tilt value, and zoom value determined in S802 to S803 are the same. In S805, the communication unit 201 transmits the control command generated in S804 to the imaging device 100. At this time, the imaging device 100 analyzes the transmitted control command and controls the range of the imaging device 100 by controlling at least one of the pan drive unit 103, tilt drive unit 104, and lens drive unit 102 based on the results of the analysis.

[0041] As described above, the information processing device 200 in this embodiment controls the imaging range of the imaging device 100 so that an image with a composition similar to that of the selected arrangement candidate is captured, according to the arrangement candidate selected from among a plurality of different arrangement candidates that are displayed. In this way, it is possible to easily control the imaging range so that the subject is located in the location desired by the user within the imaging range.

[0042] (Embodiment 2) A second embodiment of the present invention will now be described. In Embodiment 2, the display mode of an icon displayed in the placement candidate in the candidate display area is associated with the display mode of a frame indicating a reference subject corresponding to the icon in the image display area. In this embodiment, the parts that differ from Embodiment 1 will be mainly described, and the same or equivalent components and processes as in Embodiment 1 will be denoted by the same reference numerals, and redundant explanations will be omitted.

[0043] Referring to Figures 9(a) and 9(b), the information that the display control unit 206 instructs the display 400 to display will be described. The display control unit 206 causes the display 400 to display a display window 901 which includes an image display area 902, which is the area captured by the imaging device 100 and displayed, and a candidate display area 907, which is the area where multiple placement candidates are displayed. The display control unit 206 may also display the image display area 902 and the candidate display area 905 separately on the display 400 in different windows. Furthermore, as shown in Figure 9(a), an image including persons 904 and 906, which have been identified as reference subjects by the identification unit 204, is displayed in the image display area 206. At this time, the display control unit 206 overlays a frame 903 corresponding to the position of person 904 and a frame 905 corresponding to the position of person 906 onto the image. The display control unit 206 also displays a candidate display area 907, which includes placement candidates 908, 911, and 913, as placement candidates for two people. In the example shown in Figure 9(a), placement candidate 908 is a placement candidate in which the icon 909a corresponding to person 904 identified in the image is placed on the left side of the imaging range, and the icon 910 corresponding to person 906 identified in the image is placed on the right side of the imaging range. Placement candidate 911 is a placement candidate in which the icon 909a corresponding to person 904 is placed so as to be zoomed in on the left side. Placement candidate 913 is a placement candidate in which the icon 914a corresponding to person 906 is placed so as to be zoomed in on the right side. In this embodiment, as shown in Figure 9(a), the display control unit 206 displays the frame indicating the person identified as the reference subject and the icon corresponding to that person in the placement candidates in a display manner that clearly indicates the association between the frame and the icon to which that person is placed. Specifically, in the example shown in Figure 9(a), the frame 903 representing person 904 and the outer edges of icons 909a and 912a corresponding to person 904 are shown with solid lines. Also, in the example shown in Figure 9(a), the frame 905 representing person 906 and the outer edges of icons 910a and 914a corresponding to person 906 are shown with dotted lines.Furthermore, the method of clearly indicating the association between a frame representing a person and the icon corresponding to that person is not limited to using solid or dotted lines. For example, the display control unit 206 may display as follows: The display control unit 206 displays the frame 903 representing person 904 and the icons 909a and 912a corresponding to person 904 in a first color. In addition, the display control unit 206 displays the frame 905 representing person 906 and the icons 910a and 914a corresponding to person 906 in a second color different from the first color. Moreover, as a method of clearly indicating the association with the icon corresponding to that person, the display control unit 206 may use cropped images of the person. Specifically, as shown in Figure 9(b), the display control unit 206 may use cropped images 909b and 912b, which are cropped images of person 904, instead of icons 909a and 912a. In this case, the display control unit 206 may also use cropped images 910b and 914b, which are cropped images of the person 906, instead of icons 910a and 914a.

[0044] As described above, the information processing device 200 in this embodiment displays a representation that clearly shows the association between a figure superimposed on an image corresponding to a person identified from the image and a figure displayed in the placement candidates corresponding to that person. In this way, it is possible to easily control the imaging range so that the subject is located in the location desired by the user within the imaging range.

[0045] (Other embodiments) Next, with reference to Figure 10, the hardware configuration of the information processing device 200 for realizing each function of each embodiment will be described. In the following description, the hardware configuration of the information processing device 200 will be explained, but the imaging device 100 will also be realized with a similar hardware configuration.

[0046] The information processing device 200 in this embodiment includes a CPU 1000, RAM 1010, ROM 1020, HDD 1030, and I / F 1040.

[0047] The CPU 1500 is a central processing unit that provides overall control over the information processing unit 200. The RAM 1010 temporarily stores computer programs executed by the CPU 1000. The RAM 1010 also provides a work area used by the CPU 1000 when executing processes. Furthermore, the RAM 1010 can function, for example, as frame memory or buffer memory.

[0048] ROM1020 stores programs and other information that the CPU1500 uses to control the information processing unit 200. HDD1030 is a storage device that records image data and other data.

[0049] I / F1010 communicates with external devices via network 140, following protocols such as TCP / IP and HTTP.

[0050] In the above-described embodiments, examples are given in which the CPU 1000 performs processing. However, at least a portion of the CPU 1000's processing may be performed by dedicated hardware. For example, the process of displaying a GUI (Graphical User Interface) or image data on the display 400 may be performed by a GPU (Graphics Processing Unit). Also, the process of reading program code from the ROM 1020 and loading it into the RAM 1010 may be performed by a DMA (Direct Memory Access) functioning as a data transfer device.

[0051] Furthermore, the present invention can also be realized by a process in which one or more processors read and execute a program that implements one or more of the functions of the above-described embodiments. The program may be supplied to a system or device having a processor via a network or storage medium. The present invention can also be realized by a circuit (e.g., an ASIC) that implements one or more of the functions of the above-described embodiments. In addition, each part of the information processing device 200 may be realized by the hardware shown in Figure 10, or by software.

[0052] Furthermore, one or more functions of the information processing device 200 according to each embodiment described above may be provided by other devices. For example, one or more functions of the information processing device 200 according to each embodiment may be provided by the information processing device 200 itself. Furthermore, the embodiments described above may be combined, for example, by arbitrarily combining the embodiments described above.

[0053] Although the present invention has been described above along with its embodiments, these embodiments are merely examples of how the present invention can be implemented, and the technical scope of the present invention should not be limited by them. In other words, the present invention can be implemented in various forms without departing from its technical concept or its main features. For example, combinations of the embodiments are also included in the disclosure of this specification. [Explanation of Symbols]

[0054] 100 Imaging device 200 Information Processing Devices 201 Communications Department 202 Storage section 203 Detection Unit 204 Specific section 205 Operation Reception Section 206 Display Control Unit 207 Range Control Unit

Claims

1. A computer, The image captured by the imaging means capable of pan, tilt, and zoom drive, and a plurality of arrangement candidates, including a first arrangement candidate in which a first icon representing the subject is placed at a first position within a first rectangular region, and a second arrangement candidate in which a second icon representing the subject is placed at a second position within a second rectangular region different from the first rectangular region, are displayed on the display means. Based on the information of the placement candidate selected by the user from among the plurality of placement candidates, and at least one of the position and size of the subject detected in the image, the imaging range of the imaging means is controlled. A program characterized by the following features.

2. The program according to claim 1, characterized in that the image, the first arrangement candidate, and the second arrangement candidate are displayed simultaneously.

3. The subject is a person, The program according to claim 1 or 2, characterized in that the first icon and the second icon are icons representing a person.

4. The program according to any one of claims 1 to 3, characterized in that the plurality of arrangement candidates include arrangement candidates in which an icon representing a first subject and an icon representing a second subject are arranged in different positions within a single rectangular area.

5. The program according to any one of claims 1 to 4, characterized in that the frame indicating the subject detected in the image is displayed superimposed on the image.

6. The program according to any one of claims 1 to 5, characterized in that the imaging range of the imaging means is controlled based on information of a placement candidate selected by the user from among the plurality of candidates, and at least one of the position and size of a subject selected by the user from among the subjects detected in the image.

7. The information of the first arrangement candidate includes information on the relative position and relative size of the first icon with respect to the first rectangular area, The program according to any one of claims 1 to 6, characterized in that the information of the second placement candidate includes information on the relative position of the second icon with respect to the second rectangular area and information on the relative size of the second icon with respect to the second rectangular area.

8. The program according to claim 7, characterized in that the relative position of the first icon with respect to the first rectangular area is different from the relative position of the second icon with respect to the second rectangular area.

9. A display control means for displaying an image captured by an imaging means capable of pan, tilt, and zoom drive, and a plurality of arrangement candidates, including a first arrangement candidate in which a first icon indicating a subject is placed at a first position within a first rectangular region, and a second arrangement candidate in which a second icon indicating a subject is placed at a second position within a second rectangular region different from the first rectangular region, A control means for controlling the imaging range of the imaging means based on information of the placement candidate selected by the user from among the plurality of placement candidates, and at least one of the position and size of the subject detected in the image, An information processing device characterized by having the following features.

10. The information processing apparatus according to claim 9, characterized in that the image, the first arrangement candidate, and the second arrangement candidate are displayed simultaneously.

11. The subject is a person, The information processing apparatus according to claim 9 or 10, characterized in that the first icon and the second icon are icons representing a person.

12. The information processing apparatus according to any one of claims 9 to 11, characterized in that the plurality of arrangement candidates include arrangement candidates in which an icon representing a first subject and an icon representing a second subject are arranged in different positions within a single rectangular area.

13. The information processing apparatus according to any one of claims 9 to 12, characterized in that the frame indicating the subject detected in the image is displayed superimposed on the image.

14. The information processing apparatus according to any one of claims 9 to 13, characterized in that the imaging range of the imaging means is controlled based on information of a placement candidate selected by the user from among the plurality of candidates and at least one of the position and size of a subject selected by the user from among the subjects detected in the image.

15. The information of the first arrangement candidate includes information on the relative position and relative size of the first icon with respect to the first rectangular area, The information processing apparatus according to any one of claims 9 to 14, characterized in that the information of the second arrangement candidate includes information on the relative position of the second icon with respect to the second rectangular area and information on the relative size of the second icon with respect to the second rectangular area.

16. The information processing apparatus according to claim 15, characterized in that the relative position of the first icon with respect to the first rectangular area is different from the relative position of the second icon with respect to the second rectangular area.

17. Display means to display an image captured by an imaging means capable of pan, tilt and zoom drive, and a plurality of arrangement candidates, including a first arrangement candidate in which a first icon representing a subject is placed at a first position within a first rectangular region, and a second arrangement candidate in which a second icon representing a subject is placed at a second position within a second rectangular region different from the first rectangular region. Based on the information of the placement candidate selected by the user from among the plurality of placement candidates, and at least one of the position and size of the subject detected in the image, the imaging range of the imaging means is controlled. An information processing method characterized by the following:

Images