Information processing device, method for controlling information processing device, program, and imaging system

Abstract

To maintain an angle of view or the like intended by a user in each crop region when control of zooming or the like is performed on an imaging device.SOLUTION: In an imaging system, an information processing device (controller) 200 has a display control part for displaying a setting screen which can set an angle-of-view control mode for controlling setting of a plurality of crop regions and angles of view in the plurality of crop regions, with respect to a first image, outputting, as a second image, an image being set with the crop regions, and changing sizes and positions of the crop regions with respect to the image in accordance with the setting of the angle-of-view control mode. When a zoom magnification is changed, the display control part changes the size and the position with respect to the image before changing a zoom magnification of the crop regions with the angle of view control mode being set in accordance with the zoom magnification without changing, in accordance with the zoom magnification, the size and the position with respect to the image prior to the change of the zoom magnification of a crop region with no angle-of-view control mode being set among the plurality of crop regions.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, a control method for the information processing device, a program, and an imaging system. [Background technology]

[0002] In recent years, the number of video editing and distribution systems that use IP (Internet Protocol) networks such as the Internet has been increasing in video production. In a typical system, video data such as video and audio output from a camera connected to an IP network is received by a video editing device such as a switcher, and edited as necessary into broadcast video before being distributed / output.

[0003] In recent years, it has become possible for a PC or controller to remotely control the image quality settings and pan, tilt, and zoom of a camera via an IP network. Furthermore, there has been an increase in cameras that capture video via IP networks rather than traditional video cables such as SDI. This has led to an increase in cameras that can shoot without a cameraman on location (remote control cameras).

[0004] Furthermore, some cameras have the function of cropping a portion of the image data from the entire field of view captured and outputting the image. If no special processing is performed, the field of view of the cropped area will change when the pan, tilt, or zoom of the camera is controlled. On the other hand, by appropriately adjusting the position or size of the cropped area, it is possible to maintain the field of view of the cropped area even if the pan position, tilt position, or optical zoom magnification changes.

[0005] Patent Document 1 discloses a method of switching to an angle of view lock mode by an angle of view lock operation, and adjusting the electronic zoom magnification at the same time as increasing or decreasing the optical zoom magnification, thereby maintaining the angle of view of a specified area from an image to be captured at the angle of view immediately before the angle of view lock operation. In addition, in order to prevent the image quality of a specified area from being deteriorated due to a high digital zoom magnification, a method of reducing image quality deterioration by adding pixel values ​​of a predetermined number of pixels in the vertical and horizontal directions of the specified area is also disclosed. Here, the cause of image quality deterioration due to a high digital zoom magnification is that the number of pixels in the specified area is insufficient compared to the number of pixels of the resolution when the image is output, so that the image of the specified area is stretched when the image is output.

[0006] In addition, two or more areas may be set to be cropped from the captured image. Patent Document 2 discloses a method for obtaining partial images in which the imaging conditions of each area are appropriately controlled by controlling the imaging conditions of an imaging device that cuts out a plurality of specified areas from a captured image as partial images at predetermined time intervals. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2008-022386 A [Patent Document 2] JP 2016-219905 A Summary of the Invention [Problem to be solved by the invention]

[0008] When there are two or more crop areas cropped from a captured image, when a user controls the imaging device to pan, tilt, or zoom, it may not be possible to maintain the angle of view and image quality of each crop area as intended by the user.

[0009] For example, when crop area 1 and crop area 2, which are areas of a captured image, are set, performing pan, tilt, or zoom control on the camera to change the angle of view of crop area 1 also changes the angle of view of crop area 2. Also, assume that a mode that can maintain the angle of view for the crop areas is enabled using the method disclosed in Patent Document 1. In this case, if the optical zoom magnification is reduced to widen the angle of view of crop area 1, the image quality of crop area 2 may deteriorate if the number of pixels in crop area 2 becomes less than the number of pixels in the output resolution.

[0010] Therefore, an object of the present invention is to provide an information processing device that can maintain the angle of view, etc., intended by the user in each crop region when control such as zooming is performed on the imaging device. [Means for solving the problem]

[0011] In order to achieve the above-mentioned object, an information processing device as one aspect of the present invention comprises: a control means for displaying on a display device a setting screen on which a setting screen can be set for a first image captured by an imaging means in an imaging device, and a setting screen for setting a plurality of crop areas and a setting screen for setting an angle of view control mode for controlling an angle of view in the plurality of crop areas; and an output means for outputting the image in which the crop areas are set as a second image, and further comprises a change means for changing a size and a position of the crop areas relative to the image in accordance with the setting of the angle of view control mode, wherein when a zoom magnification of the imaging device is changed, the change means does not change a size and a position of a crop area among the plurality of crop areas for which the angle of view control mode is not set, relative to the image before the zoom magnification is changed, in accordance with the zoom magnification, and changes a size and a position of a crop area for which the angle of view control mode is set, relative to the image before the zoom magnification is changed, in accordance with the zoom magnification, so as to maintain the angle of view, in accordance with the zoom magnification. Effect of the Invention

[0012] According to the present invention, it is possible to provide an information processing device capable of maintaining the angle of view, etc., intended by the user in each crop region when control such as zooming is performed on the imaging device. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a configuration diagram illustrating an example of an imaging system according to an embodiment. [Diagram 2] FIG. 1 is a system configuration diagram including a camera and a controller according to an embodiment. [Diagram 3] FIG. 2 is a schematic diagram of the software configuration of a camera and a controller according to the embodiment. [Figure 4] FIG. 13 is a layout setting diagram for displaying a crop region according to the embodiment. [Diagram 5] 5 is a flowchart of a process for changing the zoom magnification of the camera according to the embodiment. [Figure 6] 11 is a flowchart of a process for setting each crop region according to the embodiment. [Figure 7] 11A to 11C are diagrams illustrating an example of display of each crop region in an image when the zoom magnification of the camera according to the embodiment is changed. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, the embodiments for carrying out the present invention will be described in detail with reference to the attached drawings. Note that the embodiments described below are examples of means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions, and the present invention is not limited to the following embodiments.

[0015] <Embodiment 1>

[0016] Fig. 1 is a configuration diagram showing an example of an imaging system in this embodiment. The imaging system in this embodiment includes a camera (imaging device) 100 and a controller (information processing device) 200, and outputs an image in which a crop area is set by cropping a part of an image captured by the imaging device. Hereinafter, with reference to Fig. 1, the configuration and function of each part of the camera 100 and the controller 200 constituting the imaging system in this embodiment will be described in detail.

[0017] The camera 100 in this embodiment is configured to include a CPU 110, a RAM 120, a ROM 130, an image capture I / F 140, a network I / F 150, an image sensor 160, a drive control unit 170, an image processing unit 180, and a communication control unit 190. The CPU 110, the RAM 120, the ROM 130, the image capture I / F 140, the network I / F 150, the drive control unit 170, the image processing unit 180, and the communication control unit 190 are connected to each other via an internal bus 1000.

[0018] The CPU 110 is a central processing unit, and controls the entire camera 100 by executing control programs stored in the ROM 130 and the RAM 120. In other words, it comprehensively controls each component in the camera 100. CPU is an acronym for Central Processing Unit.

[0019] The RAM 120 is a high-speed storage device such as a DRAM, into which the OS, various programs, and various data are loaded, and is also used as a working area for the OS and various programs. The ROM 130 is a non-volatile storage device such as a flash memory, HDD, SSD, SD card, etc., and is used as a permanent storage area for the OS, various programs, and various data, as well as a short-term storage area for various data.

[0020] An image sensor (imaging element) 160, which is made of a CCD or CMOS and also functions as an imaging means, is connected to the image capture I / F 140. The image capture I / F 140 converts image data acquired from the image sensor 160 into a predetermined format, compresses it as necessary, and transfers it to the RAM 120. The image capture I / F 140 also performs image quality adjustments such as zoom and focus and exposure control when acquiring data from the image sensor 160, and cropping to cut out only a predetermined area of ​​the image data.

[0021] The drive control unit 170 controls the pan, tilt, and zoom of the camera 100. That is, the drive control unit 170 controls the pan rotation, tilt rotation, and zoom magnification of the camera 100 by controlling each drive mechanism (drive means) not shown. These are performed according to instructions from an external communication device (information processing device) such as the controller 200 via the network I / F 150.

[0022] The image processing unit 180 controls the image capture I / F 140, and generates data obtained from the image sensor 160 as image data in response to an acquisition request for an image or video from an external communication device or instructions from other functional units. In addition, various image processes are performed on the generated image data in response to instructions from the external communication device or other functional units. The various image processes include changing brightness, color, size, cropping a predetermined area (cropping process), and the like. Furthermore, the various image processes include conversion to image data such as JPEG or video data such as H.264. The image data or video data that has been subjected to the image processing, and information on the results of the image processing are passed to the communication control unit 190.

[0023] The communication control unit 190 controls the network I / F 150 and communicates with an external communication device such as the controller 200 through the IP network 300. An example of the content of the communication is receiving an acquisition request for an image (first image) or video captured by the image sensor 160, and an image processing request for the image, which is passed to the image processing unit 180. In addition, the communication control unit 190 transmits to the external communication device (controller 200) the distribution of image data or video data received from the image processing unit 180, and result information of various image processing information as a response to the image processing request or an update notification of camera information. Here, the request may use a general communication protocol such as HTTP (Hypertext Transfer Protocol).

[0024] Network I / F 150 is an I / F for connecting to the above-mentioned IP network 300, and is responsible for communication with an external communication device such as controller 200 via a communication medium such as Ethernet. Here, communication refers to sending and receiving camera control commands from controller 200. An image (first image) or video captured by camera 100 is also delivered (transmitted) to an external communication device such as controller 200 via network I / F 150.

[0025] For simplicity of explanation, it has been described that remote camera control of camera 100 is performed via network I / F 150, but it may have another I / F such as a serial communication I / F (not shown). For simplicity of explanation, it has been described that images or videos captured by camera 100 are distributed via network I / F 150, but it may be distributed via a video output I / F such as HDMI (registered trademark) or SDI (not shown).

[0026] Next, the configuration of the controller 200 in this embodiment will be described. The controller 200 is configured to have a CPU 210, a RAM 220, a ROM 230, a user input / output I / F 240, and a network I / F 250. Furthermore, the controller 200 is configured to have an input control unit 260, a display control unit 270, and a communication control unit 280. Moreover, the CPU 210, the RAM 220, the ROM 230, the user input / output I / F 240, the network I / F 250, the input control unit 260, the display control unit 270, and the communication control unit 280 are connected to each other via an internal bus 2000.

[0027] The CPU 210 is a central processing unit, and controls the entire controller 200 by executing control programs stored in the ROM 230 and the RAM 220. In other words, the CPU 210 functions as a control unit that comprehensively controls each component of the controller 200.

[0028] The RAM 220 is a high-speed storage device such as a DRAM, into which the OS, various programs, and various data are loaded, and is also used as a working area for the OS and various programs. The ROM 230 is a non-volatile storage device such as a flash memory, HDD, SSD, SD card, etc., and is used as a permanent storage area for the OS, various programs, and various data, as well as a short-term storage area for various data.

[0029] The user input / output I / F 240 is an I / F (interface) for receiving operations from the user to the controller 200 and outputting images and videos captured by the camera 100 to the user. Examples of the user input / output I / F 240 include a mouse, a keyboard, a button, a dial, a joystick, a display, a touch panel, and the like. Here, the touch panel and the display may be configured as an integrated type such as a touch panel-mounted display that can detect the touch position. In this embodiment, the user can control each part of the camera 100 (camera control) that is communicatively connected via these I / Fs. It is also possible to display camera images, i.e., images and videos distributed from the camera 100, on the display.

[0030] The input control unit 260 controls the user input / output I / F 240, accepts user input operations from a mouse, keyboard, button, dial, joystick, touch panel, etc., and generates a corresponding camera control command (control signal) from the accepted operation information. Thereafter, the generated camera control command is passed to the communication control unit 280. In addition, as necessary, the input control unit 260 generates user setting information according to the operation information and passes it to the display control unit 270.

[0031] The display control unit 270 controls the user input / output I / F 240 in response to the image, video, and camera information received from the communication control unit 280. In this embodiment, the display control unit 270 outputs the image, video, camera status information, and the like received from the camera 100 to a display device such as a touch panel or a display, and performs OSD display (on-screen display) on the display device. In this embodiment, the display control unit 270 also functions as an output means for outputting an image (first image) captured by the camera 100, an image (second image) in which a crop region is set for the first image, and the like. User setting information received from the input control unit 260 is also similarly displayed on the OSD.

[0032] The communication control unit 280 receives various commands from the input control unit 260, and controls the network I / F 250 to perform communication processing such as sending requests and receiving responses to external communication devices such as the camera 100. Responses from the external communication devices are then sent to the display control unit 270. An example of the content of communication is the sending of a request to the camera 100 to acquire captured images or videos, a camera control request such as image processing, and the receiving of responses to these requests. Even when the camera is controlled by a device other than the controller, the camera 100 sends a camera information update notification, which is then received and passed to the display control unit 270.

[0033] Similarly, the above-mentioned crop region setting command is also sent to the camera 100. The request may use a general communication protocol such as HTTP (HyperText Transfer Protocol).

[0034] The network I / F 250 is an I / F for connecting to the aforementioned IP network 300, and is responsible for communication with external communication devices such as the camera 100 via a communication medium such as Ethernet (registered trademark). Here, communication refers to sending and receiving control commands to the camera 100, and receiving image data and video data from the camera 100.

[0035] For simplicity, the controller 200 controls the camera 100 via the network I / F 250, but it may have another I / F, such as a serial communication I / F (not shown). For simplicity, the controller 200 receives images and videos from the camera 100 via the network I / F 250, but it may receive images and videos via a video input I / F, such as HDMI or SDI (not shown).

[0036] 2 is a configuration diagram of an imaging system including a camera 100 and a controller 200 in this embodiment. The camera 100 and the controller 200 are connected to each other via an IP network 300. As a basic operation, the camera 100 distributes captured images and videos to an external communication device such as the controller 200 via the IP network 300.

[0037] In this embodiment, camera 100 is a PTZ camera (pan-tilt-zoom camera) capable of panning, tilting, and zooming. Controller 200 transmits a predetermined control signal to camera 100 (accesses camera 100) via IP network 300 to acquire images and videos captured by camera 100 and to perform various controls such as driving operations of camera 100. Note that, although only one camera is shown as an example in FIG. 2, it is also possible to control multiple cameras (multiple imaging devices) with one controller (one information processing device).

[0038] FIG. 3 is a schematic diagram of the software configuration of each of the camera 100 and the controller 200. FIG. 3(A) is a schematic diagram of the software configuration of the camera 100. FIG. 3(B) is a schematic diagram of the software configuration of the controller 200. The camera 100 has an OS 401 which is a basic program for controlling the entire camera 100. The OS 401 includes an image processing program 402 which is a program for the CPU 110 to execute processing in the image processing unit 180. The OS 401 also includes a communication control program 403 which is a program for the CPU 110 to execute processing in the communication control unit 190. Here, the image processing program 402 and the communication control program 403 are loaded onto the RAM 120, and the CPU 110 executes each program. Note that the CPU 210 may execute one or more of the image processing program 402 and the communication control program 403 loaded onto the RAM 120.

[0039] The controller 200 has an OS 411 which is a basic program for controlling the entire controller 200. The OS 401 includes an input control program 412 which is a program for the CPU 210 to execute processing in the input control unit 260. The OS 401 also includes a display control program 413 which is a program for the CPU 210 to execute processing in the display control unit 270. The OS 401 also includes a communication control program 414 which is a program for the CPU 210 to execute processing in the communication control unit 280. The input control program 412, the display control program 413, and the communication control program 414 are loaded onto the RAM 220, and the CPU 210 executes each program. Note that the CPU 110 may execute one or more of the input control program 412, the display control program 413, and the communication control program 414 loaded onto the RAM 220.

[0040] Fig. 4 is a diagram illustrating a layout setting screen (layout setting UI) when displaying crop areas according to the embodiment. Two crop areas, Crop1 (crop area 1) and Crop2 (crop area 2), are displayed on the layout setting screen shown in Fig. 4. Hereinafter, crop area setting information that is held as user setting information when a user sets a crop area via the input / output I / F 240 of the controller 200 will be described with reference to Fig. 4.

[0041] When the user starts setting a crop region via input / output I / F 240 of controller 200, input control unit 260 requests display control unit 270 to display an OSD for setting the crop region. Upon receiving the instruction to display the OSD for setting the crop region, display control unit 270 displays a layout setting screen on the screen of the display device as a user interface (UI) for displaying the crop region as shown in Fig. 4. The layout setting screen is a setting screen that allows setting of multiple crop regions for an image (first image) captured by camera 100 and setting a view angle maintenance mode that enables control to maintain the view angle in the multiple crop regions.

[0042] The layout setting screen is a screen where the user can select the image or video output from the camera 100, the resolution of the image or video in which the crop area is set, the number and arrangement of the crop areas, and the setting of the angle of view maintenance mode (angle of view control mode) for the crop area. The angle of view maintenance mode may be set, for example, via a UI such as a button for selecting the setting (enabling) and canceling (disabling) of the angle of view maintenance mode, which is arranged on the layout setting screen. Also, for example, the set crop area may be set or canceled by operating via the input / output I / F 240.

[0043] Also, the user can move, enlarge, and reduce the crop areas (rectangular areas) shown by dashed lines in FIG. 4, which are set via the input / output I / F 240, on the layout setting screen. Note that FIG. 4 shows a state in which the angle of view maintenance mode is set in the Crop2 area, and a state in which the angle of view maintenance mode is not set in the Crop1 area. Then, on the right side of the layout setting screen as seen from the front, the setting state (presence or absence of setting status) of the angle of view maintenance mode in each crop area is displayed so that the user can visually confirm it. In this way, the display control unit 270 can display on the layout setting screen a crop area (first crop area) in which the angle of view maintenance mode is set for the display image (first image) and a crop area (second crop area) in which the angle of view maintenance mode is not set.

[0044] The setting state of the angle of view maintenance mode is shown by a lock icon in FIG. 4, but may be an icon of another shape. Also, the icon may be highlighted by adding color or blinking depending on the setting state. Also, information on the width w and height h of each crop area, and coordinate information of each crop area are also displayed on the layout setting screen. By displaying in this manner, the user can visually confirm the state of the angle of view maintenance mode for each crop area and the width w, height, and coordinate information for each crop area on one screen, and therefore can easily check each piece of information in the crop area.

[0045] The crop area setting information includes information on the coordinates, width w, and height h of each crop area in an image or video acquired from camera 100, and information on the setting state of the angle of view maintenance mode for each crop area, as shown in Fig. 4. In addition, although not shown in Fig. 4, the crop area setting information also includes the spherical coordinates of four points of the crop area angle calculated from camera information such as pan / tilt / zoom position information and image inversion state. The spherical coordinates are calculated at the timing when the angle of view maintenance mode is set, and are set as the crop area setting information.

[0046] The display control unit 270 generates a cropping region setting command in accordance with the cropping region setting information, and passes it to the communication control unit 280. The communication control unit 280 then transmits the cropping region setting command to the camera 100 as a request.

[0047] Hereinafter, a processing flow for each crop area when the user changes the zoom magnification of camera 100 will be described with reference to Figs. 5, 6, and 7. Fig. 5 is a flowchart showing an example of processing when the zoom magnification of camera 100 in this embodiment is changed. Fig. 6 is a flowchart of setting processing for each crop area. Fig. 7 is a diagram showing an example of display of each crop area in an image when the zoom magnification of camera 100 according to the embodiment is changed.

[0048] 7 are images (second images) equivalent to the OSD display of crop region setting information on an image or video captured by camera 100 by display control unit 270 via input / output I / F 240. Image 701 corresponds to the OSD display when the zoom magnification of camera 100 is increased. Image 702 corresponds to the OSD display when the zoom magnification of camera 100 is decreased.

[0049] Also, crop areas 703 and 705 of Crop1 shown in Fig. 7 are crop areas set with a crop number of 1. Crop areas 703 and 705 indicate crop areas in a state where the angle of view maintaining mode is not set (not enabled). Crop areas 704 and 706 of Crop2 are crop areas set with a crop number of 2. Crop areas 703 and 705 indicate crop areas in a state where the angle of view maintaining mode is set (enabled). Details of each crop area shown in Fig. 7 will be described later.

[0050] 5 and 6 are realized by the CPU 210 of the controller 200 executing each program loaded on the RAM 220. Each process (step) is represented by adding an S to the beginning to omit the description of the process (step). It is assumed that, at the time of carrying out the following process, the user has set one or more crop areas via a layout setting screen as shown in FIG.

[0051] First, in S501, the CPU 210 acquires crop area setting information set in the camera 100 when the zoom magnification change process is started. Then, the process proceeds to S502. That is, in S501, the CPU 210 acquires crop area setting information for each crop area set by a user or the like.

[0052] Next, in S502, the CPU 210 performs a process of setting each crop region acquired in S501. After that, the zoom magnification change process ends. Here, the details of the flow of the process of setting each crop region in S502 will be described with reference to FIG.

[0053] First, in S601, the CPU 210 obtains the number N of crop regions set in the camera 100, and proceeds to S602. Next, in S602, the CPU 210 initializes a number i for identifying each crop region to 1, and proceeds to S603.

[0054] Next, in S603, CPU 210 determines whether crop region number i is equal to or less than the number of stored crop regions N. If the result of the determination is that crop region number i is equal to or less than the number of stored crop regions N, the process proceeds to S604. On the other hand, if crop region number i is not equal to or less than the number of stored crop regions N, the process of setting each crop region ends.

[0055] Next, in S604, the CPU 210 determines whether the view angle maintaining mode is set for the crop region with the number i. If the result of the determination is that the view angle maintaining mode is set for the crop region with the number i, the process proceeds to S606. On the other hand, if the view angle maintaining mode is not set for the crop region with the number i, the process proceeds to S605. Note that 1 is added to the number of the current crop region in S606, which will be described later. Therefore, for example, after the process in S604 for the first crop region with the crop number 1 is completed, if the process in S604 is to be performed again via S606, the process in S604 is performed for the crop region with the crop number 2.

[0056] Next, in S605, the CPU 210 sets the spherical coordinates included in the crop region setting information to each of the four points of the crop region corners of the current crop region number i. Then, the process proceeds to S606. Next, in S606, the CPU 210 adds 1 to the current crop region number. Then, the process proceeds to S603 and performs the same process.

[0057] Below, with reference to FIG. 7, we will explain the change (change) in size and position of the crop area when the zoom magnification of the camera 100 changes and when the view angle maintenance mode is set or not set for the crop area.

[0058] In this embodiment, when the zoom magnification is changed, the display control unit 270 changes the size and position of the cropped areas in which the angle of view maintenance mode is set among the multiple cropped areas, relative to the image before the zoom magnification change, so as to maintain the angle of view according to the zoom magnification. Here, the image before the zoom magnification change is the image when the cropped area is set and the angle of view maintenance mode is set for the set cropped area. Therefore, even if the zoom magnification is changed multiple times, the size and position of the cropped area in which the angle of view maintenance mode is set changes so as to maintain the angle of view according to the zoom magnification, based on the image when the angle of view maintenance mode is set for the cropped area. On the other hand, when the zoom magnification is changed, the display control unit 270 does not change the size and position of the cropped area in which the angle of view maintenance mode is not set relative to the image before the zoom magnification change, depending on the zoom magnification.

[0059] Then, the display control unit 270 outputs the image in which the above-mentioned cropped area is set as a second image. When outputting the image, the display control unit 270 outputs an image in which the size and position of the cropped area are changed or not changed, or both, depending on the changed zoom magnification and the setting of the angle of view maintenance mode in the cropped area. That is, when the zoom magnification is changed, the display control unit 270 outputs an image in which the size and position of the cropped area in which the angle of view maintenance mode is set, among the multiple cropped areas, are changed so as to maintain the angle of view depending on the zoom magnification, relative to the image before the zoom magnification is changed. On the other hand, the display control unit 270 outputs an image in which the size and position of the cropped area in which the angle of view maintenance mode is not set, relative to the image before the zoom magnification is changed, are not changed depending on the zoom magnification. In this way, the display control unit 270 also functions as a change unit that changes the size and position of the cropped area relative to the image, depending on the setting of the zoom magnification and the angle of view maintenance mode (angle of view control mode).

[0060] Taking FIG. 7 as an example, when the zoom magnification of the camera 100 is reduced, the size and position of the crop area 703 of Crop1 relative to the image before the zoom magnification is changed do not change, as shown in the crop area 705 of Crop1. Also, even if the zoom magnification of the camera 100 is increased, the size and position of the crop area 705 of Crop1 relative to the image before the zoom magnification is changed do not change, as shown in the crop area 703. For example, assume that the image 701 is an image when a crop area is set and the angle of view maintenance mode is set for the set crop area. In that case, when the zoom magnification is reduced, the size and position of the crop area 705 relative to the image 701 do not change according to the zoom magnification. The same applies when the zoom magnification is increased.

[0061] On the other hand, when the zoom magnification of the camera 100 is reduced, the spherical coordinate values ​​of the crop area 704 of Crop2 are set to the four points of the crop area angle when the angle of view maintenance mode is set. Therefore, like the crop area 706 of Crop2, the size of the crop area is reduced and the position is changed while the angle of view is maintained with respect to the image before the zoom magnification is changed. Also, when the zoom magnification of the camera 100 is increased, the spherical coordinate values ​​of the crop area 706 of Crop2 are set to the four points of the crop area angle when the angle of view maintenance mode is set. Therefore, like the crop area 704 of Crop2, the size of the crop area is increased and the position is changed while the angle of view is maintained with respect to the image before the zoom magnification is changed. For example, assume that the image 701 is an image when a crop area is set and the angle of view maintenance mode is set for the set crop area. In this case, when the zoom magnification is reduced, the size and position of the crop area 705 for the image 701 are changed so as to maintain the angle of view according to the zoom magnification. The same applies when the zoom magnification is increased.

[0062] In addition, when the camera 100 is subjected to any of pan, tilt, and zoom control, if the angle of view of the cropped area to which the angle of view maintenance mode is set cannot be maintained, the setting of the angle of view maintenance mode of this cropped area may be cancelled (invalidated). That is, when the zoom magnification of the camera 100 changes and the angle of view of the cropped area to which the angle of view maintenance mode is set cannot be maintained, the above-mentioned control means invalidates the setting of the angle of view maintenance mode of the cropped area to which the angle of view instruction mode is set. Here, the inability to maintain the angle of view of the cropped area to which the angle of view maintenance mode is set includes the whole or part of the cropped area being at the edge of the camera image (for example, the image before the zoom magnification is changed), being cut off by the edge, or being outside the range. For example, it is assumed that the angle of view of the cropped area to which the angle of view maintenance mode is set cannot be maintained when the zoom magnification increases.

[0063] The same applies when the camera 100 is panned or tilted, and when a combination of a plurality of operations including panning, tilting, and zoom magnification change is performed. That is, the above-mentioned control means cancels (invalidates) the setting of the angle of view maintenance mode in a cropped area when at least a part of the cropped area in which the angle of view maintenance mode is set becomes the edge or outside the range of the image before the imaging direction of the camera 100 is changed (changed). Specifically, when the imaging direction of the camera 100 is changed, the setting of the angle of view maintenance mode is canceled when at least a part of the cropped area in which the angle of view maintenance mode is set becomes the edge or outside the range of the image before the zoom magnification is changed.

[0064] In addition, the setting of the angle of view maintenance mode in the cropped area where the angle of view maintenance mode is set may be temporarily cancelled (disabled). For example, assume that the angle of view maintenance mode was set, but the entire or part of the cropped area once went outside the range of the camera image (e.g., the image before the zoom magnification was changed), and the setting was cancelled. In this case, when the entire cropped area from which the setting was cancelled returns to the range of the camera image (e.g., the image before the zoom magnification was changed) as the zoom magnification of the camera 100 is reduced, the setting of the angle of view maintenance mode in the cropped area may be restored (enabled).

[0065] Also, for example, a case is assumed in which the angle of view maintenance mode has been set, but once the entire or part of the crop area goes outside the range of the camera image (for example, the image before the imaging direction is changed), and the setting is cancelled. In this case, when the entire crop area from which the setting has been cancelled returns to the range of the camera image (for example, the image before the imaging direction is changed) due to a change in the imaging direction, the angle of view maintenance mode setting for that crop area may be restored (enabled).

[0066] The control means described above may control the pan rotation, tilt rotation, and zoom magnification of the camera 100 so as to limit the range within which the angle of view of the cropped area in which the angle of view maintenance mode is set can be maintained. In other words, the control means may control the camera 100 so as to limit the zoom magnification within a range within which the angle of view of the cropped area in which the angle of view maintenance mode is set can be maintained. The control means may also control the camera 100 so as to limit the drive of the drive mechanism in each rotation direction (limit the drive of the drive mechanism that changes the imaging direction) so as to limit the range within which the angle of view of the cropped area in which the angle of view maintenance mode is set can be maintained.

[0067] Furthermore, when the zoom magnification of camera 100 becomes smaller, if the image quality of a cropped region in which the angle of view maintaining mode is set cannot be maintained, the above-mentioned control means may cancel (disable) the setting of the angle of view maintaining mode of this cropped region. Here, not being able to maintain the image quality of the cropped region includes the number of pixels of the cropped region becoming less than the number of pixels of the output resolution (the resolution when display control unit 270 outputs the second image). In other words, when the zoom magnification of camera 100 becomes smaller, if the number of pixels of a cropped region in which the angle of view maintaining mode is set becomes less than the output resolution, the above-mentioned control means may disable the setting of the angle of view maintaining mode of the cropped region.

[0068] In this case, the cancellation of the angle of view maintenance mode in the cropped area where the angle of view maintenance mode is set may be temporary. For example, when the zoom magnification of the camera 100 is increased, if the image quality of the cropped area where the angle of view maintenance mode is set can be maintained, the angle of view maintenance mode setting may be restored (enabled). Here, being able to maintain the image quality of the cropped area includes the number of pixels of the cropped area being equal to or greater than the number of pixels of the output resolution. That is, when the zoom magnification of the camera 100 is increased, if the number of pixels of the cropped area where the angle of view maintenance mode setting is released becomes equal to or greater than the output resolution, the above-mentioned control means may restore (enable) the angle of view maintenance mode setting of the cropped area.

[0069] The above-mentioned control means may control the camera 100 so that the zoom magnification of the camera 100 can be reduced only within a range in which the image quality of the cropped region in which the angle of view maintenance mode is set can be maintained when the zoom magnification of the camera 100 is reduced. In other words, the control means may control the camera 100 so as to limit the zoom magnification within a range in which the image quality of the cropped region in which the angle of view maintenance mode is set can be maintained.

[0070] The user may also specify a rate of degradation in image quality. In this case, the control means may use this rate as a threshold value to determine whether or not to apply the angle of view maintaining mode to the crop region when the user sets the angle of view maintaining mode to the crop region. Here, the rate of degradation in image quality includes the ratio between the number of pixels in the crop region and the number of pixels in the output resolution. In other words, the control means controls disabling and enabling of the setting of the angle of view maintaining mode in the crop region according to the ratio between the number of pixels in the crop region in which the angle of view maintaining mode is set and the number of pixels in the output resolution.

[0071] As described above, according to this embodiment, by making it possible to set a view angle maintaining mode for each crop area, it becomes possible to maintain the intended view angle and image quality of each crop area when the user operates the pan / tilt / zoom of camera 100.

[0072] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, the present invention can be embodied as a system, device, method, program, or recording medium (storage medium). Specifically, the present invention may be applied to a system composed of multiple devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or may be applied to an apparatus composed of a single device.

[0073] Also, for example, some or all of the functions of the functional units of the controller 200 shown in FIG. 1 may be included in a device different from the controller 200. Specifically, a device or storage device different from the controller 200 may have these functional units, and the functions of each embodiment may be realized by communicating with the controller 200 based on a wired or wireless connection. Examples of the different device include an external communication device such as the camera 100 or a server. Also, for example, one or more functional units in FIG. 1 may be realized by one or more computers different from the controller 200. Also, some or all of the functions of the functional units in FIG. 1 may be included in one or more devices different from the controller 200.

[0074] The disclosure of this embodiment includes the following configuration, method, program, and system.

[0075] (Configuration 1) a control means for displaying on a display device a setting screen on which a plurality of cropping areas and a view angle control mode for controlling a view angle in the plurality of cropping areas can be set for a first image captured by an imaging means in the imaging device; an output means for outputting the image in which the crop region is set as a second image; a change unit that changes a size and a position of the crop area relative to an image in accordance with a setting of the angle of view control mode, when the zoom magnification of the imaging device is changed, the change means does not change, in accordance with the zoom magnification, a size and a position of a crop region in which the angle of view control mode is not set, among the plurality of crop regions, relative to an image before the zoom magnification is changed, and changes, in accordance with the zoom magnification, a size and a position of a crop region in which the angle of view control mode is set, relative to an image before the zoom magnification is changed, so as to maintain the angle of view. 23. An information processing apparatus comprising:

[0076] (Configuration 2) The information processing device described in configuration 1, wherein the control means displays on the display device a setting screen capable of setting a first cropping region in which the angle of view control mode is set and a second cropping region in which the angle of view control mode is not set for the first image.

[0077] (Configuration 3) The information processing device according to configuration 1 or 2, characterized in that, when the angle of view of the cropping area to which the angle of view control mode is set cannot be maintained when the zoom magnification is changed, the control means disables the setting of the angle of view control mode in the cropping area to which the angle of view control mode is set.

[0078] (Configuration 4) 4. The information processing device according to any one of configurations 1 to 3, wherein the control means limits the zoom magnification within a range in which the angle of view in the crop region in which the angle of view control mode is set can be maintained.

[0079] (Configuration 5) The information processing device according to configuration 1 or 2, characterized in that the control means, when the zoom magnification becomes large and at least a part of the cropping region in which the angle of view control mode is set becomes outside the edge or range of the image before the zoom magnification is changed, disables the setting of the angle of view control mode in the cropping region in which the angle of view control mode is set.

[0080] (Configuration 6) The information processing device described in configuration 5, characterized in that when the zoom magnification becomes small and the entire crop region to which the angle of view control mode is set falls within the image before the zoom magnification is changed, the control means enables the setting of the angle of view control mode for the crop region for which the setting of the angle of view control mode has been disabled.

[0081] (Configuration 7) The information processing device described in any one of configurations 1 to 6, characterized in that, when an imaging direction of the imaging device is changed, at least a part of the cropping region in which the angle of view control mode is set becomes outside the edge or range of the image before the imaging direction is changed, the control means disables the setting of the angle of view control mode in the cropping region in which the angle of view control mode is set.

[0082] (Configuration 8) The information processing device according to configuration 7, wherein the control means, when the imaging direction changes and the entire area of ​​the cropping area to which the angle of view control mode is set is contained within the image, enables the setting of the angle of view control mode for the cropping area for which the setting of the angle of view control mode has been disabled.

[0083] (Configuration 9) The information processing device according to any one of configurations 1 to 8, wherein the control means limits driving of a drive means that changes an imaging direction of the imaging device so that the angle of view in the crop area in which the angle of view control mode is set is within a range in which the angle of view can be maintained.

[0084] (Configuration 10) The information processing device described in any one of configurations 1 to 9, characterized in that the control means disables the setting of the angle of view control mode for the crop region to which the angle of view control mode is set when the zoom magnification becomes small and image quality of the crop region to which the angle of view control mode is set cannot be maintained.

[0085] (Configuration 11) 11. The information processing apparatus according to configuration 10, wherein the control means limits the zoom magnification within a range in which the image quality in the crop region in which the angle of view control mode is set can be maintained.

[0086] (Configuration 12) The information processing device described in any one of configurations 1 to 9, characterized in that the control means disables the setting of the angle of view control mode of the cropping area to which the angle of view control mode is set when the zoom magnification becomes small and the number of pixels of the cropping area to which the angle of view control mode is set becomes less than the number of pixels of the resolution when the output means outputs the second image.

[0087] (Configuration 13) The information processing device described in configuration 12, characterized in that when the zoom magnification becomes large and the number of pixels in the cropped area to which the angle of view control mode is set becomes equal to or greater than the number of pixels of the resolution, the control means enables the angle of view control mode for the cropped area for which the angle of view control mode setting has been disabled.

[0088] (Configuration 14) The information processing device described in any one of configurations 1 to 13, characterized in that the control means disables or enables the setting of the angle of view control mode in the crop area depending on the ratio of the number of pixels in the crop area to which the angle of view control mode is set and the number of pixels of the resolution when the output means outputs the second image.

[0089] (Configuration 15) An information processing device according to any one of configurations 1 to 14, characterized in that the setting screen displays at least information on the coordinates, width and height of the set crop area, and the setting state of the angle of view control mode.

[0090] (Configuration 16) A method for controlling an information processing device, comprising: a display step of displaying on a display device a setting screen on which a plurality of cropping regions and a view angle control mode for controlling a view angle in the plurality of cropping regions can be set for a first image captured by an imaging means in the imaging device; an output step of outputting the image in which the crop region is set as a second image; and changing a size and a position of the cropping area relative to an image in accordance with a setting of the angle of view control mode, In the changing step, when the zoom magnification of the imaging device is changed, a size and a position of a crop region in which the angle of view control mode is not set among the plurality of crop regions with respect to an image before the zoom magnification is changed are not changed in accordance with the zoom magnification, and a size and a position of a crop region in which the angle of view control mode is set with respect to an image before the zoom magnification is changed are changed in accordance with the zoom magnification so as to maintain the angle of view. 23. A method for controlling an information processing apparatus comprising:

[0091] (Configuration 17) A program for causing a computer to execute a control method for an information processing device, the program comprising: a display step of displaying on a display device a setting screen on which a plurality of cropping regions and a view angle control mode for controlling a view angle in the plurality of cropping regions can be set for a first image captured by an imaging means in the imaging device; an output step of outputting the image in which the crop region is set as a second image; a changing step of changing a size and a position of the cropping area with respect to the image in accordance with a setting of the angle of view control mode; In the changing step, when the zoom magnification of the imaging device is changed, a size and a position of a crop region in which the angle of view control mode is not set among the plurality of crop regions with respect to an image before the zoom magnification is changed are not changed in accordance with the zoom magnification, and a size and a position of a crop region in which the angle of view control mode is set with respect to an image before the zoom magnification is changed are changed in accordance with the zoom magnification so as to maintain the angle of view. A program characterized by:

[0092] (Configuration 18) An imaging system including at least one imaging device and an information processing device communicably connected to the imaging device, The imaging device includes: An imaging means for acquiring a first image; a zoom unit for changing a zoom magnification of the imaging device, The information processing device includes: a control means for displaying on a display device a setting screen on which a plurality of cropping regions can be set and a view angle control mode for controlling a view angle in the plurality of cropping regions can be set for the first image; an output means for outputting the image in which the crop region is set as a second image; a change unit that changes a size and a position of the crop area relative to an image in accordance with a setting of the angle of view control mode, when the zoom magnification is changed, the change means does not change, in accordance with the zoom magnification, a size and a position of a crop region in which the angle of view control mode is not set, among the plurality of crop regions, relative to the image before the zoom magnification is changed, and changes, in accordance with the zoom magnification, a size and a position of a crop region in which the angle of view control mode is set, relative to the image before the zoom magnification is changed, so as to maintain the angle of view. 1. An imaging system comprising:

[0093] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. [Explanation of symbols]

[0094] 100 Cameras 110 CPU 120 RAM 130 ROM 140 Image capture I / F 150 Network Interface 160 Image Sensor 170 Drive control unit 180 Image Processing Unit 190 Communication control section 1000 Internal Path 200 Controller 210 CPU 220 RAM 230 ROM 240 User Input / Output Interface 250 Network Interface 260 Input control section 270 Display control unit 280 Communication Control Unit 2000 Internal Path 300 IP Network

Claims

1. a control means for displaying on a display device a setting screen on which a plurality of cropping regions and a view angle control mode for controlling a view angle in the plurality of cropping regions can be set for a first image captured by an imaging means in the imaging device; an output means for outputting the image in which the crop region is set as a second image; a change unit that changes a size and a position of the crop area relative to an image in accordance with a setting of the angle of view control mode, when the zoom magnification of the imaging device is changed, the change means does not change, in accordance with the zoom magnification, a size and a position of a crop region in which the angle of view control mode is not set, among the plurality of crop regions, relative to an image before the zoom magnification is changed, and changes, in accordance with the zoom magnification, a size and a position of a crop region in which the angle of view control mode is set, relative to an image before the zoom magnification is changed, so as to maintain the angle of view.

23. An information processing apparatus comprising:

2. The information processing device according to claim 1, characterized in that the control means displays on the display device a setting screen capable of setting a first cropping region in which the angle of view control mode is set and a second cropping region in which the angle of view control mode is not set for the first image.

3. 2. The information processing device according to claim 1, wherein, when the angle of view of the cropped area to which the angle of view control mode is set cannot be maintained when the zoom magnification is changed, the control means disables the setting of the angle of view control mode in the cropped area to which the angle of view control mode is set.

4. The information processing apparatus according to claim 1 , wherein the control means limits the zoom magnification within a range in which the angle of view in the crop region in which the angle of view control mode is set can be maintained.

5. 2. The information processing device according to claim 1, wherein the control means invalidates the setting of the angle-of-view control mode in the crop region to which the angle-of-view control mode is set when the zoom magnification becomes large and at least a part of the crop region to which the angle-of-view control mode is set becomes an edge or outside a range of the image before the zoom magnification is changed.

6. 6. The information processing device according to claim 5, wherein when the zoom magnification becomes small and the entire crop region to which the angle of view control mode is set falls within the image before the zoom magnification is changed, the control means enables the setting of the angle of view control mode for the crop region for which the setting of the angle of view control mode has been disabled.

7. 2. The information processing device according to claim 1, wherein, when an imaging direction of the imaging device is changed, if at least a part of a cropping region in which the angle of view control mode is set becomes an edge or outside a range of an image before the imaging direction is changed, the control means invalidates the setting of the angle of view control mode in the cropping region in which the angle of view control mode is set.

8. 8. The information processing device according to claim 7, wherein when the imaging direction changes and the entire area of ​​the cropping area to which the angle of view control mode is set falls within the image, the control means enables the setting of the angle of view control mode for the cropping area for which the setting of the angle of view control mode has been disabled.

9. 2 . The information processing apparatus according to claim 1 , wherein the control means limits driving of a drive means that changes the imaging direction of the imaging device so that the angle of view in the crop region in which the angle of view control mode is set is within a range in which the angle of view can be maintained.

10. 2. The information processing device according to claim 1, wherein the control means disables the setting of the angle-of-view control mode for the cropping area to which the angle-of-view control mode is set when the zoom magnification becomes small and image quality of the cropping area to which the angle-of-view control mode is set cannot be maintained.

11. 11. The information processing apparatus according to claim 10, wherein the control means limits the zoom magnification within a range in which the image quality in the crop region in which the angle of view control mode is set can be maintained.

12. 2. The information processing device according to claim 1, wherein the control means disables the setting of the angle of view control mode of the cropping area to which the angle of view control mode is set when the zoom magnification becomes small and the number of pixels of the cropping area to which the angle of view control mode is set becomes less than the number of pixels of a resolution when the output means outputs the second image.

13. 13. The information processing device according to claim 12, wherein when the zoom magnification becomes large and the number of pixels in the cropped area to which the angle of view control mode is set becomes equal to or greater than the number of pixels of the resolution, the control means enables the angle of view control mode for the cropped area for which the angle of view control mode setting has been disabled.

14. 2. The information processing device according to claim 1, wherein the control means disables or enables the setting of the angle of view control mode in the cropping area depending on a ratio between the number of pixels of the cropping area in which the angle of view control mode is set and the number of pixels of a resolution when the output means outputs the second image.

15. 2 . The information processing apparatus according to claim 1 , wherein the setting screen displays at least information on the coordinates, width and height of the set crop area, and a setting state of the angle of view control mode.

16. A method for controlling an information processing device, comprising: a display step of displaying on a display device a setting screen on which a plurality of cropping regions and a view angle control mode for controlling a view angle in the plurality of cropping regions can be set for a first image captured by an imaging means in the imaging device; an output step of outputting the image in which the crop region is set as a second image; and changing a size and a position of the cropping area relative to an image in accordance with a setting of the angle of view control mode, In the changing step, when the zoom magnification of the imaging device is changed, a size and a position of a crop region in which the angle of view control mode is not set among the plurality of crop regions with respect to an image before the zoom magnification is changed are not changed in accordance with the zoom magnification, and a size and a position of a crop region in which the angle of view control mode is set with respect to an image before the zoom magnification is changed are changed in accordance with the zoom magnification so as to maintain the angle of view.

23. A method for controlling an information processing apparatus comprising:

17. A program for causing a computer to execute a control method for an information processing device, the program comprising: a display step of displaying on a display device a setting screen on which a plurality of cropping regions and a view angle control mode for controlling a view angle in the plurality of cropping regions can be set for a first image captured by an imaging means in the imaging device; an output step of outputting the image in which the crop region is set as a second image; a changing step of changing a size and a position of the cropping area with respect to the image in accordance with a setting of the angle of view control mode; In the changing step, when the zoom magnification of the imaging device is changed, a size and a position of a crop region in which the angle of view control mode is not set among the plurality of crop regions with respect to an image before the zoom magnification is changed are not changed in accordance with the zoom magnification, and a size and a position of a crop region in which the angle of view control mode is set with respect to an image before the zoom magnification is changed are changed in accordance with the zoom magnification so as to maintain the angle of view. A program characterized by:

18. An imaging system including at least one imaging device and an information processing device communicably connected to the imaging device, The imaging device includes: An imaging means for acquiring a first image; a zoom unit for changing a zoom magnification of the imaging device, The information processing device includes: a control means for displaying on a display device a setting screen on which a plurality of cropping regions can be set and a view angle control mode for controlling a view angle in the plurality of cropping regions can be set for the first image; an output means for outputting the image in which the crop region is set as a second image; a change unit that changes a size and a position of the crop area relative to an image in accordance with a setting of the angle of view control mode, when the zoom magnification is changed, the change means does not change, in accordance with the zoom magnification, a size and a position of a crop region in which the angle of view control mode is not set, among the plurality of crop regions, relative to the image before the zoom magnification is changed, and changes, in accordance with the zoom magnification, a size and a position of a crop region in which the angle of view control mode is set, relative to the image before the zoom magnification is changed, so as to maintain the angle of view.

1. An imaging system comprising:

Images