Imaging system, method, and program
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- CANON KK
- Filing Date
- 2023-06-16
- Publication Date
- 2026-06-25
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an imaging system, a method, and a program. [Background technology]
[0002] In recent years, imaging devices that can be operated remotely have been increasing in number to reduce manpower. One of these is a live streaming camera that distributes captured video images over a network. One of the functions related to driving the pan, tilt, and zoom of a live streaming camera is a function (hereinafter referred to as loop operation) that gives a driving instruction to the imaging device in advance using a device such as a switcher that gives driving instructions such as panning, tilting, and zooming to the imaging device, and repeats the set operation. This function continues driving even after the connection with the switcher is cut off, so by using this function, multiple imaging devices can be driven simultaneously by one switcher.
[0003] Another technology for driving an imaging device even when no driving instruction is given is a preset function used in surveillance imaging devices. This function is a function for setting an event or time as a trigger in advance, setting a driving path and driving speed, and driving the device. Many patent documents have been published regarding variations in driving methods using the preset function. For example, Patent Document 1 associates an image displayed on a Graphical User Interface (GUI) with pan, tilt, and zoom information of the imaging device in order to perform control that reflects the client's intention. Patent Document 1 also discloses a technology for specifying multiple points from an image displayed on a GUI, generating a line based on the multiple points, and calculating the preset position, tour order, driving speed, etc. that are most similar to the trajectory of the generated line. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2017-017646 A Summary of the Invention [Problem to be solved by the invention]
[0005] However, in the conventional technology disclosed in the above-mentioned patent document, the drive path is determined by specifying the patrol position while viewing the image on the GUI, so when the drive path of the imaging device is determined by a control method that does not involve a GUI, such as a joystick connected to the imaging device, it is not possible to modify the drive path.
[0006] Therefore, an object of the present invention is to provide an imaging system, method, and program that makes it possible to easily correct the set loop operation path, even when the path for the imaging device to perform loop operation is determined without using a GUI. [Means for solving the problem]
[0007] In order to solve the above-mentioned problems, the imaging system of the present invention is characterized in that it comprises an operation means for receiving movement information specifying a path of an imaging angle of view of the imaging device and moving the imaging angle of view of the imaging device, a display means for displaying a pre-correction path, which is the path of the imaging angle of view of the imaging device specified by the movement information, and a post-correction path, which is the path of the imaging angle of view of the imaging device generated by correcting the pre-correction path, and a correction means for generating the post-correction path by correcting the pre-correction path based on at least one of the position, direction, and speed of the imaging angle of view of the imaging device included in the movement information. Effect of the Invention
[0008] According to the present invention, even in the case where the route for the loop operation of the imaging device is determined without using a GUI, it is possible to easily correct the route of the set loop operation. [Brief description of the drawings]
[0009] [Figure 1]1 is a block diagram showing an example of a configuration of an imaging system according to an embodiment. [Diagram 2] 2 is a block diagram showing an example of a hardware configuration of a processing unit according to the embodiment; FIG. [Diagram 3] 11 is a flowchart illustrating an example of a process in which an imaging system according to an embodiment corrects an uncorrected path to generate a corrected path and displays the corrected path on a display unit. [Figure 4] FIG. 11 is a diagram illustrating an example of a process for generating a composite image according to the embodiment. [Diagram 5] 11A and 11B are diagrams illustrating an example of a composite image and a pre-correction path displayed on a display unit according to an embodiment. [Figure 6] 10 is a flowchart illustrating an example of a process for displaying a corrected route on a display unit according to an embodiment. [Figure 7] 13 is a diagram showing an example of a screen displayed on a display unit when a corrected path according to the embodiment is generated; FIG. [Figure 8] 13 is a diagram showing an example of a screen displayed on a display unit when a corrected path according to the embodiment is generated; FIG. [Figure 9] 13 is a diagram showing an example of a screen displayed on a display unit when a corrected path according to the embodiment is generated; FIG. [Figure 10] 11A to 11C are diagrams illustrating an example of a process for correcting the position of the imaging angle of view of the imaging device according to the embodiment. [Figure 11] 11A and 11B are diagrams for explaining manual position correction executed on a UI according to an embodiment. [Figure 12] 11A and 11B are diagrams for explaining manual position correction executed on a UI according to an embodiment. [Figure 13] 11A and 11B are diagrams for explaining manual position correction executed on a UI according to an embodiment. [Figure 14] 11A to 11C are diagrams illustrating an example of the positional relationship between an imaging device and a subject when the imaging system according to the embodiment corrects the speed of the imaging angle of view of the imaging device. [Figure 15] 11A to 11C are diagrams illustrating an example of a method for correcting the speed of the imaging angle of view of an imaging device according to an embodiment and a method for displaying the speed of the imaging angle of view of an imaging device. [Figure 16] 11A to 11C are diagrams illustrating an example of a method for correcting the speed of the imaging angle of view of an imaging device according to an embodiment and a method for displaying the speed of the imaging angle of view of an imaging device. [Figure 17] 11A to 11C are diagrams illustrating an example of a method for correcting the speed of the imaging angle of view of an imaging device according to an embodiment and a method for displaying the speed of the imaging angle of view of an imaging device. [Figure 18] 11A to 11C are diagrams illustrating an example of a method for correcting the speed of the imaging angle of view of an imaging device according to an embodiment and a method for displaying the speed of the imaging angle of view of an imaging device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the configurations described below.
[0011] Fig. 1 is a block diagram showing an example of the configuration of an imaging system according to an embodiment. An imaging device 100 shown in Fig. 1 captures an image of an area set by a client. As shown in Fig. 1, the imaging device 100 includes an imaging unit 101, a lens driving unit 102, a processing unit 103, a driving control unit 104, a tilt driving unit 105, a pan driving unit 106, a storage unit 107, and an interface unit 108.
[0012] The imaging unit 101 includes an imaging optical system and an imaging element. The imaging optical system includes a focus lens and a zoom lens for forming an image from incident light. The imaging element photoelectrically converts a subject image (optical image) obtained via the imaging optical system and outputs a pixel signal. The lens driving unit 102 is a zoom means that drives the focus lens and zoom lens to change the focal position and zoom magnification.
[0013] The processing unit 103 performs image processing, image analysis processing, and image distribution processing on the input image from the imaging element, and also performs control processing of each driving unit via a driving control unit 104 composed of a motor driver, etc. The processing unit 103 is, for example, a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Gate Array), but is not limited to these.
[0014] 1, the processing unit 103 includes a correction unit 1031. The correction unit 1031 generates a corrected path by correcting the pre-correction path based on at least one of the position, direction, and speed of the imaging angle of view of the imaging device included in the movement information. Details of the process executed by the correction unit 1031 will be described later.
[0015] The drive control unit 104 drives and controls the lens drive unit 102, tilt drive unit 105, and pan drive unit 106 based on a control signal received from the processing unit 103, thereby causing the imaging device 100 to perform focus, zoom, tilt operation, and pan operation. The tilt drive unit 105 is a mechanism that causes the imaging device 100 to perform a tilt operation. The pan drive unit 106 is a mechanism that causes the imaging device 100 to perform a pan operation. Note that, although the processing unit 103 and the drive control unit 104 are configured separately here, the processing unit 103 may also function as the drive control unit 104, and the processing unit 103 may drive and control the lens drive unit 102, tilt drive unit 105, and pan drive unit 106.
[0016] The storage unit 107 stores data of software for controlling the entire imaging device 100, a drive path for loop operation, a drive speed, and the like. The storage unit 107 is, for example, a non-volatile memory having a capacity of several kB to several hundred MB. The non-volatile memory employed as the storage unit 107 is, for example, a flash ROM or an EEPROM (Electrically Erasable and Programmable ROM), but is not limited to these.
[0017] The interface unit 108 includes an output I / F for distributing video such as HDMI (High-Definition Multimedia Interface), (registered trademark), SDI (Serial Digital Interface), and a composite signal. The interface unit 108 also includes an IO port for controlling an imaging device such as Ethernet and RS-422 (Recommended Standard 422).
[0018] The imaging device 100 transmits and receives captured images and various setting information to and from a user interface (UI) unit 109 via an interface unit 108 over Ethernet or the like. The UI unit 109 includes a display unit 110 and an operation unit 111.
[0019] The display unit 110 includes a PC (Personal Computer), a liquid crystal display, etc., and displays captured images and various setting information transmitted from the imaging device 100. The display unit 110 displays a pre-correction path that is a path of the imaging angle of view of the imaging device 100 specified by the movement information, and a post-correction path that is a path of the imaging angle of view of the imaging device 100 generated by correcting the pre-correction path.
[0020] The operation unit 111 includes a switcher including a joystick capable of indicating a drive direction of the imaging device 100. The operation unit 111 receives movement information that specifies a path of the imaging angle of view of the imaging device 100 and moves the imaging angle of view of the imaging device 100.
[0021] The client can change the imaging angle of view, change the imaging conditions, and control various functions by transmitting control information from the display unit 110 and the operation unit 111 to the imaging device 100 .
[0022] Next, another example of the hardware configuration of the processing unit 103 and the drive control unit 104 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the hardware configuration of the processing unit according to the embodiment. Since the hardware configuration of the processing unit 103 and the hardware configuration of the drive control unit 104 are the same or similar, the hardware configuration of the processing unit 103 will be described as an example. As shown in Fig. 2, the processing unit 103 includes a CPU (Central Processing Unit) 1001, a ROM (Read Only Memory) 1002, a RAM (Random Access Memory) 1003, a storage 1004, an input / output I / F 1005, and a bus 1006.
[0023] The CPU 1001 reads out and executes a program for implementing the functions of the processing unit 103. The ROM 1002 stores a program for implementing the functions of the processing unit 103. The RAM 1003 is used when the CPU 1001 expands and executes a program read from the ROM 1002. The storage 1004 is, for example, a hard disk drive (HDD) or a solid state drive (SSD), and stores files, data, and the like. The input / output I / F 1005 includes a plurality of interfaces, and is connected to input devices such as a mouse and a keyboard, and output devices such as a display and a speaker. The bus 1006 connects the CPU 1001, the ROM 1002, the RAM 1003, the storage 1004, and the input / output I / F 1005 in a manner that allows them to communicate with each other.
[0024] Note that both the functions of the processing unit 103 and the drive control unit 104 may be realized by a single configuration shown in FIG.
[0025] Next, an example of processing executed by the imaging system according to the embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an example of processing in which the imaging system according to the embodiment corrects an uncorrected path to generate a corrected path and displays the corrected path on a display unit.
[0026] In step S201, the imaging device 100 captures images using the imaging unit 101 while performing pan driving and tilt driving so as to cover the entire imaging range within a range where pan driving and tilt driving are possible. The imaging device 100 generates a panoramic image by synthesizing the captured images using the processing unit 103. Then, the imaging device 100 associates the panoramic image with the pan information and tilt information of the panoramic image and stores them in the storage unit 107.
[0027] In step S202, movement information of the imaging device 100 on a route to be registered as a loop operation is input from the operation unit 111 to the processing unit 103 via the interface unit 108. The processing unit 103 converts the movement information of the imaging device 100 on the route into pan drive control information, tilt drive control information, and zoom drive control information of the route, and transmits the pan drive control information, tilt drive control information, and zoom drive control information to the drive control unit 104.
[0028] In step S203, the drive control unit 104 drives the lens drive unit 102, the tilt drive unit 105, and the pan drive unit 106 based on the pan drive control information, tilt drive control information, and zoom drive control information input from the processing unit 103 to the drive control unit 104.
[0029] In step S204, the display unit 110 displays the pre-correction path superimposed on the panoramic image. This process is executed based on the pan drive control information, tilt drive control information, and zoom drive control information of the pre-correction path calculated by the processing unit 103, the panoramic image recorded in the storage unit 107, and the pan drive information and tilt drive information associated with the panoramic image.
[0030] In step S205, the processing unit 103 transmits the panoramic image on which the trajectory of the pre-correction path is superimposed to the display unit 110 via the interface unit 108, and displays the panoramic image on which the trajectory of the pre-correction path is superimposed on the display unit 110.
[0031] In step S206, the processing unit 103 determines whether or not to correct the pre-correction path. For example, when an instruction to correct the pre-correction path is accepted via the UI unit 109, the processing unit 103 advances the process to step S207. On the other hand, for example, when an instruction to correct the pre-correction path is not accepted via the UI unit 109, the processing unit 103 advances the process to step S209.
[0032] In step S207, the correction unit 1031 calculates at least one corrected path based on the pan drive control information, tilt drive control information, and zoom drive control information of the uncorrected path. A method of calculating the corrected path will be described later.
[0033] In step S208, the display unit 110 superimposes the corrected path calculated in step S207 on the panoramic image.
[0034] In step S209, the processing unit 103 accepts, from the UI unit 109, a selection of a corrected path for executing a loop operation from among the corrected paths displayed superimposed on the panoramic image displayed on the display unit 110.
[0035] The imaging system 100 displays a pre-correction path on a panoramic image and displays a UI for correcting the displayed pre-correction path. This allows the imaging system 100 to easily change the drive position, drive direction, and drive speed of the drive path in the set loop operation, even when the imaging device determines the drive path for loop operation without using a GUI.
[0036] Next, a method for displaying a path that performs a loop operation during correction will be described with reference to Fig. 4 and Fig. 5. When executing this display method, the display unit 110 displays a composite image generated by combining a plurality of images captured while the imaging angle of view of the imaging device 100 is moving based on the movement information. The composite image is, for example, the above-mentioned panoramic image. Then, the display unit 110 displays at least one of the pre-correction path and the post-correction path by superimposing it on the composite image.
[0037] In this display method, the path of the imaging angle of view of the imaging device 100 may be a locus of the center of the range in which the imaging device 100 can capture images. Alternatively, the path of the imaging angle of view of the imaging device 100 may be a locus of a line or surface that combines points on any coordinates within the range in which the imaging device 100 can capture images. Displaying a path that performs a loop operation requires two procedures: generating a panoramic image, and superimposing pan drive information, tilt drive information, and zoom drive information of the path on the panoramic image. A specific example of this display method will be described.
[0038] Fig. 4 is a diagram showing an example of a process for generating a composite image according to the embodiment. Specifically, Fig. 4 shows an example of a process for generating a panoramic image. The imaging range of the imaging device 100 depends on the pan driveable range and tilt driveable range of the imaging device 100. The panoramic partial image 301 is an image captured when the pan is driven to the left end and the tilt is driven to the top end. On the other hand, the panoramic partial image 302 is an image captured when the pan is driven to the right end and the tilt is driven to the bottom end.
[0039] The imaging device 100 drives the pan and tilt to cover the panorama partial image 301 to the panorama partial image 302, thereby generating a panorama image 303. The imaging device 100 records the captured image and the pan driving information and the tilt driving information at the time of capturing the image in association with each other, thereby being able to calculate the pan driving information and the tilt driving information at an arbitrary position 304 on the panorama image.
[0040] When displaying the pre-correction path on the panoramic image, displaying a representative point in an arbitrary position 304 on the panoramic image makes it easier to view the pre-correction path on the display unit 110. Therefore, in this embodiment, the imaging center position at the arbitrary position 304 on the panoramic image is set as the representative point 305, and is recorded in association with the pan driving information and the tilt driving information. Here, the position on the panoramic image associated with the pan driving information and the tilt driving information is set as the imaging center position, but is not limited to this, and may be a line or area combining arbitrary positions within the imaging range at the arbitrary position 304 on the panoramic image.
[0041] Fig. 5 is a diagram showing an example of a composite image and a pre-correction path displayed on a display unit according to an embodiment. Specifically, Fig. 5 shows an example of a screen displayed on the display unit 110 when a pre-correction path displayed on the display unit 110 is superimposed on a panoramic image. In the loop function, since the imaging device usually moves the same path repeatedly, it is desirable that the start point and the end point coincide with each other.
[0042] Therefore, it is preferable that the correction unit 1031 generates a corrected path in which the position of the imaging angle of view at the start point of the corrected path coincides with the position of the imaging angle of view at the end point of the corrected path. It is also preferable that the correction unit 1031 generates a corrected path in which the direction of the imaging angle of view at the start point of the corrected path coincides with the direction of the imaging angle of view at the end point of the corrected path. It is also preferable that the correction unit 1031 generates a corrected path in which the speed of the imaging angle of view at the start point of the corrected path coincides with the speed of the imaging angle of view at the end point of the corrected path.
[0043] In this embodiment, it is assumed that a pre-correction route in which the start point and the end point do not match is input from the operation unit 111. By acquiring pan driving information, tilt driving information, and zoom driving information based on the information input from the operation unit 111 and drawing the trajectory of the representative point 305 shown in FIG. 4, it is possible to superimpose the pre-correction route 401 on the panoramic image. Here, by drawing not only the representative point 305 but also the trajectory from the start point position image 402 to the end point position image 403, it is also possible to show the imaging range 404 on the route and further the non-imaging range 405 on the panoramic image. By clarifying the imaging range, the user can easily determine whether or not to correct the pre-correction route.
[0044] A display method on the display unit 110 during correction will be described with reference to Fig. 6 to Fig. 9. When executing this display method, the display unit 110 displays a second user interface for specifying a range to be corrected in the pre-correction path. Then, the correction unit 1031 generates a corrected path by correcting the range of the pre-correction path specified by the second user interface. A specific example of this display method will be described.
[0045] Fig. 6 is a flowchart showing an example of a process for displaying a corrected route on the display unit according to the embodiment. Specifically, Fig. 6 shows a method for displaying a corrected route on the display unit 110 when the process according to the embodiment is executed based on the flowchart shown in Fig. 3.
[0046] In step S501, the display unit 110 displays the pre-correction route superimposed on the panoramic image, similarly to step S204 described above.
[0047] In step S502, the display unit 110 displays a pop-up near the displayed pre-correction path, prompting the user to confirm whether or not to correct the pre-correction path.
[0048] In step S503, the processing unit 103 determines whether or not to correct the pre-correction path, similar to step S206 described above. If the processing unit 103 determines that the pre-correction path is to be corrected, the processing proceeds to step S504. On the other hand, if the processing unit 103 determines that the pre-correction path is not to be corrected, the processing proceeds to step S508.
[0049] In step S504, the display unit 110 displays the path correction start point and the path correction end point superimposed on the displayed pre-correction path. The user can change the positions of the path correction start point and the path correction end point from the UI unit 109 to any positions on the pre-correction path.
[0050] In step S505, the display unit 110 displays a pop-up near the displayed pre-correction route, prompting the user to change the route correction start point and the route correction end point.
[0051] In step S506, the correction unit 1031 calculates at least one corrected path based on the pan drive control information, tilt drive control information, and zoom drive control information of the pre-correction path, similarly to step S207. A method of calculating the corrected path will be described later.
[0052] In step S507, the display unit 110 superimposes the corrected path calculated in step S207 on the panoramic image, similarly to step S208.
[0053] In step S508, similar to step S209, the processing unit 103 accepts from the UI unit 109 a selection of a corrected path that will execute a loop operation from among the corrected paths displayed superimposed on the panoramic image displayed on the display unit 110.
[0054] Fig. 7 to Fig. 9 are diagrams showing examples of screens displayed on the display unit when a corrected route according to the embodiment is generated. Specifically, Fig. 7 to Fig. 9 show examples of screens displayed on the display unit 110 during route correction. The display screen of the display unit 110 changes according to the flowchart shown in Fig. 6.
[0055] As shown in Fig. 7, when input of movement information of the imaging device 100 from the operation unit 111 is completed, the display unit 110 displays a pre-correction path 601 superimposed on a panoramic image. Furthermore, the display unit 110 displays a pop-up 602 notifying the user to input whether or not to correct the pre-correction path. Here, when the user selects to correct the pre-correction path, the display unit 110 displays a pop-up 605 on the display unit 110 notifying the user to determine a path correction start point 603 and a path correction end point 604, as shown in Fig. 8.
[0056] The user inputs a route correction start point 603 and a route correction end point 604 via the UI unit 109. Here, the route correction start point 502 and the route correction end point 604 can be selected only on the pre-correction route 601. Note that, when it is assumed that the entire pre-correction route input from the operation unit 111 is to be corrected from the start point to the end point, the route correction start point 603 and the route correction end point 604 may be automatically calculated by the processing unit 103 without inputting them from the operation unit 111.
[0057] As shown in FIG. 9, the correction unit 1031 corrects the pre-correction path after the correction start point 603 and the correction end point 604 are determined. Then, the display unit 110 displays the pre-correction path 601 together with the post-correction path 606. Furthermore, the display unit 110 displays a pop-up 607 for confirming with the user whether or not to use the path. Note that a specific process in which the correction unit 1031 corrects the pre-correction path and generates the post-correction path will be described later. In FIG. 9, only one post-correction path is displayed, but this is not limited thereto, and a plurality of post-correction paths calculated by the correction unit 1031 may be displayed. Furthermore, a pop-up for redoing the correction again without selecting a path may be displayed, and a process of repeating the correction again may be executed.
[0058] Next, a method for correcting a pre-correction path will be described with reference to Fig. 10. When performing this method, the correction unit 1031 generates a corrected path by correcting the shortest pre-correction path among a plurality of pre-correction paths. When performing this method, the correction unit 1031 also generates a corrected path based on the curvature of the pre-correction path. A specific example of this method will be described.
[0059] Fig. 10 is a diagram showing an example of a process for correcting the position of the imaging angle of view of the imaging device according to the embodiment. Specifically, Fig. 10 shows only the pre-correction path and the post-correction path that are displayed superimposed on the panoramic image. Below, a method in which the correction unit 1031 corrects the pre-correction path 701 and calculates the post-correction path 702 will be described. Here, the correction unit 1031 corrects from the correction start point 703 to the end point. Two correction methods will be described.
[0060] The first method is to minimize the path difference 704. The path difference 704 is a value calculated as the difference on a panoramic image between an uncorrected path 701 and a corrected path 702. By minimizing the path difference 704, it is possible to calculate a corrected path that drives the image capture device 100 in the shortest time while taking into account the user's intention by maintaining the uncorrected path information from the correction start point to the end point.
[0061] The second method is to calculate a corrected path 702 while maintaining the curvature of the pre-corrected path before the path correction start point 703. A curvature calculation range 705 is a range of the pre-corrected path that is as equal as possible to the curvature of the post-corrected path 702. The user can select any range for the curvature calculation range 705. By selecting the curvature from the pre-corrected path that has already been visualized, it becomes possible to easily draw a continuous and smooth post-corrected path.
[0062] These two methods may be combined. For example, the curvature may be changed arbitrarily from the UI based on the route that minimizes the route difference 704.
[0063] Next, a method for manually correcting the position of a route from the UI will be described with reference to Figs. 11 to 13. When executing this method, the display unit 110 displays a third user interface for specifying at least one of the size, position, and shape of the corrected route. Then, the correction unit 1031 generates a corrected route having the size, position, and shape specified by the third user interface. A specific example of this method will be described.
[0064] Fig. 11 to Fig. 13 are diagrams for explaining manual position correction executed on a UI according to an embodiment. Fig. 11 shows a pre-correction path 801 and a post-correction path 802. Fig. 12 shows a pre-correction path and a post-correction path 803. Fig. 13 shows a pre-correction path and a post-correction path 804. Three methods for correcting a pre-correction path from the UI will be described.
[0065] The first method is to change the size of the path on the panoramic image without changing the aspect ratio of the path on the panoramic image, as shown in FIG. 11. When executing this method, the correction unit 1031 generates a corrected path having an aspect ratio equal to the aspect ratio of the path before correction. The second method is to change the aspect ratio of the path on the panoramic image, as shown in FIG. 12. The third method is to move the path by vector without changing the size, as shown in FIG. 13. Any of these methods can be combined, and the user can correct the path before correction in a method with excellent usability by operating the operation unit 111 while looking at the display unit 110. Since the pan and tilt information at any position on the panoramic image is known, the processing unit 103 can convert the corrected path into pan driving information and tilt driving information based on the path displayed on the display unit 110.
[0066] By resizing the pre-correction path from the UI without changing the aspect ratio, it is possible to correct the pre-correction path without changing the pan and tilt directions of the imaging device 100. Furthermore, by resizing while changing the aspect ratio, even if the input pre-correction path is an unintended path, it is possible to easily adjust the pre-correction path from the GUI without re-inputting movement information of the imaging device from the operation unit 111. Furthermore, by correcting the pre-correction path only by vector movement, it is possible to correct the pre-correction path without changing the pan and tilt directions of the imaging device 100 even if the initial position of the imaging device 100 is shifted.
[0067] Next, a method for correcting the drive speed and a method for displaying the drive speed on the path of the UI will be described with reference to FIG. 14 to FIG. 18. When executing this display method, the display unit 110 displays a fourth user interface for specifying a parameter of the image capture angle of view of the image capture device 100. Then, the drive control unit 104 moves the image capture angle of view of the image capture device 100 based on the parameter stored in the storage unit 107 or the parameter specified by the fourth user interface. In addition, the parameter stored in the storage unit 107 may be the distance from the image capture device to the subject. In addition, the display unit 110 may change the color of the corrected path or the width of the line indicating the corrected path according to the speed of the image capture angle of view of the image capture device 100. Furthermore, the display unit 110 may display a parameter of the speed of the image capture angle of view of the image capture device 100 according to the speed of the image capture angle of view of the image capture device 100.
[0068] Fig. 14 is a diagram showing an example of the positional relationship between the imaging device and the subject when the imaging system according to the embodiment corrects the speed of the imaging angle of view of the imaging device. Fig. 15 to Fig. 18 are diagrams showing examples of a method for correcting the speed of the imaging angle of view of the imaging device according to the embodiment and a method for displaying the speed of the imaging angle of view of the imaging device. Two methods for correcting the drive speed will be described.
[0069] The first method is to keep the drive speed constant. There are several ways to determine the drive speed, such as by allowing the drive speed displayed on the display unit 110 to be specified from the UI unit 109, or by unifying the drive speed to an average speed of the input drive speeds, but this is not limited as long as the drive speed can be corrected to a constant speed. The method of correcting the drive speed to a constant speed is effective when the distance to the subject can be considered to be a constant distance. In addition, correcting the drive speed so that the drive speed at the start point and the end point are the same is effective because it results in continuous drive during loop operation and allows natural video to be acquired.
[0070] The second method is to change the drive speed according to a parameter. For example, in a case where the imaging device 901 captures an image of a subject 902 located close to the imaging device and an object 903 located far from the imaging device, it is assumed that the drive speed on the path 904 is corrected according to the distance to the subject. If the drive speed is changed according to the distance from the imaging device 901 to the subject 902 and the subject 903, the subject can be imaged at the same size and the same pan and tilt speed regardless of the distance from the imaging device. In order to keep the size of the subject constant, the zoom ratio is appropriately changed according to the distance. However, if the pan and tilt drive speed is constant regardless of the zoom ratio, the pan and tilt drive speed for the zoom ratio becomes faster as the zoom ratio becomes smaller, which may make the image difficult to see. Therefore, if the drive speed is corrected to be faster as the distance from the imaging device 901 to the subject is shorter and slower as the distance to the subject is longer, the image becomes easier to see. There are other methods, such as a method that allows the drive speed displayed on the display unit 110 to be changed from the operation unit 111, but this is not limited to this.
[0071] Next, a method of displaying the drive speed parameter on the UI will be described with reference to Figs. 15 to 18. The first method is to express the speed by color, as in the case of the route 1001 shown in Fig. 15. The second method is to express the drive speed by line width, as in the case of the route 1002 shown in Fig. 16. The third method is to express the speed by a display marker, as in the case of the route 1003 shown in Fig. 17. The fourth method is to display the drive speed parameter of a selected location, as in the case of the route 1004 shown in Fig. 18. In this method, the display unit 110 displays a first user interface indicating the speed of the imaging angle of view of the imaging device 100. The speed parameter may be the actual pan and tilt drive speed, or may be the drive speed on the panoramic image. Although four examples have been given here, the display method is not limited to these as long as the drive speed can be discriminated.
[0072] By changing the method of displaying the route depending on the speed, the visibility of the drive speed on the route on the display unit is improved, making it easier to judge the appropriateness of the drive speed.
[0073] The imaging system according to the embodiment has been described above. However, the imaging system according to the embodiment is not limited to the above. For example, the correction unit 1031 may receive data specifying points through which the corrected path should pass, and generate a corrected path that passes through the points. Also, for example, the drive control unit 104 moves the imaging angle of view of the imaging device 100 so that at least one of the speed and acceleration in the corrected path displayed superimposed on the composite image is constant. Also, for example, the display unit 110 may simultaneously display a plurality of corrected paths.
[0074] The present invention includes the following inventions that appropriately combine the above-mentioned features.
[0075] (Configuration 1) an imaging system comprising: an operation means for receiving movement information specifying a path of an imaging angle of view of an imaging device and moving the imaging angle of view of the imaging device; a display means for displaying a pre-correction path, which is the path of the imaging angle of view of the imaging device specified by the movement information, and a corrected path, which is the path of the imaging angle of view of the imaging device generated by correcting the pre-correction path; and a correction means for generating the corrected path by correcting the pre-correction path based on at least one of a position, a direction, and a speed of the imaging angle of view of the imaging device included in the movement information.
[0076] (Configuration 2) The imaging system according to configuration 1, characterized in that the correction means generates the corrected path in which the position of an imaging angle of view at a start point of the corrected path coincides with the position of an imaging angle of view at an end point of the corrected path.
[0077] (Configuration 3) The imaging system according to configuration 1 or 2, characterized in that the correction means generates the corrected path in which a direction of an imaging angle of view at a start point of the corrected path coincides with a direction of an imaging angle of view at an end point of the corrected path.
[0078] (Configuration 4) The imaging system according to any one of configurations 1 to 3, wherein the correction means generates the corrected path in which a speed of an imaging angle of view at a start point of the corrected path matches a speed of an imaging angle of view at an end point of the corrected path.
[0079] (Configuration 5) 5. The imaging system according to any one of configurations 1 to 4, wherein the correction means generates the corrected path having an aspect ratio equal to an aspect ratio of the uncorrected path.
[0080] (Configuration 6) The imaging system according to any one of configurations 1 to 5, wherein the correction means receives data specifying points through which the corrected path should pass, and generates the corrected path that passes through the points.
[0081] (Configuration 7) The imaging system according to any one of configurations 1 to 6, characterized in that the display means displays a composite image generated by synthesizing a plurality of images captured while an imaging angle of view of the imaging device is moving based on the movement information, and displays at least one of the pre-correction path and the post-correction path by superimposing it on the composite image.
[0082] (Configuration 8) The imaging system according to configuration 7, further comprising a drive control means for moving an imaging angle of view of the imaging device so that at least one of a speed and an acceleration in the corrected path displayed superimposed on the composite image is constant.
[0083] (Configuration 9) 9. The imaging system according to any one of configurations 1 to 8, wherein the display means further displays a first user interface indicating a speed of an imaging angle of view of the imaging device.
[0084] (Configuration 10) 10. The imaging system according to any one of configurations 1 to 9, wherein the display means displays a plurality of the corrected paths simultaneously.
[0085] (Configuration 11) 11. The imaging system according to any one of configurations 1 to 10, wherein a path of an imaging angle of view of the imaging device is a locus of a center of a range in which imaging by the imaging device can be performed.
[0086] (Configuration 12) The imaging system according to any one of configurations 1 to 11, characterized in that the path of the imaging angle of view of the imaging device is a trajectory of a line or surface that combines points on any coordinate system within a range that can be captured by the imaging device.
[0087] (Configuration 13) The imaging system described in any one of configurations 1 to 12, characterized in that the display means further displays a second user interface for specifying a range of the pre-correction path to be corrected, and the correction means generates the corrected path by correcting the range of the pre-correction path specified by the second user interface.
[0088] (Configuration 14) 11. The imaging system according to configuration 10, wherein the correction means generates the corrected path by correcting the shortest uncorrected path among the plurality of uncorrected paths.
[0089] (Configuration 15) The imaging system according to any one of configurations 1 to 14, wherein the correction means generates the corrected path based on a curvature of the uncorrected path.
[0090] (Configuration 16) The imaging system described in any one of configurations 1 to 15, characterized in that the display means further displays a third user interface for specifying at least one of a size, position, and shape of the corrected path, and the correction means generates the corrected path having the size, position, and shape specified by the third user interface.
[0091] (Configuration 17) The imaging system according to any one of configurations 1 to 16, characterized in that the display means further displays a fourth user interface for specifying parameters of an imaging angle of view of the imaging device, and further comprises a drive control means for moving the imaging angle of view of the imaging device based on parameters stored in a storage means or parameters specified by the fourth user interface.
[0092] (Configuration 18) 18. The imaging system according to configuration 17, wherein the parameter stored in the storage means is a distance from the imaging device to a subject.
[0093] (Configuration 19) The imaging system according to configuration 4, wherein the display means changes a color of the corrected path or a width of a line indicating the corrected path in accordance with a speed of an imaging angle of view of the imaging device.
[0094] (Configuration 20) 5. The imaging system according to configuration 4, wherein the display means displays a parameter of a speed of the imaging angle of view of the imaging device in accordance with the speed of the imaging angle of view of the imaging device.
[0095] (Method 1) a pre-correction path, which is the path of an imaging angle of view of an imaging device specified by movement information that specifies a path of the imaging angle of view of the imaging device, and a post-correction path, which is the path of the imaging angle of view of the imaging device generated by correcting the pre-correction path, and a method for generating the post-correction path by correcting the pre-correction path based on at least one of a position, a direction, and a speed of the imaging angle of view of the imaging device that is included in the movement information.
[0096] (Program 1) a pre-correction path, which is the path of an imaging angle of view of an imaging device specified by movement information that specifies a path of the imaging angle of view of the imaging device, and a post-correction path, which is the path of the imaging angle of view of the imaging device generated by correcting the pre-correction path, and a program for generating the post-correction path by correcting the pre-correction path based on at least one of a position, a direction, and a speed of the imaging angle of view of the imaging device that is included in the movement information.
[0097] <Other embodiments> The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a recording medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit for implementing one or more of the functions, such as an ASIC (Application Specific Integrated Circuit).
[0098] The above describes preferred embodiments of the present invention. However, the present invention is not limited to the above-described embodiments. In other words, the present invention includes embodiments in which various modifications have been made based on the gist of the present invention, and these embodiments are not excluded from the scope of the present invention. [Explanation of symbols]
[0099] 100 Imaging device 101 Imaging unit 102 Lens drive unit 103 Processing section 104 Drive control unit 105 Tilt drive unit 106 Pan drive unit 107 Storage section 108 Interface section 109 User Interface Section 110 Display section 111 Operation section
Claims
1. A display means for displaying a pre-correction path which is the path of the imaging angle of the imaging device specified by movement information that specifies the path of the imaging angle of the imaging device, and a post-correction path which is the path of the imaging angle of the imaging device generated by correcting the pre-correction path, Correction means for generating the corrected path by correcting the pre-correction path based on at least one of the position, direction, and speed of the imaging angle of the imaging device included in the movement information, An imaging system characterized by comprising the following features.
2. The correction means generates a corrected path in which the position of the imaging field of view at the starting point of the corrected path coincides with the position of the imaging field of view at the ending point of the corrected path. The imaging system according to feature 1.
3. The correction means generates a corrected path in which the direction of the imaging angle of view at the starting point of the corrected path coincides with the direction of the imaging angle of view at the ending point of the corrected path. The imaging system according to feature 1.
4. The correction means generates a corrected path in which the velocity of the imaging angle at the starting point of the corrected path matches the velocity of the imaging angle at the ending point of the corrected path. The imaging system according to feature 1.
5. The correction means generates the corrected path having an aspect ratio equal to the aspect ratio of the path before correction. The imaging system according to any one of claims 1 to 4.
6. The correction means receives data specifying points that the corrected path should pass through, and generates the corrected path that passes through those points. The imaging system according to any one of claims 1 to 4.
7. The display means displays a composite image generated by combining multiple images captured while the imaging angle of the imaging device is moving based on the movement information, and displays at least one of the pre-correction path and the post-correction path superimposed on the composite image. The imaging system according to any one of claims 1 to 4.
8. The system further includes a drive control means for moving the imaging angle of the imaging device so that at least one of the velocity and acceleration in the corrected path, which is superimposed on the composite image, remains constant. The imaging system according to feature 7.
9. The display means further displays a first user interface indicating the speed of the imaging angle of the imaging device. The imaging system according to any one of claims 1 to 4.
10. The display means simultaneously displays a plurality of the corrected paths. The imaging system according to any one of claims 1 to 4.
11. The path of the imaging angle of the imaging device is the trajectory of the center of the range that can be imaged by the imaging device. The imaging system according to any one of claims 1 to 4.
12. The path of the imaging angle of the imaging device is the trajectory of a line or plane formed by combining points on arbitrary coordinates within the range in which imaging is possible by the imaging device. The imaging system according to any one of claims 1 to 4.
13. The display means further displays a second user interface for specifying the range to be corrected from the pre-correction path, The correction means generates the corrected path by correcting the range specified by the second user interface from the pre-correction path. The imaging system according to any one of claims 1 to 4.
14. The correction means generates the corrected path by correcting the shortest of the multiple pre-correction paths. The imaging system according to feature 10.
15. The correction means generates the corrected path based on the curvature of the path before correction. The imaging system according to any one of claims 1 to 4.
16. The display means further displays a third user interface for specifying at least one of the size, position, and shape of the corrected path. The correction means generates the corrected path having the size, position and shape specified by the third user interface. The imaging system according to any one of claims 1 to 4.
17. The display means further displays a fourth user interface for specifying the parameters of the imaging angle of view of the imaging device. The system further comprises drive control means for moving the imaging angle of view of the imaging device based on parameters stored by storage means or parameters specified by the fourth user interface. The imaging system according to any one of claims 1 to 4.
18. The parameter stored by the storage means is the distance from the imaging device to the subject. The imaging system according to feature 17.
19. The display means changes the color of the corrected path or the width of the line indicating the corrected path according to the speed of the imaging angle of the imaging device. The imaging system according to feature 4.
20. The display means displays the parameters of the imaging angle speed of the imaging device according to the imaging angle speed of the imaging device. The imaging system according to feature 4.
21. The imaging system according to claim 1, further comprising an operating means for receiving the movement information and moving the imaging angle of the imaging device.
22. The display means displays the pre-correction path, which is the path of the imaging angle of the imaging device specified by movement information that specifies the path of the imaging angle of the imaging device, and the post-correction path, which is the path of the imaging angle of the imaging device generated by correcting the pre-correction path. The corrected path is generated by correcting the pre-correction path based on at least one of the position, direction, and speed of the imaging angle of the imaging device included in the movement information. A method characterized by the following:
23. The display means displays the pre-correction path, which is the path of the imaging angle of the imaging device specified by movement information that specifies the path of the imaging angle of the imaging device, and the post-correction path, which is the path of the imaging angle of the imaging device generated by correcting the pre-correction path. The corrected path is generated by correcting the pre-correction path based on at least one of the position, direction, and speed of the imaging angle of the imaging device included in the movement information. A program characterized by the following features.