Imaging apparatus and setting method for imaging apparatus

Abstract

To provide a technique for easily setting, to an imaging apparatus, a range in which imaging suitable for avoiding reflection of an unintended person can be performed.SOLUTION: An imaging apparatus comprises: an imaging section which can change an imaging direction; a reception section which receives information for determining a first range that is a range in which a target is retrieved; a detection section which successively fetches images in the first range while changing the imaging direction of the imaging section and detects the target existing within the first range from the fetched images; and a determination section in which, in a case where a plurality of targets are detected from the inside of the first range, a user is notified of information representing detection results of the plurality of targets, information on one or more selected targets selected from among the plurality of targets by the user is received from the user, and a second range that is a range in which imaging can be performed by the imaging section is determined based on detection positions of the selected targets in such a manner that the selected targets are out of an angle of view of the imaging section.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present invention relates to an imaging device and a setting method for the imaging device. [Background technology]

[0002] In recent years, pan-tilt-zoom cameras that allow remote control of the camera's pan (movement of the imaging direction horizontally), tilt (movement of the imaging direction up and down), and zoom have been used in video production. Furthermore, systems that use multiple pan-tilt-zoom cameras for imaging are also known. By arranging multiple cameras, including a main camera and multiple sub-cameras that are different from the main camera, it becomes possible to obtain multiple imaging angles. In such systems, there is a desire to reduce the number of cameramen operating the cameras in order to reduce imaging costs.

[0003] In some cases, a technology is used to automatically track a subject detected from a captured image, and the pan, tilt, and zoom of the camera are automatically controlled so that the subject is located at a desired position within the camera's angle of view.

[0004] However, because multiple cameras are automatically controlled and operated, situations can occur where one camera unintentionally appears in the video being shot by the other camera. In particular, in situations where this control is expected to be used, such as live music concerts and weddings, where it is difficult to reshoot the video, it is important to eliminate objects that obstruct the content of the video.

[0005] Patent Document 1 proposes a technique for controlling the pan / tilt direction or zoom of a camera based on the relative positions of the cameras so that other cameras do not enter the imaging space of the camera itself.It also proposes a technique for notifying an operator of the proximity of other cameras to prevent switching to an image that includes other cameras or for urging the operator to move the camera.

[0006] Patent Document 2 proposes a technology for setting a fixed visible range that allows pan-tilt-zoom for each of multiple pan-tilt-zoom cameras. If the visible range of each camera can be set appropriately so that each camera does not capture the other cameras' images, it is possible to prevent images from being captured. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2023-67619 A [Patent Document 2] JP 2020-025248 A Summary of the Invention [Problem to be solved by the invention]

[0008] When the object reflected can be limited to the camera, it is relatively easy to set a fixed visible range based on the positions of surrounding cameras, as in the method proposed in Patent Document 2. However, in situations where many cameras work together to capture the same space, such as live music concerts and sports broadcasts, or where multiple subjects are frequently switched, unintended reflections of not only cameras but also various objects (including people) are expected. When such various reflections are expected, it is difficult to uniquely determine what is appropriate as the standard for setting the visible range. Therefore, it is difficult to automatically set the visible range of each camera.

[0009] Furthermore, Patent Document 1 employs a method of automatically limiting the visible range to avoid reflections. However, the method of Patent Document 1 may limit pan / tilt / zoom or notify the user not only when an object that the user wants to avoid being reflected in the image is nearby, but also when an object that the user allows to be reflected in the image is nearby. Excessively narrowing the visible range of the imaging device in this way is undesirable, as it may impede imaging operations or reduce usability.

[0010] The present invention has been made in consideration of the above-mentioned circumstances, and has an object to provide a technique for easily setting an appropriate capture range for an imaging device in order to avoid unintended objects being captured. [Means for solving the problem]

[0011] The present disclosure includes an imaging device having an imaging unit capable of changing an imaging direction; a receiving unit that receives information for determining a first range that is a range for searching for an object; a detection unit that sequentially captures images of the first range while changing the imaging direction of the imaging unit and detects objects present within the first range from the captured images; and a determination unit that, when a plurality of objects are detected within the first range, notifies a user of information representing the detection results of each of the plurality of objects, receives from the user information of one or more selected objects selected by the user from the plurality of objects, and determines a second range that is a range that can be imaged by the imaging unit based on the detection position of the selected object so that the selected object is outside the angle of view of the imaging unit. Effect of the Invention

[0012] According to the above configuration, it is possible to easily set an appropriate capture range for the imaging device in order to avoid unintended objects being captured. [Brief description of the drawings]

[0013] [Figure 1] Camera control system configuration diagram [Diagram 2] Camera function block diagram [Diagram 3] A diagram showing an example of a multi-camera control system. [Figure 4] Internal configuration of the detection processing unit [Diagram 5] A diagram showing the relationship between the movable range and the visible range [Figure 6] FIG. 13 is a diagram showing an example of the difference in the angle of view depending on the zoom setting value of a camera. [Figure 7]FIG. 1 shows an example of scanning for screening a scanning range. [Figure 8] FIG. 13 is a diagram showing an example of the detected position of an object and the position coordinates of its end portion. [Figure 9] FIG. 13 is a diagram showing an example of detection result information notified to a user; [Figure 10] Flowchart showing the process of the detection processing unit [Figure 11] A flowchart showing a process for setting a visible range according to a user's selection. [Figure 12] A diagram showing the UI used for user notification and selection processing [Figure 13] FIG. 13 is a diagram showing an example of a visible range edge selection result by a user; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] A first embodiment of the present invention will be described below with reference to FIGS.

[0015] 1 shows a configuration diagram of a camera control system in this embodiment. The camera control system includes an image capture device 100, a control device 110, a display device 120, and an input device 130. The image capture device 100 and the control device 110 are connected to each other via a network 140 so as to be able to communicate with each other. The camera control system may include a plurality of image capture devices 100. In that case, the system is also called a multi-camera control system.

[0016] The imaging device 100 (hereinafter, referred to as camera) is a network camera that can be remotely controlled via a network 140. The camera 100 is a so-called PTZ camera equipped with a pan-tilt-zoom mechanism. Panning is an operation of rotating the camera 100 in the horizontal direction to move the imaging direction to the right or left. Tilt is an operation of rotating the camera 100 in the vertical direction to move the imaging direction up or down. Zooming is an operation of changing the imaging magnification of the camera 100. The camera 100 can perform panning, tilting, and zooming operations by receiving instructions for panning, tilting, and zooming from the control device 110 via the network 140. The video captured by the camera 100 is transmitted to the control device 110 via the network 140.

[0017] The control device 110 is a computer (arithmetic processing device) that controls the imaging device 100 and processes, switches, and outputs images. The control device 110 includes an image output unit 111 and an external input receiving unit 112. The image output unit 111 outputs images transmitted from the camera 100 via a network 140. The external input receiving unit 112 receives input from an input device 130. The control device 110 generates pan / tilt / zoom drive instructions in response to the received input, and transmits the instructions to the camera 100 via the network 140.

[0018] The display device 120 displays the video output from the control device 110 via the video output unit 111. The display device 120 is configured, for example, as a liquid crystal display, an organic EL display, etc. Although the control device 110 and the display device 120 are shown as separate entities in Fig. 1, the control device 110 may be configured, for example, as a computer integrated with a display.

[0019] The input device 130 is a device through which a user inputs instructions and operations to the camera 100 or the control device 110. Instructions and operations by the input device 130 are input to the control device 110 via the external input receiving unit 112. The input device 130 is, for example, a mouse, a trackball, a touchpad, a keyboard, or the like. The input device 130 may not be a general-purpose device, but may be a dedicated controller for operating the camera 100. Also, a touch panel display serving as both the display device 120 and the input device 130 may be used.

[0020] The network 140 connecting the camera 100 and the control device 110 may be the Internet, a LAN, or a WAN, or may be serial communication such as RS-232C or USB. The connection may be wired or wireless. The video captured by the camera 100 may be transmitted using SDI or HDMI (registered trademark).

[0021] 1 is composed of one camera 100 and one control device 110, the camera control system may have multiple cameras 100 and may have multiple control devices 110. For example, the system may be configured so that one camera 100 can be controlled by any of multiple control devices 110, or so that one control device 110 can control multiple cameras 100.

[0022] 2 shows a functional block diagram of a camera which is an imaging device in this embodiment. The camera 100 mainly comprises an imaging section 201, an image processing section 202, a system control section 203, and a control command processing section 204. The camera 100 also has a pan driving section 205, a tilt driving section 206, a pan / tilt control section 207, a lens driving section 208, a lens control section 209, a storage section 210, a communication section 211, and a detection processing section 212.

[0023] The imaging unit 201 is composed of an imaging lens and an imaging element, and is used for imaging a subject and for electrically The image processing unit 202 performs image processing such as noise removal and gamma correction on the electric signal converted by the imaging unit 201 to generate image data, and transmits the image data to the system control unit 203. Note that the imaging lens may be a detachable interchangeable lens.

[0024] The system control unit 203 is composed of one or more processors such as a CPU or MPU, and controls the entire camera 100 by executing a program stored in the storage unit 210. The system control unit 203 also controls the transmission of generated image data to the communication unit 211. The system control unit 203 also instructs the image processing unit 202 on imaging parameters such as image quality adjustment settings, and the imaging unit 201 performs adjustment control based on the transmitted instructions. The system control unit 203 also performs processing to write setting values ​​such as image quality adjustment parameters and video output settings, settings related to pan / tilt control, and the like into the storage unit 210.

[0025] Furthermore, the system control unit 203 includes an internal control command processing unit 204. The control command processing unit 204 interprets pan / tilt / zoom control instruction commands received by the communication unit 211 from the control device 110 via the network 140, and issues instructions to a pan / tilt control unit 207 and a lens control unit 209.

[0026] The pan driving unit 205 is composed of a mechanical driving system that performs a panning operation, a motor that is the driving source of the mechanical driving system, and an angle sensor that detects the angle (azimuth angle) of the panning direction. The mechanical driving system rotates when the motor is driven, and the imaging direction (azimuth angle) of the imaging unit 201 can be changed. The tilt driving unit 206 is composed of a mechanical driving system that performs a tilting operation, a motor that is the driving source of the mechanical driving system, and an angle sensor that detects the angle (elevation angle) of the tilting direction. The mechanical driving system rotates when the motor is driven, and the imaging direction (elevation angle) of the imaging unit 201 can be changed. The pan / tilt control unit 207 controls the pan driving unit 205 and the tilt driving unit 206 based on an instruction signal transmitted from the system control unit 203.

[0027] The lens driving unit 208 is composed of a mechanical driving system for the focus lens and zoom lens, a motor as a driving source thereof, and a position sensor that detects the position of the driving unit, and its operation is controlled by a lens control unit 209. The lens control unit 209 controls the lens driving unit 208 based on an instruction signal transmitted from the system control unit 203.

[0028] The storage unit 210 is a memory device that non-temporarily stores programs and setting values ​​used by the system control unit 203. The system control unit 203 reads and writes data in the storage unit 210.

[0029] The communication unit 211 communicates with the control device 110 via the network 140 , transmits received communication data to the system control unit 203 , and performs communication processing to transmit communication data transmitted from the system control unit 203 to the control device 110 .

[0030] The detection processing unit 212 is a processing block that provides the camera 100 with a function of setting a visible range in order to prevent an unintended object from being captured in the image of the camera 100 (an unintended object from being included in the angle of view of the camera 100). The "visible range" refers to a range that can be captured by the imaging unit 201 of the camera 100 (a range in which the camera 100 is permitted to capture images: a second range). First, the detection processing unit 212 (accepting unit) accepts information for determining a scanning range via the communication unit 211 or directly from the user. The "scanning range" is a range (a first range) in which a search for an object that is not desired to be captured is performed. The detection processing unit 212 (detecting unit) detects an object present within the scanning range from images that are sequentially captured while changing the imaging direction (pan, tilt) of the imaging unit 201. Hereinafter, the process of scanning the entire scanning range and searching (detecting) an object is called "screening" or simply "detection process". When multiple objects are detected within the scanning range, the detection processing unit 212 automatically The visible range is not set, and information representing the detection results of each of a plurality of objects is notified to the user, and the user is prompted to select "an object whose reflection is to be avoided." In a situation where many objects exist around the camera 100, it is highly likely that it is difficult to automatically determine which of the objects corresponds to "an object whose reflection is to be avoided" and which corresponds to "an object whose reflection is to be allowed" as intended by the user. Then, when the detection processing unit 212 (determination unit) receives information on one or more selected objects selected by the user from the user, it determines the visible range based on the detection position of the selected object so that the selected object is out of the angle of view of the imaging unit 201. According to such a method of setting the visible range, even in a situation where many objects exist around the camera 100, it is possible to easily set an appropriate visible range for avoiding the reflection of an object unintended by the user. If there are very few objects detected within the scanning range, the detection processing unit 212 (determination unit) may automatically determine the visible range so that the detected object is out of the angle of view of the imaging unit 201. The number of detection objects that is used as the threshold for whether to query the user / automatically determine the visible range may be set arbitrarily. In this embodiment, the threshold is set to "1" as described below (i.e., the user is queried if there are two or more detection objects), but the threshold may be a value of, for example, about 1 to 5. Alternatively, the user may be able to set the threshold. Note that if no object is detected within the scanning range, there is no need to limit (set) the visible range of the imaging unit 201.

[0031] The configuration and processing contents of the detection processing unit 212 will be described in more detail later. Note that, in this embodiment, the detection processing unit 212 is implemented as a processing block independent of the system control unit 203, but is not limited to this configuration, and the detection processing unit 212 may be implemented as one function of the system control unit 203. In the latter case, the function of the detection processing unit 212 is realized by the processor of the system control unit 203 executing a program.

[0032] FIG. 3 shows an example of a configuration in which multiple cameras operate in cooperation with each other in the camera control system according to this embodiment. In the example of FIG. 3, three cameras (imaging devices) 100A, 100B, and 100C and one control device 110 are connected via a network 140 and operate in cooperation with each other. A camera control system in which multiple cameras cooperate to perform imaging operations in this way is also called a multi-camera control system. Note that the number of cameras is not limited to three, and may be two, or four or more. Also, multiple control devices 110 may be connected to the network 140, and the multiple control devices 110 may cooperate to control one or multiple cameras 100.

[0033] 4 shows an example of a detailed configuration of the detection processing unit included in the camera 100 according to this embodiment. The detection processing unit 212 has a scanning range receiving unit 401, a detection scanning processing unit 402, an end calculation unit 403, an end setting unit 404, and a notification unit 405.

[0034] The scanning range receiving unit 401 (receiving unit) receives information for determining the scanning range and determines the scanning range based on the received information. The scanning range receiving unit 401 may receive information for determining the zoom magnification range together with the information for determining the scanning range, and determine the scanning range and the zoom magnification range based on both pieces of information.

[0035] The detector scanning processing unit 402 (detector unit) detects an object from images of the scanning range sequentially captured from the imaging unit 201. The detector scanning processing unit 402 may use any object detection algorithm or object recognition algorithm. For example, it may be an algorithm using deep learning (CNN, R-CNN, Fast R-CNN, Faster R-CNN, YOLO, SSD, etc.), or it may be a classical algorithm (template matching, SVM, etc.).

[0036] The detection and scanning processing unit 402 may execute the object detection process by using its own detection function (i.e., by using the computational resources of the camera 100). 2 may utilize a detection function provided by another device (for example, a computational resource such as a control device 110 or a cloud server connected via a network 140). Alternatively, the detection scanning processing unit 402 may estimate an object present within the scanning range of its own camera 100 based on information (detection information) of an object detected by another camera (image of another camera) constituting a multi-camera control system as shown in FIG.

[0037] The detector scanning processor 402 may be designed to detect all kinds of objects (including people; the same applies below) present within the scanning range by so-called general object recognition, or may be designed to detect only predetermined types of objects. Alternatively, the detector scanning processor 402 may be designed to detect only specific objects or people registered in advance, or may be designed to be able to change the object to be detected according to detection object conditions set by the user or the control device 110. The object to be detected by the detector scanning processor 402 can be designed or set arbitrarily according to the application of the system, and for example, if the application is to film a live music concert, other cameras, people (such as film staff), backstage equipment, etc. may be set as the object.

[0038] The notification unit 405 has a function of inquiring of the user which of the multiple objects corresponds to the "object to avoid reflection" when the number of objects detected within the scanning range exceeds a threshold value. Specifically, the notification unit 405 notifies the user by transmitting information indicating the detection result of each of the multiple objects to the control device 110, and receives information of one or more selected objects selected by the user from the multiple objects from the user.

[0039] Based on the detected position of the "object to be avoided from being reflected" (for example, information on a bounding box surrounding the object), the edge calculation unit 403 calculates the edge positions of the visible range so that the object does not enter the angle of view of the imaging unit 201. The edge setting unit 404 sets the visible range of the imaging unit 201 in accordance with the edge positions calculated by the edge calculation unit 403. In this embodiment, the notification unit 405, the edge calculation unit 403, and the edge setting unit 404 constitute a determination unit of the present invention.

[0040] FIG. 5 shows the relationship between the movable range and the visible range. The range (movable range) in which the imaging direction (orientation) of the camera 100 can be physically changed is shown as the movable range 500. In the example of FIG. 5, the movable range in the pan direction (horizontal direction) is 170° left and right from the center point (neutral position) 510, and the movable range in the tilt direction (vertical direction) is -30° downward to 90° upward from the center point 510. A visible range 501 can be set for the camera 100 as a range in which an image can be captured as compared to the physical movable range 500. The visible range 501 is the same as or narrower than the movable range 500. When the visible range 501 is set as shown in FIG. 5, the angle of view 520 of the camera 100 is controlled by the pan / tilt control unit 207 and the lens control unit 209 so as to drive only within a range that does not go beyond the visible range 501. The visible range 501 is defined by the ends in each of the four directions, that is, a tilt upper end 511, a tilt lower end 513, a pan right end 512, and a pan left end 514. The end setting unit 404 updates the ends 511 to 514 in each of the four directions that constitute the visible range 501.

[0041] FIG. 6 shows an example of the difference in the angle of view depending on the zoom setting value of the camera. FIG. 6 shows the image capture angle of view from camera 100A toward subject A. Assume that camera 100B is installed above subject A and suspended from the ceiling. The image capture angle of camera 100A changes according to the zoom setting, and becomes a WIDE image capture angle of view 601 at the minimum allowable zoom setting, and becomes a TELE image capture angle of view 602 at the maximum allowable zoom setting. Even if the camera is in the same imaging position, the ceiling-mounted camera 100B is not reflected when capturing an image at the TELE image capture angle of view 602. However, the ceiling-mounted camera 100B is reflected when capturing an image at the WIDE image capture angle of view 601.

[0042] In this way, the angle of view of the camera 100 changes depending on the zoom setting value. It is preferable that the zoom setting value of the camera 100 be taken into account when the image forming apparatus 2 screens the scanning range and when the detection processing unit 212 determines the visible range 501.

[0043] If no particular constraint is imposed, the minimum and maximum values ​​of the zoom magnification are determined by the specifications (performance) of the camera 100. However, when screening the scanning range, it is not appropriate to use the minimum zoom magnification or maximum zoom magnification at the performance limit of the camera 100. When the minimum zoom magnification is used, the angle of view becomes wider, and the object captured in the image becomes smaller. Therefore, there is an increased possibility that the detection processing unit 212 (detection scanning processing unit 402) cannot detect the object from the image. On the other hand, when the maximum zoom magnification is used, the angle of view becomes narrow, and only a part of the object is captured in the image, and the detection processing unit 212 (detection scanning processing unit 402) may fail to detect the object. Therefore, when screening the scanning range or determining the visible range 501, it is preferable to limit the range of the zoom magnification to some extent.

[0044] For example, in the multi-camera control system shown in FIG. 3, it is assumed that the sub-cameras 100A and 100C operate in a manner that follows the main camera 100B. The subjects to be imaged in this case are subjects A to C on the stage, and the pan / tilt range and zoom magnification range required for the main camera 100B are set in advance in the control device 110 based on the movement range of the subjects A to C and their relative positions with the sub-cameras 100A and 100C. This set value, that is, the pan / tilt range and zoom magnification range that the camera 100B can actually take, is called the "effective range." In the case of a multi-camera control system, information on the effective range of the pan / tilt and zoom magnification set in the control device 110 is notified from the control device 110 to the detection processing unit 212 (scanning range receiving unit 401) of the camera 100B. The detection processing unit 212 (detection scanning processing unit 402) may determine the scanning range and the visible range based on the information on the effective range of the pan / tilt and zoom magnification.

[0045] In this embodiment, the detection processing unit 212 determines the scanning range at the WIDE imaging angle of view 601 that corresponds to the minimum zoom value in the effective range. Since the scanning range can be set to the widest by using the WIDE imaging angle of view 601 as a reference, it is possible to prevent the target from being overlooked in screening. When the difference between the maximum and minimum zoom values ​​set as the effective range is large, the determination of the scanning range and screening may be performed using the median value of the zoom magnification. When the zoom magnification is large, the range that can be imaged at one time becomes narrow, so there is a disadvantage that the time required to screen the scanning range increases. On the other hand, when the zoom magnification is small, the risk of a decrease in detection accuracy due to the object appearing in the image becoming smaller increases. Therefore, by using the median value of the zoom magnification, it is possible to perform processing that balances the screening time and detection accuracy. When the median value of the zoom magnification is used, a position with a predetermined margin from the edge of the detection position of the target may be set as the edge of the visible range. This makes it possible to respond to changes in the imaging angle of view due to zooming out, etc.

[0046] FIG. 7 shows an example of scanning for screening the scanning range. The rectangular area indicated by the reference numeral 700 in FIG. 7 is the scanning range determined based on the information on the effective range of pan / tilt and the effective range of the zoom magnification received by the scanning range receiving unit 401. The rectangular area indicated by the reference numeral 701 is the angle of view of the imaging unit 201 calculated based on the zoom magnification. Based on the information on the scanning range 700 and the angle of view 701 of the imaging unit 201, the system control unit 203 of the camera 100 makes a plan on what route to move the imaging direction of the imaging unit 201 in order to capture images of the entire scanning range 700 without omission. In this embodiment, as shown by the arrow in FIG. 7, the angle of view 701 is slid from the upper left of the scanning range 700 to the lower right in sequence, thereby scanning the entire scanning range 700.

[0047] The detection and scanning processing unit 402 detects an object to be prevented from being reflected from the images sequentially captured by the imaging unit 201. As described above, the object detection algorithm and the object to be detected are The object is appropriately designed according to the purpose and configuration of the system. FIG. 7 shows an example in which another camera 100B is detected from an image at the right end of the scanning range 700. It can be seen that if the camera 100 pans to the right beyond the coordinate position of the left end 703 of the detection position (bounding box) 702 of the other camera 100B, the other camera 100B will be reflected in the angle of view 701 of the camera 100 itself. When an object is detected from the scanning range 700, the detection result may be notified to the user, and the user may select an object that should not be reflected. As described above, when the number of detected objects is equal to or less than a threshold (for example, equal to or less than one), the visible range of the camera 100 may be automatically set based on the detection position 702.

[0048] FIG. 8 shows an example of the detection position of an object and the position coordinates of its ends. In the example of FIG. 8, the result of detecting the object A as the object is shown, and the dashed rectangle (bounding box) represents the detection position of the object A. When the position coordinates of the upper end 801 of the detection position are expressed by the angle in the pan direction (azimuth angle) and the angle in the tilt direction (elevation angle), it becomes (80°, 60° left). Similarly, the right end 802 is (30°, 45° left), the lower end 803 is (80°, 15° left), and the left end 804 is (99°, 45° left). In order to prevent the object A from being reflected in the image, in terms of the tilt direction, the lower end of the visible range should be set above the upper end 801 of the detection position of the object A, or the upper end of the visible range should be set below the lower end 803 of the detection position of the object A. Regarding the panning direction, the left end of the visible range may be set to the right of the right end 802 of the detection position of subject A, or the right end of the visible range may be set to the left of the left end 804 of the detection position of subject A.

[0049] Fig. 9 shows an example of detection result information notified to the user. The detection result information includes a "detected object ID" assigned to each detected object (hereinafter also referred to as a "detected object"), a "name" of the detected object that is displayed as an auxiliary when notifying the user of the detection result information, and information on the four end points of the detection result of the detected object. The detection result information shown on the left side of Fig. 9 represents the detection result of subject A in Fig. 8, and includes the detected object ID <h001>, name <Person A>, top end <left 80, 60>, bottom end <left 80, 15>, right end <left 30, 45>, left end <left 99, 45>. If multiple objects are detected within the scanning range, the detection results for the number of objects detected are notified to the user. In the example of Figure 9, the detected object ID <h001>In addition to person A, detected object ID <c001>The camera C is detected. Note that the configuration of the detection result information shown in FIG. 9 is merely an example. The information representing the detection result may include at least information representing the detection position, and preferably information representing the type of the detected object. Furthermore, it is also preferable that the information representing the detection result includes a thumbnail image of the detected object cut out from the image in which the detected object is detected. By displaying the detection result information together with the thumbnail image when notifying the user of the detection result information, it is possible to assist the user in making a selection.

[0050] An example of the flow of reflection detection processing by detection processing unit 212 will be described with reference to Fig. 10. Note that the processing of this flowchart starts in a state of waiting for reception of a reflection detection start instruction from control device 110.

[0051] First, in step S1001, camera 100 receives an instruction to start reflection detection from a user via control device 110. When camera 100 receives the instruction to start detection, it notifies detection processing unit 212 that the instruction has been received, and processing from step S1002 onwards is started.

[0052] In step S1002, when the control device 110 notifies the effective range of pan / tilt and the effective range of zoom magnification, the scanning range receiving unit 401 determines the angle of view (reference numeral 701 in FIG. 7) and the scanning range (reference numeral 700 in FIG. 7) during scanning. Furthermore, the scanning range receiving unit 401 determines the screening route of the scanning range.

[0053] Next, in step S1003, when the detection target conditions are notified from the control device 110, the conditions of the target object to be detected in the reflection detection process performed by the detection scanning processing unit 402 are set. For example, if it is desired to set other cameras existing around the camera 100 as the detection target, it is preferable to set a condition to that effect. Note that, if the detection target conditions are not notified from the control device 110, all objects that the detection scanning processing unit 402 can detect may be set as the detection target.

[0054] In step S1004, an image is captured from the imaging unit 201 to the detector scanning processing unit 402, which performs processing to detect an object from the captured image. In step S1005, the detector scanning processing unit 402 determines whether an object has been detected. If an object has been detected, the process proceeds to step S1006. If an object has not been detected from the image, the process proceeds to step S1007. In step S1006, the detector scanning processing unit 402 stores the detection result (information on the object and the detection position shown in FIG. 9) in the memory unit 210 in the camera 100. Then, the process proceeds to step S1007.

[0055] In step S1007, the imaging direction of the imaging unit 201 is changed according to the route set in step S1002, and then the process returns to step S1004. If scanning of the entire scanning range is completed, the process exits from the loop process of steps S1004 to S1007 and proceeds to step S1008.

[0056] In the next step S1008, the notification unit 405 determines the number of detection targets detected within the scanning range. If the number of detected targets is equal to or less than the threshold (one or less in this embodiment), the process proceeds to step S1009. If the number of detected targets exceeds the threshold (two or more in this embodiment), the process proceeds to step S1010.

[0057] In step S1009, edge calculation unit 403 calculates the edge positions of the visible range based on the detection position of the detection target so that the detection target does not fall within the angle of view of imaging unit 201. Then, edge setting unit 404 automatically sets the visible range in accordance with the edge positions calculated by edge calculation unit 403. For example, in the example of Fig. 7, the right edge of the visible range may be set to the left of left edge 703 of detection position 702 so that detection position 702 of camera 100B is excluded. Note that when the number of detections is 0 (when there is no detection target within the scanning range), there is no need to limit the visible range, and so the visible range is set to match the scanning range.

[0058] In step S1010, notification unit 405 collectively notifies the user of the detection results (see FIG. 9) accumulated in storage unit 210 of camera 100.

[0059] Then, the process proceeds to step S1011, and the reflection detection process is terminated. In this embodiment, the edge of the visible range is automatically updated when the number of detection targets is small, but by setting the threshold to 0, the visible range may be automatically set only when there is no detection target, and the process may proceed to step S1010 when there is a detection target. In this way, when a detection target is found, the selection is left to the user regardless of the number of detection targets. In addition, the processes of steps S1008 to S1010 may be performed separately for the pan direction and the tilt direction. That is, even if multiple detection targets are found, if the positions of the multiple detection targets in the pan direction are all the same, the number of detection targets in the pan direction may be considered to be 1, and the left end or right end of the visible range may be automatically set. Similarly, even if multiple detection targets are found, if the positions of the multiple detection targets in the tilt direction are all the same, the number of detection targets in the tilt direction may be considered to be 1, and the upper end or lower end of the visible range may be automatically set.

[0060] Next, the flow of processing for setting the visible range according to the user's selection will be described with reference to the flowchart of Fig. 11. Note that the processing of this flowchart corresponds to step S1010 of Fig. 10. After the camera 100 notifies the control device 110 of the detection result in FIG. 9, the control device 110 starts waiting for an instruction to start the visible range setting process.

[0061] First, in step S1101, the camera 100 receives an instruction to start visible range setting processing from the user via the control device 110. When the camera 100 receives an instruction to start visible range setting processing, it notifies the detection processing unit 212 that the instruction has been received, and the process proceeds to step S1102 to start this process.

[0062] In step S1102, the camera 100 waits for notification of the visible range end selection result from the user. When the notification is received, the process proceeds to the subsequent step S1103. Unless the selection result is notified, there is no need to update the visible range. A timeout process may be provided in which the waiting for acceptance in step S1102 is cancelled when a predetermined time (for example, several minutes to several tens of minutes) has elapsed, and the process shown in FIG. 11 is terminated.

[0063] Next, in step S1103, the visible range end selection result notified by the user is notified to the detection processing unit 212. After that, the process proceeds to step S1104, where the end calculation unit 403 and the end setting unit 404 start updating the end position of the visible range. An example of the configuration of the information of the visible range end selection result is shown in FIG. 13. The visible range end selection result includes an "end setting type" and two sets of "end direction and type of detection target". Note that the configuration of the visible range end selection result shown in FIG. 13 is only an example. For example, it may be configured with an "end setting type" and one set of "end direction and type of detection target", or it may be configured to combine information on both the pan direction and the tilt direction. Incidentally, the example of FIG. 13 shows a user selection result in which the "right end" of the "pan direction" of the visible range is set based on the detection position of the detection target <camera 1>, and the "upper end" of the <tilt direction> of the visible range is set based on the detection position of the detection target <person 1>.

[0064] In step S1104, the edge calculation unit 403 determines whether the information on the visible range edge selection result received from the user includes information on the pan direction. If the information on the pan direction is included, it is determined that the user has an intention to update the edge position of the visible range in the pan direction, and the process proceeds to step S1105 to perform a visible range edge update process in the pan direction. If the information on the pan direction is not included, the process proceeds to step S1107.

[0065] In step S1105, edge calculation unit 403 starts a calculation process of the edge position to be set as the right edge in the pan direction based on the information of the visible range edge selection result. The calculation process method follows the procedure in Fig. 7. Only when the visible range edge selection result shown in Fig. 13 contains information on the <right edge> of the <pan direction>, edge calculation unit 403 determines the right edge of the visible range based on the left edge of the detection position of the corresponding detection target. Note that if the visible range edge selection result does not contain information on the <right edge> of the <pan direction>, the process of step S1105 is skipped, and the right edge of the visible range is excluded from the update process target.

[0066] In step S1106, the edge calculation unit 403 starts a process of calculating the edge position to be set as the left edge in the pan direction based on the information of the visible range edge selection result. If the visible range edge selection result includes information on the <left edge> in the <pan direction>, the edge calculation unit 403 determines the left edge of the visible range based on the right edge of the detection position of the corresponding detection target. In the case of the visible range edge selection result shown in Fig. 13, information on the <left edge> in the <pan direction> is not included, so the left edge of the visible range is excluded from the update process.

[0067] In step S1107, the edge calculation unit 403 determines whether the information on the visible range edge selection result received from the user includes tilt direction information. If tilt direction information is included, it is determined that the user has an intention to update the tilt direction edge position of the visible range, and the process proceeds to step S1108 to perform visible range edge update processing in the tilt direction. If tilt direction information is not included, the process proceeds to step S1110.

[0068] In step S1108, edge calculation unit 403 first starts a calculation process of the edge position to be set as the upper end in the tilt direction based on the information of the visible range edge selection result. The calculation process method follows the procedure in Fig. 7. Only when the visible range edge selection result shown in Fig. 13 contains information of the <upper end> ​​of the <tilt direction>, edge calculation unit 403 determines the upper end of the visible range based on the lower end of the detection position of the corresponding detection object. Note that if the visible range edge selection result does not contain information of the <upper end> ​​of the <tilt direction>, the process of step S1108 is skipped, and the upper end of the visible range is excluded from the update process target.

[0069] In step S1109, edge calculation unit 403 starts a process of calculating an edge position to be set as the bottom edge in the tilt direction based on the information of the visible range edge selection result. If the visible range edge selection result includes information on the <bottom edge> of the <tilt direction>, edge calculation unit 403 determines the bottom edge of the visible range based on the top edge of the detection position of the corresponding detection object. In the case of the visible range edge selection result shown in Fig. 13, information on the <bottom edge> of the <tilt direction> is not included, so the bottom edge of the visible range is excluded from the update process.

[0070] In step S1110, edge setting unit 404 performs processing to update the edge positions of the visible range only for those edges for which edge position information was calculated in steps S1108 to S1109. Then, the process proceeds to step S1111, where the visible range setting processing is completed, the process transitions to a steady state, and the process ends.

[0071] Fig. 12 shows an example of a user interface used in a process of notifying a user of detected objects and having the user select an object to be avoided from among them (an object to be used for setting the edge position of the visible range). The example in Fig. 12 shows the detection result in the pan direction when the scanning range 700 shown in Fig. 7 is screened.

[0072] The screen 1201 on the left side is a UI for selecting a detection target to be the right end of the pan direction. Reference numerals 1202 to 1205 indicate detection targets detected in an area to the right of the origin (0°). Thumbnail images of each detection target are displayed together with the angle of the pan direction. This UI makes it easy to grasp what kind of object is present at what angle to the right of the origin (0°). In addition, the screen 1201 displays "70° to the right", which is the current right end setting value of the visible range, as <current right end>. According to this screen 1201, it is easy to confirm and determine the location of an object whose reflection should be avoided, and further the appropriateness of the current visible range. For example, if it is desired to avoid the reflection of another camera, it is understood that the right end of the visible range should be set to about "60° to the right". Note that the user's operation is, for example, to simply select an object (thumbnail image) whose reflection should be avoided from among the detection targets 1202 to 1205 displayed on the screen 1201. Therefore, the end of the visible range can be specified with a very simple operation.

[0073] The screen 1206 on the right side is a UI for selecting a detection target to be the left end of the pan direction. Reference numerals 1207 and 1208 indicate detection targets detected in the area to the left of the origin (0°). By using such a screen 1206, the end of the visible range can be set with a simple operation, just like the screen 1201.

[0074] Although not shown, a UI for selecting the detection target to be the top end of the tilt direction and a UI for selecting the detection target to be the bottom end of the tilt direction are similar to the screen shown in FIG. 12 (the difference is that thumbnail images of the detection targets are arranged vertically).

[0075] FIG. 13 shows a state where a user selects a detection target to be set at the end of the visible range using the control device 110. 12 shows an example of information on the visible range end selection result notified to the camera 100 based on the selection result when the user selects the visible range end. The visible range end selection result is composed of "end setting type", "end direction", and "detection target type". There are two types of "end setting type", namely <pan direction> or <tilt direction>. When the <pan direction> is set, the "end direction" has two elements, namely <right end> ​​and <left end>. When the <tilt direction> is set, the "end direction" has two elements, namely <upper end> ​​and <lower end>. The "end direction" has as an element "detection target type", which is information for identifying the object (selected object) selected by the user as the detection target for which the user wants to avoid being reflected. The "detection target type" may be the "detection target ID" or "name" in FIG. 9. When the user selects the detection target 1204 on the screen 1021 in FIG. 12, the "end setting type" = pan direction, the "end direction" = right end, and the "detection target type" is set to the detection target ID or name corresponding to the detection target 1204. If the left end of the pan direction is not selected on screen 1206, nothing is set in the element "type of detection target" in the left end direction.

[0076] As described above, according to the configuration of the imaging device of the present embodiment described in Fig. 1 to Fig. 13, even if it is difficult to automatically and uniquely determine the validity of what should be adopted as a criterion for selecting the visible range, the user can easily determine the visible range to be set. For example, in an environment where multiple PTZ cameras work in cooperation with each other, where there are multiple surrounding cameras that may be in close proximity, even if it is difficult to select the visible range to avoid proximity, the risk of excessively narrowing the visible range can be avoided. In addition, in a multi-camera control system in which multiple cameras work in cooperation with each other, it is also possible to automatically set the cameras and the imaging targets as detection targets when selecting the visible range based on information about the cameras and imaging targets belonging to the system, and then execute the reflection detection process.

[0077] <Other embodiments> The above-described embodiments merely show specific examples of the present invention. The scope of the present invention is not limited to the configurations of the above-described embodiments, and various embodiments can be adopted without changing the gist of the present invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and gist of the invention.

[0078] The disclosed technology can be embodied as, for example, a system, an apparatus, a method, a program, or a recording medium (storage medium), etc. Specifically, the technology may be applied to a system composed of multiple devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or may be applied to an apparatus composed of a single device.

[0079] Needless to say, the object of the present invention can be achieved by the following: That is, a recording medium (or storage medium) on which is recorded a program code (computer program) of software that realizes the functions of the above-mentioned embodiments is supplied to a system or device. Needless to say, such a storage medium is a computer-readable storage medium. Then, a computer (or a CPU or MPU) of the system or device reads and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the functions of the above-mentioned embodiments, and the recording medium on which the program code is recorded constitutes the present invention.

[0080] The disclosure of this specification includes the following configurations, methods, and programs.

[0081] (Configuration 1) An imaging unit capable of changing an imaging direction; a reception unit that receives information for determining a first range in which a search for an object is performed; a detection unit that sequentially captures images of the first range while changing an imaging direction of the imaging unit, and detects an object present within the first range from the captured images; a determination unit that, when a plurality of objects are detected within the first range, notifies a user of information representing the detection results of each of the plurality of objects, receives from the user information of one or more selected objects selected by the user from the plurality of objects, and determines a second range that is a range that can be captured by the imaging unit based on the detection positions of the selected objects so that the selected objects are out of the angle of view of the imaging unit; An imaging device having the above configuration.

[0082] (Configuration 2) When one object is detected within the first range, the determination unit automatically determines the second range based on a detection position of the one object so that the one object is out of the angle of view of the imaging unit. 2. The imaging device according to configuration 1.

[0083] (Configuration 3) The determination unit receives, as information on the selection object, at least one of information on a first selection object for determining a right end in a pan direction and information on a second selection object for determining a left end in a pan direction from the user, determines the right end of the second range based on a detection position of the first selection object, and determines the left end of the second range based on the detection position of the second selection object. 3. The imaging device according to configuration 1 or 2.

[0084] (Configuration 4) the determination unit receives, as information on the selection object from the user, at least one of information on a third selection object for determining an upper end of a tilt direction and information on a fourth selection object for determining a lower end of a tilt direction, determines an upper end of the second range based on a detection position of the third selection object, and determines a lower end of the second range based on the detection position of the fourth selection object. The imaging device according to any one of configurations 1 to 3.

[0085] (Configuration 5) the imaging device is one of cameras constituting a multi-camera control system in which a plurality of cameras cooperate to perform imaging operations, The object detected by the detection unit includes a camera other than the imaging device among the plurality of cameras constituting the multi-camera control system. The imaging device according to any one of configurations 1 to 4.

[0086] (Configuration 6) The object detected by the detection unit includes a person. The imaging device according to any one of configurations 1 to 5.

[0087] (Configuration 7) The object detected by the detection unit is a specific object or a specific person that is registered in advance. The imaging device according to any one of configurations 1 to 6.

[0088] (Configuration 8) The detection unit is capable of changing the object to be detected according to information received from a user or another device. The imaging device according to any one of configurations 1 to 7.

[0089] (Configuration 9) The detection unit detects an object present within the first range from the image by using a detection function that the detection unit has or a detection function provided by another device. The imaging device according to any one of configurations 1 to 8.

[0090] (Configuration 10) The detection unit acquires detection information by the other camera, and detects an object present within the first range based on the detection information. The imaging device according to configuration 5.

[0091] (Configuration 11) the determination unit determines a right end of the second range based on a left end of a detection position of the first selection object, and determines a left end of the second range based on the right end of a detection position of the second selection object. 4. The imaging device according to configuration 3.

[0092] (Configuration 12) the determination unit determines a position a predetermined margin away from a left end of the detection position of the first selection object as the right end of the second range, and determines a position a predetermined margin away from a right end of the detection position of the second selection object as the left end of the second range. 12. The imaging device according to configuration 11.

[0093] (Configuration 13) the determination unit determines an upper end of the second range based on a lower end of a detection position of the third selection object, and determines a lower end of the second range based on an upper end of a detection position of the fourth selection object. 5. The imaging device according to configuration 4.

[0094] (Configuration 14) the determination unit determines a position at a predetermined margin from a lower end of the detection position of the third selection object as an upper end of the second range, and determines a position at a predetermined margin from an upper end of the detection position of the fourth selection object as a lower end of the second range. 14. The imaging device according to claim 13.

[0095] (Method 15) 1. A method for setting an imaging device, the method being executed by an imaging device including an imaging unit capable of changing an imaging direction or a control device connected to the imaging device via a network, the method comprising: Receiving information for determining a first range in which a search for an object is performed; detecting an object present within the first range from images of the first range that are sequentially captured while changing an imaging direction of the imaging unit; when a plurality of objects are detected within the first range, notifying a user of information representing the detection results of each of the plurality of objects, receiving from the user information of one or more selected objects selected by the user from the plurality of objects, and determining a second range that is a range that can be imaged by the imaging unit based on the detection positions of the selected objects so that the selected objects are out of the angle of view of the imaging unit; A setting method having the following features:

[0096] (Program 16) An imaging device having an imaging unit capable of changing an imaging direction, or the imaging device is connected to the imaging device via a network. A processor of a control device connected to the Receiving information for determining a first range in which a search for an object is performed; detecting an object present within the first range from images of the first range that are sequentially captured while changing an imaging direction of the imaging unit; when a plurality of objects are detected within the first range, notifying a user of information representing the detection results of each of the plurality of objects, receiving from the user information of one or more selected objects selected by the user from the plurality of objects, and determining a second range that is a range that can be imaged by the imaging unit based on the detection positions of the selected objects so that the selected objects are out of the angle of view of the imaging unit; A program for executing. [Explanation of symbols]

[0097] 100 Cameras 110 Control device 212 Detection processing unit

Claims

1. An imaging unit capable of changing an imaging direction; a reception unit that receives information for determining a first range in which a search for an object is performed; a detection unit that sequentially captures images of the first range while changing an imaging direction of the imaging unit, and detects an object present within the first range from the captured images; a determination unit that, when a plurality of objects are detected within the first range, notifies a user of information representing the detection results of each of the plurality of objects, receives from the user information of one or more selected objects selected by the user from the plurality of objects, and determines a second range that is a range that can be captured by the imaging unit based on the detection positions of the selected objects so that the selected objects are out of the angle of view of the imaging unit; An imaging device having the above configuration.

2. When one object is detected within the first range, the determination unit automatically determines the second range based on a detection position of the one object so that the one object is out of an angle of view of the imaging unit. The imaging device according to claim 1 .

3. the determination unit receives, as information on the selection object from the user, at least any one of information on a first selection object for determining a right end in a pan direction and information on a second selection object for determining a left end in a pan direction, determines the right end of the second range based on a detection position of the first selection object, and determines the left end of the second range based on the detection position of the second selection object; The imaging device according to claim 1 .

4. the determination unit receives, as information on the selection objects from the user, at least one of information on a third selection object for determining an upper end of a tilt direction and information on a fourth selection object for determining a lower end of a tilt direction, determines an upper end of the second range based on a detected position of the third selection object, and determines a lower end of the second range based on a detected position of the fourth selection object. The imaging device according to claim 1 .

5. the imaging device is one of cameras constituting a multi-camera control system in which a plurality of cameras cooperate to perform imaging operations, The object detected by the detection unit includes a camera other than the imaging device among the plurality of cameras constituting the multi-camera control system. The imaging device according to claim 1 .

6. The object detected by the detection unit includes a person. The imaging device according to claim 1 .

7. The object detected by the detection unit is a specific object or a specific person that is registered in advance. The imaging device according to claim 1 .

8. The detection unit is capable of changing the object to be detected according to information received from a user or another device. The imaging device according to claim 1 .

9. The detection unit detects an object by using a detection function that the detection unit has or a detection function provided by another device. Detecting an object present within the first range from the image using the The imaging device according to claim 1 .

10. The detection unit acquires detection information by the other camera, and detects an object present within the first range based on the detection information. The imaging device according to claim 5 .

11. the determination unit determines a right end of the second range based on a left end of a detection position of the first selection object, and determines a left end of the second range based on the right end of a detection position of the second selection object. The imaging device according to claim 3 .

12. the determination unit determines a position a predetermined margin away from a left end of the detection position of the first selection object as the right end of the second range, and determines a position a predetermined margin away from a right end of the detection position of the second selection object as the left end of the second range. The imaging device according to claim 11.

13. the determination unit determines an upper end of the second range based on a lower end of a detection position of the third selection object, and determines a lower end of the second range based on an upper end of a detection position of the fourth selection object. The imaging device according to claim 4.

14. the determination unit determines a position at a predetermined margin from a lower end of the detection position of the third selection object as an upper end of the second range, and determines a position at a predetermined margin from an upper end of the detection position of the fourth selection object as a lower end of the second range. The imaging device according to claim 13.

15. 1. A method for setting an imaging device, the method being executed by an imaging device including an imaging unit capable of changing an imaging direction or a control device connected to the imaging device via a network, the method comprising: Receiving information for determining a first range which is a range in which a search for an object is performed; detecting an object present within the first range from images of the first range that are sequentially captured while changing an imaging direction of the imaging unit; when a plurality of objects are detected within the first range, notifying a user of information representing the detection results of each of the plurality of objects, receiving from the user information of one or more selected objects selected by the user from the plurality of objects, and determining a second range that is a range that can be imaged by the imaging unit based on the detection positions of the selected objects so that the selected objects are out of the angle of view of the imaging unit; A setting method having the following features:

16. A processor included in an imaging device including an imaging unit capable of changing an imaging direction or a control device connected to the imaging device via a network, Receiving information for determining a first range which is a range in which a search for an object is performed; detecting an object present within the first range from images of the first range that are sequentially captured while changing an imaging direction of the imaging unit; when a plurality of objects are detected within the first range, notifying a user of information representing the detection results of each of the plurality of objects, receiving from the user information of one or more selected objects selected by the user from the plurality of objects, and determining a second range that is a range that can be imaged by the imaging unit based on the detection positions of the selected objects so that the selected objects are out of the angle of view of the imaging unit; A program for executing the above.

Images