Search control device, imaging apparatus, and search control method

Abstract

To detect a subject within a search range in a short time.SOLUTION: A search control device has search means 207, 208, 209 that perform a subject search using image data obtained by an imaging apparatus 100 and control means 209 for controlling the focal distance of the imaging apparatus. The control means controls the focal distance in accordance with a specified search distance and the size of a search target subject among a range of search distances for which a subject search can be performed.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a technique for subject search using image data obtained by imaging.

Background Art

[0002] Automatic imaging devices that perform imaging regularly and continuously without the user giving an imaging instruction have been proposed. For example, there is a life log camera that records the scenes that the user sees in daily life as videos at regular time intervals while being worn on the user's body by a strap or the like. There are also cameras that recognize sudden movements of people's faces such as smiles and animals such as dogs and cats and perform automatic imaging, and cameras that have a PTZ (Pan / Tilt / Zoom) control mechanism and perform automatic imaging while freely changing the imaging range.

[0003] Furthermore, cameras have been proposed in which the user can register and specify in advance a subject (priority subject) to be preferentially searched and imaged. In the search for a priority subject, it is preferable that a subject outside the angle of view can be searched or that the search can be performed in the distance direction from the camera to search for an appropriate priority subject among nearby and distant subjects.

[0004] Also, as methods for detecting a specific subject, a method using feature points extracted from an image and a method using pattern matching are generally used. In these subject detections, it is necessary to keep the size of the subject on the image within a certain range. For this reason, when detecting a subject from a distant subject to a nearby subject, it is necessary to change the focal length of the camera according to the change in distance, or cut out and enlarge a part of the image, and in each case, obtain an image and perform a detection process. As a result, it takes a long time to complete the detection process in the entire search range. Even if the distance of the subject to be imaged is determined and the detection process is performed in the entire search range, it takes time until the subject to be imaged is detected, and there is a risk of missing the opportunity to image.

[0005] Patent Document 1 discloses a method for detecting a subject in a short time by limiting the distance range for subject detection using distance distribution information for each region (partial image) in an image obtained by imaging. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Patent No. 3897087 [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] However, the method disclosed in Patent Document 1 cannot shorten the detection time for subjects outside the field of view in cameras that search for subjects in a wide search range while changing the orientation of the camera, such as PTZ cameras.

[0008] The present invention provides a search control device that can detect an object within the search range in a short amount of time. [Means for solving the problem]

[0009] An imaging device, as one aspect of the present invention, comprises a search means for performing subject search using image data obtained by the imaging device, and a control means for controlling the focal length of the imaging device. The control means is characterized by controlling the focal length according to a specified search distance and the size of the target subject within the range of search distances in which subject search is possible. An imaging device comprising an imaging unit for imaging a subject and the above-mentioned search control device also constitutes another aspect of the present invention.

[0010] Furthermore, another aspect of the present invention is a control method for performing subject search, which includes the steps of obtaining image data from an imaging device and controlling the focal length of the imaging device. The method is characterized by controlling the focal length according to a specified search distance and the size of the target subject, within the range of search distances in which subject search is possible. A program that causes a computer to execute processing according to the above control method also constitutes another aspect of the present invention. [Effects of the Invention]

[0011] According to the present invention, subjects within the search range can be detected in a short amount of time. [Brief explanation of the drawing]

[0012] [Figure 1] A diagram showing the configuration and rotation axis of the imaging device in Examples 1 and 2. [Figure 2] A block diagram showing the configuration of the imaging device in Examples 1 and 2. [Figure 3] A diagram showing the relationship between the imaging device and the subject in Examples 1 and 2. [Figure 4] A flowchart showing the search sequence in Example 1. [Figure 5] A block diagram showing the configuration of the image processing apparatus in Examples 1 and 2. [Figure 6] A diagram showing the search distance setting UI in Examples 1 and 2. [Figure 7] A diagram illustrating user notifications regarding priority distance settings in Examples 1 and 2. [Figure 8] A flowchart showing the search sequence in Example 2. [Figure 9] A flowchart showing the execution sequence for automated imaging in Examples 1 and 2. [Figure 10] A diagram showing the search distance and search time according to the specified distance in Example 2. [Figure 11] A diagram showing the P / T drive amount and P / T search range according to the search distance in Example 2. [Modes for carrying out the invention]

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment

[0014] In Embodiment 1, an automatic imaging camera (imaging device) that limits the search distance for a subject to be imaged will be described.

[0015] The camera of this embodiment is a digital camera capable of PTZ control. This camera is used in various imaging situations, such as being installed on a dining table to image people during meals or being installed in a park to image children playing from a distance, such as over a bench. At this time, the distance between the camera and the subject to be imaged (hereinafter referred to as the specific subject) varies depending on the imaging situation. When the camera searches the entire imaging range without omission, searching in areas where the specific subject does not exist may result in missing the imaging opportunity for the specific subject.

[0016] Therefore, the camera of this embodiment allows the user to specify the search distance. The camera calculates a focal length suitable for recognizing a face at the specified distance based on the specified search distance (hereinafter referred to as the specified distance) and the average size of a human face, which is the target subject for search. During subject search, imaging and subject determination are performed using that focal length. Thereby, automatic imaging can be performed without missing the imaging chance for the subject at the position intended by the user.

[0017] FIG. 1(A) shows the camera 100 of this embodiment. The camera 100 includes an imaging unit 101 as an imaging means, a tilt rotation unit 102 that drives the imaging unit 101 in the tilt direction, a pan rotation unit 103 that drives the imaging unit 101 in the pan direction, and a main body box 104 that houses a control unit for controlling imaging and the like. The tilt rotation unit 102 and the pan rotation unit 103 function as direction changing means for changing the imaging direction of the imaging means.

[0018] Figure 2 shows the internal configuration of the camera 100. The imaging unit 101 includes a lens unit 201 as an imaging optical system and an imaging section 202 which includes an image sensor that performs photoelectric conversion (imaging) of the subject image formed by the light from the lens unit 201. The imaging section 202 is also provided with an A / D conversion unit that converts the analog imaging signal output from the image sensor into a digital imaging signal (imaging data). The imaging unit 101 is held by the main body box 104 via a tilt rotation unit 102 and a pan rotation unit 103.

[0019] The tilt rotation unit 102 includes a tilt motor as an actuator and a rotation mechanism that uses the tilt motor to rotate the imaging unit 101 in the pitch direction shown in Figure 1(B). The pan rotation unit 103 includes a pan motor as an actuator and a rotation mechanism that uses the pan motor to rotate the imaging unit 101 in the pan direction shown in Figure 1(B). The pitch direction is the rotation direction around the horizontal axis (X-axis) of the imaging device 100, and the pan direction is the rotation direction around the vertical axis (Y-axis) of the imaging device 100.

[0020] As shown in Figure 2, the main box 104 houses a control unit 207 that controls the zoom and focus operations of the lens unit 201 within the imaging unit 101, controls imaging by the imaging unit 202, and controls the driving of the tilt and pan rotation units 102 and 103. As a control means, the control unit 207 controls the operation of the lens unit 201 via the lens drive unit 205 and controls the driving of the tilt and pan rotation units 102 and 103 via the lens barrel rotation drive unit 206.

[0021] Furthermore, the main unit box 104 houses an image processing unit 203, a storage unit 204, a search distance setting unit 208, a subject detection unit 209, a subject information recording unit 210, and a communication unit 211. The image processing unit 203 performs image processing such as distortion correction, white balance adjustment, and color interpolation on the imaging data output from the imaging unit 202 to generate image data. The generated image data is output to the storage unit 204 and the subject detection unit 209.

[0022] The memory unit 204 converts the image data generated by the image processing unit 203 into a recording format such as JPEG or MPEG, and records it on a recording medium such as non-volatile memory. The memory unit 204 also records information about the captured subject (hereinafter referred to as subject information) as past imaging information, associated with the recorded image data.

[0023] The lens drive unit 205 has a motor and driver that drive the zoom lens and focus lens included in the lens unit 201, and drives each lens based on the target position and drive speed input from the control unit 207. The lens barrel rotation drive unit 206 drives the tilt and pan rotation units 102 and 103 based on the target position and drive speed input from the control unit 207, and rotates the imaging unit 101 in the tilt and pan directions. Here, the target position and drive speed are specified as the input to the lens barrel rotation drive unit 206 and the output to the control unit 207, but acceleration or time-division data to the target position may also be specified.

[0024] In addition to controlling the operations and drives described above, the control unit 207 executes sequences for automatic imaging that do not require user intervention. Specifically, it executes a search sequence to detect the subject to be imaged, a tracking sequence to ensure that the subject detected in the search sequence is not lost, and an imaging sequence to adjust the image composition and control the shutter timing. The search sequence will be described later.

[0025] In the tracking and imaging sequences, the next subject to be imaged is determined based on subject information input from the subject detection unit 209, and the target positions for lens drive and lens barrel rotation drive necessary to keep the subject within the imaging angle of view of the imaging unit 101 and obtain an appropriate imaging composition are determined. The determined target position information is output to the lens drive unit 205 and the lens barrel rotation drive unit 206. In addition, instructions to start and end imaging are output to the storage unit 204 at the appropriate timing.

[0026] The search distance setting unit 208 stores the specified distance input by the user. The specified distance is the distance from the camera 100, and is a specified search distance within the range of search distances in which a subject can be searched. The specified distance is specified by the user via the image processing device described later and input to the search distance setting unit 208 via the communication unit 211. In this embodiment, the specified distance is an abstract distance, and in this embodiment, it is one of "short distance," "medium distance," "long distance," or "automatic." The search distance setting unit 208 records the input specified distance and outputs the specified distance in response to a request from the control unit 207. In addition, as a measure to prevent the user from forgetting the setting, the search distance setting is changed back to the initial value after a predetermined time has elapsed since the specified distance was set or when use ends due to power cut-off.

[0027] The subject detection unit 209 has the functions of determining the orientation of the subject's face, detecting the position of the face, detecting the size of the face, registering the subject, and uniquely identifying the registered subject. The subject detection unit 209 extracts subject information (such as the size of the detected face, the position and orientation of the face in the image, and the identification result of the registered subject) from the face region of the image data input from the image processing unit 203, and outputs it to the control unit 207 and the subject information recording unit 210. The subject detection unit 209, the search distance setting unit 208, and the control unit 207 constitute the search means. Furthermore, this search means and the control unit 207, which acts as a control means, constitute the search control device.

[0028] However, there are limitations on the size of faces that can be detected in an image by the subject detection unit 209. For example, if a subject is captured at the edge or across the entire image and part of its face is missing, the face may not be accurately identified, or it may not even be recognized as a face at all. Also, when the captured image is a VGA (640 x 480 pixels) image, in order to accurately recognize it as a human face, it is desirable that the area from the top of the head to the chin fits within a certain size on the captured image, for example, between 160 and 360 pixels in the short side direction of the captured image.

[0029] The subject information recording unit 210 registers subjects based on the subject detection results input from the subject detection unit 209. The subject information recording unit 210 also records the name, birthday, and priority setting information for each subject, which are input from the communication unit 211 through means (user interface, etc.) for setting subject priorities for the user provided by the image processing device, which will be described later. Furthermore, the subject information recording unit 210 outputs the priority setting information corresponding to the subject to the control unit 207 according to the subject detection results input from the subject detection unit 209.

[0030] The communication unit 211 has the function of performing long-distance wireless communication and communicates with the image processing device described later via wireless communication. The communication unit 211 outputs the specified distance specified by the user to the search distance setting unit 208 via the image processing device and outputs the name, birthday, and priority setting information for each subject to the subject information recording unit 210.

[0031] The relationship between the imaging device 100 and the subject's position will be explained using Figures 3(A) and (B). The search distance x is the distance actually used for the search, and is a specific numerical value indicating the distance between the imaging device 100 and the search target subject 301. The search distance x is determined by the control unit 207 based on the specified distance set by the search distance setting unit 208.

[0032] As shown in Figure 3(A), let a be the actual size of the target subject 301 (hereinafter referred to as the actual size). The actual size a is a size corresponding to a representative value such as the average or median length from the top of the head to the tip of the chin of a human face, and here it is set to 23 cm. Also, as shown in Figure 3(B), let b be the size of a predetermined appropriate target subject on the imaging surface 300 of the image sensor (size on imaging: hereinafter referred to as the imaging size). As mentioned above, size b is set to be about 1 / 3 to 3 / 4 of the length of the shorter side of the imaging surface 300 (i.e., the captured image).

[0033] The image size b is determined according to the detection rate of the subject detection unit 209 and the number of subjects in the image. Alternatively, the image size b may be calculated based on the size of the target subject on the captured image and the optical parameters of the lens unit 201. The focal length f of the lens unit 201 can be calculated using x, a, and b as shown in equation (1) below. f = bx / a (1) The focal length f calculated by equation (1) is used by the control unit 207 when executing the search sequence.

[0034] The flowchart in Figure 9 illustrates the series of automatic imaging processes from subject search to image capture. The automatic imaging process is divided into an automatic search sequence, a search sequence, a tracking sequence, and an imaging sequence, and the control unit 207 executes these sequences as needed. The control unit 207, acting as a computer, executes this process according to a computer program.

[0035] The control unit 207 starts the automatic imaging process in response to the camera being powered on or waking from sleep mode. In step S901, the control unit 207 obtains the specified distance from the search distance setting unit 208 and determines whether "Automatic" or another distance is selected. If "Automatic" is selected, the automatic search sequence is executed in step S902. If "Short distance," "Medium distance," or "Long distance" is selected, the search sequence S903 is executed in step S903.

[0036] The search sequence (step S903), whose details will be described later, is a sequence in which the search distance is determined based on a specified distance, and the subject is searched for (detected) while scanning the search range as a spatial range by sequentially changing the pan / tilt angle. When the control unit 07 detects a subject by the search sequence, it stores subject information indicating whether the detected subject is a priority subject (specific subject) or a non-priority subject, and proceeds to step S904. A priority subject is the face of a specific person that is prioritized for search and imaging among the human faces that are the search target subjects.

[0037] The automatic search sequence (step S902) is a sequence that detects subjects by scanning the search area by sequentially changing the pan / tilt angle and zoom position without limiting the search distance. In the automatic search sequence, as in the search sequence, subject information indicating whether the detected subject is a priority subject or a non-priority subject is stored and the process proceeds to step S904.

[0038] In step S904, the control unit 207 determines from the subject information stored in step S902 or step S903 whether the subject detected in those steps is a priority subject or a non-priority subject. If a priority subject is detected, the process proceeds to step S905 to execute the tracking sequence; if a non-priority subject is detected, the process proceeds to step S906 to execute the imaging sequence.

[0039] The tracking sequence (step S905) is a sequence that tracks a subject by driving the pan / tilt / zoom to keep the subject, which is already within the imaging field of view, in focus. The control unit 207 calculates the pan / tilt / zoom drive amount to limit the subject to be tracked to one and keep that subject in the center of the imaging field of view (imaging image). Specifically, the control unit 207 calculates the pan / tilt drive amount to cancel out the positional difference between the center of the imaging image and the center of the subject. It also calculates the zoom drive amount to cancel out the difference between the actual subject size on the imaging screen and the ideal subject size. The ideal subject size may be the expected subject size b on the imaging surface 300 as explained in Figure 3, or a size smaller than size b may be used to prevent tracking failures for moving subjects.

[0040] Furthermore, during the tracking sequence, the control unit 207 determines whether or not to perform further imaging, and if imaging is to be performed, it proceeds to an imaging subsequence (not shown). The imaging subsequence performs the same processing as the imaging sequence in step S906, which will be described later, but when the imaging subsequence is completed, it returns to the tracking sequence and resumes tracking the subject. In this way, the tracking sequence is a sequence that performs tracking and imaging until the priority subject can no longer be tracked.

[0041] The imaging sequence (step S906) is a sequence for imaging the subject detected in the preceding sequence. In the imaging sequence, the control unit 207 adjusts the composition based on the subject information, performs imaging, and outputs the captured image to the storage unit 204 for storage on the recording medium. The composition adjustment is performed using pan / tilt / zoom drive amounts calculated based on the number, position, size, and face orientation of subjects on the captured image. Furthermore, composition adjustment and imaging may be performed not just once, but several times. In this case, variations in composition adjustment can be introduced to perform imaging with different compositions each time.

[0042] When the tracking sequence or imaging sequence S906 is completed, the control unit 207 determines in step S907 whether to terminate automatic imaging based on the number of images taken, power consumption, etc. If automatic imaging is to continue, the process returns to step S901; if automatic imaging is to be terminated, this process is terminated.

[0043] The flowchart in Figure 4 shows the details of the search sequence performed in step S903. As mentioned earlier, the search sequence is a sequence of searching for a subject based on a specified distance.

[0044] In step S401, which precedes the subject search, the control unit 207 first acquires the specified distance set by the search distance setting unit 208.

[0045] Next, in step S402, the control unit 207 determines the search distance based on the specified distance. The search distance corresponding to the specified distance is predetermined to be 1m for "short distance", 5m for "medium distance", and 10m for "long distance". In this case, the conversion from the specified distance to the search distance may be performed using a pre-prepared conversion table. Alternatively, the user may be able to specify the value of the search distance, or the conversion value to the search distance may be dynamically changed according to other settings.

[0046] Next, in step S403, the control unit 207 calculates the focal length by substituting the search distance and the size of the target subject (face) into equation (1), and in step S404, it zooms the lens unit 201 to obtain this focal length. This process makes it possible to image the subject at an ideal size for detecting the subject that is present at the search distance.

[0047] The next steps, S405-S407, are a detection cycle for comprehensively detecting subjects within the searchable distance range while periodically changing the orientation of the imaging unit 101 by pan / tilt drive. First, in step S405, the control unit 207 calculates the target orientation (pan / tilt angle) of the imaging unit 101, and in step S406, it performs pan / tilt drive so that the imaging unit 101 is in the target orientation. Then, in step S407, it detects subjects in the captured image after the pan / tilt drive. If no subjects are detected, the process returns to step S405 to execute the next detection cycle. If a subject is detected, as described above, subject information indicating whether the subject is a priority subject or a non-priority subject is stored, and this sequence ends.

[0048] The calculation of the pan / tilt angle in step S405 will be explained with an example. Hereafter, the pan / tilt angle will be written as (pan angle, tilt angle). Here, we will explain the case where no subject is detected, and each time the pan / tilt angle is calculated, step S405 and step SS406 are passed through and step S404 is executed again. In this embodiment, the search range is ±120deg for the pan angle and 0~90deg for the tilt angle, respectively.

[0049] First, the orientation of the imaging unit 101 (initial pan / tilt angle) is set to the minimum value of the search range, which is (-120deg, 0deg). Then, the pan angle is increased by 1dg at a time until the imaging unit 101 is panned to (120deg, 0deg).

[0050] When the pan angle reaches the upper limit of the search range, 120 degrees, the imaging unit 101 is returned to (-120 degrees, 0 degrees), and then the tilt angle is increased by 1 degree, with the imaging unit 101 tilted to (-120 degrees, 1 degree) as the pan angle is increased. After that, each time the pan angle reaches 120 degrees and is reset to -120 degrees, the tilt angle is increased by 1 degree. When the tilt angle reaches the upper limit of the search range, 90 degrees, the pan / tilt angle is returned to the initial (-120, 0). By performing subject detection while periodically changing the pan / tilt angle in this way, subjects within the search range can be comprehensively searched.

[0051] Figure 5 shows the configuration of an image processing device 500, which is provided separately from the camera 100. The image processing device 500 is composed of a personal computer, a smartphone terminal, or the like.

[0052] The image processing device 500 includes a control unit 506, a storage unit 503, a display unit 501, an operation unit 504, a power supply unit 505, and a communication unit 502. The control unit 506 controls the entire image processing device 500 according to a control program stored in the storage unit 503. The storage unit 503 stores various information such as the control program and parameters necessary for communication.

[0053] The power supply unit 505 supplies power to each part of the image processing device 500. The display unit 501 has a display device that displays information such as an LCD or LED, and a speaker capable of outputting sound. The operation unit 504 has operating members such as buttons that accept operations from the user. If the image processing device 500 is a smartphone terminal, the display unit 501 and the operation unit 504 are composed of a touch panel.

[0054] The image processing device 500 exchanges data and commands with the camera 100 via the communication unit 502. The data includes audio data, image data, compressed audio data, and compressed image data. The commands include requests for setting changes, data transmission, and command response requests from the image processing device 500 to the camera 100. Wireless LAN or Bluetooth Low Energy (BLE) (registered trademark), compliant with the IEEE 802.11 standard series, is used for communication.

[0055] The search control device, which is configured within the camera 100, may be located within the image processing device 500, or it may be located as a separate device from the camera 100 and the image processing device 500. In this case, the search control device acquires image data from the camera 100 (imaging unit 201) via communication.

[0056] Figure 6 shows a user interface (UI) for setting the search distance provided by the image processing device 500 through the display unit 501 (operation unit 504). This search distance setting UI is provided as part of an application for operating the camera 100. The search distance setting UI includes a priority distance setting button 601 to allow the user to choose whether or not to set a priority search distance (specified distance), and a distance selection bar 602 to allow the user to select the specified distance from "short distance," "medium distance," and "long distance." When the priority distance setting button 601 is set to OFF (disabled), the specified distance is set to "automatic." When the priority distance setting button 601 is set to ON (enabled), the specified distance can be selected using the distance selection bar 602. The information of the set specified distance is transmitted to the camera 100 via the communication unit 502. The camera 100 stores the received specified distance in the search distance setting unit 208.

[0057] Figures 7(A) and 7(B) show examples of notifications to the user displayed by the image processing device 500 according to the search distance setting. The image processing device 500 displays notifications 701 and 703 corresponding to notification commands issued by the camera 100 on the display unit 501 via the communication unit 502.

[0058] Figure 7(A) shows a notification 701 that informs the user that the search distance has been set (specified distance). This notification 701 informs the user in text 702 that the search distance (long distance in the figure) has been set. If the image processing device 500 is running a dedicated application, the notification 701 will be displayed within that application. If the image processing device 500 is not running a dedicated application, it will be displayed in the notification area provided by the operating system of the image processing device 500. For example, if the image processing device 500 is a smartphone terminal equipped with an operating system such as Android or iOS, it will be displayed in the notification area provided by that operating system. The same applies to notification 703, which will be described later.

[0059] Camera 100 issues a notification command immediately after the user changes the search distance setting, or immediately after the camera 100 is powered on or wakes from sleep mode, if the search distance setting is enabled by the user. The image processing device 500 may always display notification 701 if the search distance is enabled by the user.

[0060] Figure 7(B) shows a notification 703 prompting the user to review the search distance setting when a subject cannot be detected despite a predetermined time being spent searching for a subject at the search distance. Notification 703 includes text 704 informing the user that a subject could not be detected. If the camera 100 fails to detect a subject while the search distance is set to a value enabled by the user and the execution time of the search sequence exceeds a predetermined time (e.g., 10 minutes), it issues a notification command to the image processing device 500 to display notification 703.

[0061] These notifications may be output as sound from the speaker of the display unit 501 of the image processing device 500, or as images or sound from a display unit or sound output unit (not shown) provided on the camera 100.

[0062] According to this embodiment, in a camera 100 that comprehensively searches the search area while changing the orientation (imaging direction) of the imaging unit 101, subjects present in a wide search area can be detected in a short time.

[0063] In the above embodiment, the case in which the imaging direction is changed by a mechanical PTZ mechanism was described, but the imaging direction may also be changed electronically by cropping from the captured image.

[0064] Furthermore, although the above embodiment described a case where "Automatic" can be selected as one of the specified distances, it is also possible to always select the specified distance from "Short Distance," "Medium Distance," and "Long Distance." In this case, steps S901 and S902 in Figure 9 should be deleted so that the search sequence in step S903 is executed after the automatic imaging process has started.

[0065] Furthermore, in the above embodiment, a search distance setting UI having a priority distance setting button 601 and a distance selection bar 602 was described. Alternatively, a distance selection button such as a "long distance priority button" may be provided, or the specified distance may be switched alternately between "long distance" and "automatic" each time the distance selection button is operated. In addition, it may be possible to input a specific numerical value as the specified distance. Furthermore, it may be possible to select multiple specified distances.

[0066] Furthermore, the camera 100 and the image processing device 500 may communicate via a wired connection such as a USB connection. In addition, the camera 100 and the image processing device 500 may be configured as an integrated device, and communication may be performed internally via a bus or the like.

[0067] Furthermore, although the above embodiment described the detection of a human face as the subject, the detection target may also be a part of a person such as the eyes or hands, or the entire body. In addition, the detection target may be an animal such as a dog or cat, or a vehicle such as a car or airplane. [Examples]

[0068] Example 2 describes a search sequence that is an extension of the search sequence (step S903) described in Example 1. Example 1 described the case where only a specified distance specified by the user is searched. When the distance to the subject is fixed to some extent, such as when imaging a sports scene, the subject can be imaged by searching the specified distance using the search sequence of Example 1. However, when the distance to the subject changes over time or fluctuates greatly temporarily, such as when imaging in a large area like a park, the amount of movement of the subject in the distance direction is large, and simply searching only the specified distance may not be able to detect a subject that is moving a large amount. Example 2 improves the detection rate of the subject in such situations by prioritizing the search at the specified distance while also searching at other distances. The configuration of the camera and image processing device in this example is the same as in Example 1. The tracking sequence and imaging sequence when a subject is detected are also the same as in Example 1.

[0069] The flowchart in Figure 8 shows the search sequence in this embodiment, with step S402 in the search sequence shown in Figure 4 changed to step S801, and step S802 added to the search sequence shown in Figure 4. In step S801, the control unit 207 determines the search distance and the search conditions at that distance. In step S802, it is determined whether or not the search termination conditions included in the search conditions determined in step S801 are satisfied. After performing a search at a search distance corresponding to a specified distance by steps S801 and S802, the search is repeated by changing the search distance and performing the search, thereby performing a search in a search range that includes distances other than the specified distance. The details of the processing in steps S801 and S802 will be explained below.

[0070] In step S401, the control unit 207 acquires the specified distance and in step S801, it determines the search distance and the search conditions at that distance. The search distance and search conditions are determined using the tables shown in Figures 10 and 11, which will be described later. The search distance is the value obtained by converting the specified distance acquired in step S401 into a specific distance. The search conditions are the pan / tilt drive amount per subject detection cycle (i.e., the orientation of the imaging unit 101), the time for performing subject search (search time), and the range of pan / tilt angles for performing subject search (search range), which are determined based on the search distance. The search termination condition is that no subject was detected within the search time or within a predetermined search amount.

[0071] In step S403, the control unit 207 calculates the focal length using equation (1) based on the search distance determined in step S801.

[0072] Next, in step S404, the control unit 207 calculates the pan / tilt angle based on the pan / tilt drive amount and search range, which are search conditions determined in step S801.

[0073] Then, in the loop of steps S405, S406, S407, and S802, a cycle of subject detection at the search distance determined in step S801 is executed. If the search termination condition is satisfied in step S802 without a subject being detected, the process returns to step S801, and the search distance and search conditions are determined again and the search is performed.

[0074] The search distance and search conditions in step S801 are determined assuming that the search distance corresponding to the specified distance will be searched preferentially, while other distances will also be searched evenly. Specifically, the search conditions are determined so that the search for the specified distance is performed for a sufficiently long time (first time), and the search for other distances is performed for a short time (second time).

[0075] The table in Figure 10 is used in step S801 of Figure 9 and shows the correspondence between the search distance and the search time, which is the search termination condition, for a given distance. Column No. 1 in this table shows the search distance and search time for each specified distance. Columns No. 2 to No. 6 show the search distances and search times for subsequent searches after the search in No. 1.

[0076] In step S801, the control unit 207 searches for the specified distance, No. 1, for the specified time. If the search time for No. 1 elapses without a subject being detected, the system returns from step S802 to step S801 and searches for the next distance, No. 2, for the specified time. If the search time for No. 2 also elapses without a subject being detected, the system returns from step S802 to step S801 and searches for the next distance, No. 3, for the specified time. In this way, the system continues searching, changing the search distance and search time, until a subject is detected. If a subject is not detected even after searching for the specified distance, No. 6, for the specified time, the system may search for the specified distance, No. 1, again.

[0077] The table in Figure 11 shows the pan drive amount, tilt drive amount, pan search range, and tilt search range as search conditions for each search distance. The pan drive amount and tilt drive amount are used by the control unit 207 when calculating the pan / tilt angle in step S405 of Figure 9. The pan search range and tilt search range indicate the range over which the pan / tilt is driven when performing the search for that search distance. In the detection cycle from step S405 to step S802, the control unit 207 sequentially changes the pan / tilt angle to keep it within the pan / tilt search range.

[0078] The calculation of the pan / tilt angle in step S405 will be explained using the example of a search distance of 9m. Here, the pan / tilt angle will also be written as (pan angle, tilt angle). Furthermore, this section will explain the case where no subject is detected in each detection cycle, and after the pan / tilt angle is calculated in step S405, the process returns to step S405 again via steps S406, S407, and S802.

[0079] When the search distance is 9m, as shown in Figure 11, the pan search range is ±60deg and the tilt search range is 0~30deg. Therefore, the control unit 207 first sets the orientation of the imaging unit 101 to (-60deg, 0deg). Next, the control unit 207 adds the pan drive amount of 4.8deg to the pan angle and drives the imaging unit 101 to (-55.2deg, 0deg).

[0080] Subsequently, the control unit 207 adds 4.8 degrees to the pan angle and sequentially drives the imaging unit 101 down to (60 degrees, 0 degrees). When the pan angle reaches 60 degrees, the upper limit of the pan search range, the control unit 207 adds 2.4 degrees, the tilt drive amount, to the tilt angle and drives the imaging unit 101 to (60 degrees, 2.4 degrees). At this time, a flag is set to subtract from the pan angle calculation from now on. The control unit 207 then subtracts 4.8 degrees from the pan angle based on this flag and drives the imaging unit 101 to (55.2 degrees, 2.4 degrees). From here on, 4.8 degrees is subtracted from the pan angle. When the pan angle reaches -60 degrees, the lower limit of the pan search range, the control unit 207 adds 2.4 degrees to the tilt angle and drives the imaging unit 101 to (-60 degrees, 5.8 degrees). From this point onward, 4.8 degrees is added to the pan angle. When the pan angle reaches 60 degrees, the upper limit of the pan search range, the control unit 207 adds another 2.4 degrees to the tilt angle and drives the imaging unit 101 to (60 degrees, 7.2 degrees). In this way, through the addition and subtraction of the pan angle and the addition of the tilt angle, when the tilt angle reaches 30 degrees, the upper limit of the tilt search range, the pan / tilt angle is returned to the initial (-60 degrees, 0 degrees).

[0081] In step S801, when the tracking sequence (step S905) or imaging sequence (step S906) shown in Figure 9 has finished and the system has returned to the search sequence (step S903), the control unit 207 changes the method for determining the search distance and search conditions based on the stored subject information. Specifically, if the stored subject information is for a non-priority subject, the system inherits the search distance and search conditions from the previously performed subject search and resumes the subject search from where it left off. On the other hand, if the stored subject information is for a priority subject, the system resets the search distance and search conditions from the previously performed subject search and resumes the subject search from the initial state. This allows the system to comprehensively search for priority subjects within the search range while also being able to temporarily image non-priority subjects when they are detected.

[0082] The six search distances shown in Figures 10 and 11 will be explained. In this embodiment, the searchable distance range is set to 1 to 10 m, and these six search distances are used with the aim of detecting subjects within that range without missing any, while dividing the distance range into as few divisions as possible. By setting the focal length of the lens unit 201 to a certain search distance and performing subject detection using the captured image at that time, it is possible to detect not only subjects located at that search distance but also nearby subjects. For example, if the size of a subject's face on the captured image, where the subject can be detected with sufficient accuracy, is 160 to 360 pixels in the short side direction, the search distance is set to 1.2 m, and the focal length is set so that the size of the subject's face on the captured image is 300 pixels. In this case, subjects located between 1 m and 1.5 m can be detected. Similarly, if the search distance is set to 1.8 m, subjects located between 1.5 m and 2.25 m can be detected. In this way, there is a range of subject distances that can be detected at a certain focal length, so by selecting search distances that do not overlap this range, the search time can be shortened.

[0083] The pan drive and tilt drive amounts shown in Figure 11 will be explained further. These pan / tilt drive amounts indicate the amount of change in the pan / tilt angle of the imaging unit 101 for each cycle of subject detection. The pan / tilt angle is shown. If the pan / tilt drive amount is too large, the subject may not be captured within the imaging field of view, potentially leading to detection failure. Conversely, if the pan / tilt drive amount is too small, while there will be no missed detections of the subject, the number of detection cycles required to complete the search range will increase, resulting in a longer search time. Furthermore, even if the subject is captured within the imaging field of view, if the subject is cut off at the edge of the captured image, it cannot be detected accurately. Therefore, it is necessary to set the pan / tilt drive amount so that parts of the captured images from the preceding and succeeding detection cycles overlap.

[0084] In this embodiment, when the captured image is a VGA (640 x 480 pixels) image, the subject size on the captured image is controlled to be 300 pixels. Assuming the shape of the subject's face is circular, the subject will appear as a circle with a diameter of 300 pixels on the captured image. To capture this circle without it being cut off at the edges of the image, it is sufficient if the pan drive amount is within 1 / 2 of the width of the captured image and the tilt drive amount is within 1 / 3 of the height of the captured image. Therefore, in this embodiment, as shown in Figure 11, the pan drive amount is set to 1 / 2 of the image field of view at each focal length, and the tilt drive amount is set to 1 / 3 of the image field of view at each focal length.

[0085] The pan search range and tilt search range shown in Figure 11 will be explained. In this embodiment, the pan search range is limited as the search distance increases. When a subject moves within a certain range, the further away the subject is, the narrower the pan angle that covers its movement range becomes. Therefore, by limiting the pan search range, it becomes possible to detect subjects within a more targeted range. On the other hand, when the subject is located close to the camera 100, a wide angle of search is required relative to the amount of movement of the subject, so the pan search range is not limited, and the entire searchable range is searched.

[0086] Furthermore, the tilt search range is also limited as the search distance increases. This is to ensure that the camera aligns with the position of a person's face at that distance. Specifically, the tilt search range is limited according to the average height of a person. However, at close range, a wider tilt search range is possible because the position of camera 100 and the variation in people have a greater impact.

[0087] As explained above, this embodiment prioritizes searching for subjects within a specified search distance while also searching for subjects at other distances. This allows for the detection of subjects that have moved a large amount in a short amount of time.

[0088] Furthermore, a detection means may be provided to detect whether the imaging device is being held by the user or mounted on a fixed object. The system may detect a change in the camera's mounting position, reset the search distance selection in the search sequence (step S903), and restart the search from No. 1 in Figure 10. Alternatively, different methods for determining the search distance may be used for the handheld state and the fixed object mounting state. For example, when the device is mounted on a fixed object, the system may search for both the specified distance and an unspecified distance, while when the device is handheld, it may search only for the specified distance.

[0089] Furthermore, although the method of transitioning to the automatic search sequence S902 by step S901 in Figure 9 has been explained when "Automatic" is set for the specified distance, it is also possible to use a method in which all steps except S901 and S902 are executed in step S903. In this case, the search distance and search conditions determined in step S801 are set to conditions that search the entire search range evenly. (Other examples) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0090] The embodiments described above are merely representative examples, and various modifications and changes can be made to each embodiment when implementing the present invention. [Explanation of symbols]

[0091] 100 Automatic Imaging Cameras 101 Telescope Tube 102 Tilt Rotation Unit 103 Pan Rotating Unit 201 Lens Unit 202 Imaging Department 208 Search distance setting section 207 Control Unit 209 Subject detection unit

Claims

1. a search means for performing subject search using image data obtained by an imaging device; a control means for controlling the focal length of the imaging device; The search control device is characterized in that the control means controls the focal length in accordance with a search distance designated within a range of search distances in which the subject search is possible and the size of a search target subject.

2. 2. The search control device according to claim 1, wherein said control means controls said focal length using the actual size and the imaged size of said search target subject as the size of said search target subject.

3. the search target subject is a human face, 3. The search control device according to claim 2, wherein the actual size is a size corresponding to a representative value of the face length.

4. 4. The search control device according to claim 1, wherein, when the object search at the specified search distance is completed, the control means performs the object search at a search distance different from the specified search distance.

5. the control means determines search conditions according to a search distance for performing the subject search; 5. The search control device according to claim 1, wherein the search means performs the subject search under the search conditions.

6. 6. The search control device according to claim 5, wherein the control means determines the search distance and the search conditions in the next object search to be performed based on the search distance and the search conditions in the object search that has already been performed.

7. 7. The search control device according to claim 5, wherein the search conditions include a search time as a time for performing the subject search.

8. 7. The search control device according to claim 6, wherein the control means performs the subject search at the specified search distance for the search time, and then performs the subject search at a search distance different from the specified search distance.

9. 9. The search control device according to claim 7, wherein the control means shortens the search time when the subject search is performed at a search distance different from the specified search distance compared to the search time when the subject search is performed at the specified search distance.

10. 9. The search control device according to claim 4, wherein the search conditions include a search range as a spatial range in which the subject search is performed while changing the orientation of the imaging device.

11. 11. The search control device according to claim 10, wherein said control means limits said search range in accordance with a general height of a human being.

12. The search control device according to any one of claims 1 to 11, characterized in that the control means sets the search distance to an initial value at least when a predetermined time has elapsed after the search distance was specified and when the power to the imaging device is cut off.

13. a notification means for notifying a user; 13. The search control device according to claim 1, wherein the control means causes the notification means to issue the notification in at least one of the following cases: when the search distance is specified; when the search distance is changed; and when a subject is not detected in the subject search for more than a predetermined time.

14. 13. The search control device according to claim 1, wherein, when the subject detected by the subject search is a specific subject, the control means causes the imaging device to track the specific subject.

15. an imaging unit for capturing an image of a subject; An imaging device comprising the search control device according to any one of claims 1 to 14.

16. A control method for performing subject search, comprising: obtaining image data with an imaging device; controlling a focal length of the imaging device; A control method comprising controlling the focal length in accordance with a search distance designated within a range of search distances in which the subject can be searched and a size of a search target subject.

17. A program causing a computer to execute a process according to the control method of claim 16.

Images