Imaging apparatus, method for controlling imaging apparatus, and program

Abstract

To perform recognition processing on one recognition target and subsequently quickly perform recognition processing on the other recognition target.SOLUTION: An imaging apparatus has: a first imaging unit that can photograph a first area; a second imaging unit that can photograph a second area including the first area by performing control to change a photographing direction; detection means that detects a detection target from a first image acquired by the first imaging unit; control means that controls the second imaging unit so that the second imaging unit tracks the detection target detected by the detection means by changing the photographing direction of the second imaging unit; first recognition means that performs recognition processing on a first recognition target in a second image acquired by the second imaging unit; and second recognition means that changes the recognition target from the first recognition target to a second recognition target according to a result of the recognition processing on the first recognition target, and performs recognition processing on the second recognition target different from the first recognition target in the second image.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to an imaging device, and a control method and program for the imaging device. [Background technology]

[0002] With the spread and high speed of the Internet and intranets, video and audio are being distributed over networks. Network cameras that use this technology to store and manage captured video and audio on storage servers are also becoming more and more popular.

[0003] Among network cameras, there are known fixed-angle cameras (hereinafter referred to as wide-area cameras) that use wide-angle lenses to capture a wide area and monitor events occurring within the range of their capture.

[0004] There are also network cameras (hereafter referred to as PTZ cameras) that have driving functions such as panning (horizontal), tilting (vertical), and zooming, and capture narrow-angle, detailed images. In particular, there are cameras that can rotate 360 ​​degrees to capture images, and cameras that can capture a wide range of areas in detail are also known. Devices that integrate these two types of cameras used for different purposes are also known.

[0005] Hereinafter, panning, tilting, and zooming will simply be referred to as pan, tilt, and zoom, respectively. In an imaging device that combines a wide-area camera and a PTZ camera, it is possible to use the wide-area camera to capture a wide area, and when an area requiring attention is found, the PTZ camera to capture the details.

[0006] Network cameras are also known to have functions such as a motion detection function that compares captured images between frames to detect areas of movement or change, and a tracking function that controls the camera to keep the subject at the center of the shooting angle according to the detected movement.Furthermore, a recognition function is also known that performs recognition work by comparing information stored in a database with the results of analyzing the features of a detected subject.

[0007] Patent Document 1 discloses a technique for detecting a moving object using a wide-area camera capable of capturing a wide range of images, and for performing recognition processing for recognizing whether the detected subject is a target subject using a PTZ camera. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] JP 2019-186727 A Summary of the Invention [Problem to be solved by the invention]

[0009] However, in the technology disclosed in the above-mentioned Patent Document 1, the PTZ camera performs recognition processing using the detection results of the wide-area camera, but it does not yet support cases where further recognition processing is desired. Therefore, in order to perform further recognition processing with the PTZ camera, it is necessary to reset the settings, which takes time.

[0010] An object of the present disclosure is to enable rapid recognition processing of another recognition target after a recognition processing of a certain recognition target has been performed. [Means for solving the problem]

[0011] The imaging device has a first imaging unit capable of imaging a first area, a second imaging unit capable of imaging a second area including the first area by controlling and changing an imaging direction, a detection means for detecting a detection target from within a first image acquired by the first imaging unit, a control means for controlling the second imaging unit so that the second imaging unit tracks the detection target detected by the detection means by changing the imaging direction of the second imaging unit, a first recognition means for performing recognition processing of a first recognition target in a second image acquired by the second imaging unit, and a second recognition means for switching the recognition target from the first recognition target to a second recognition target depending on a result of the recognition processing of the first recognition target, and performing recognition processing of the second recognition target different from the first recognition target in the second image. Effect of the Invention

[0012] According to the present disclosure, after a recognition process is performed on a certain recognition target, a recognition process on another recognition target can be performed quickly. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an imaging system. [Diagram 2] FIG. 1 illustrates an example of the configuration of an imaging device. [Diagram 3] FIG. 2 is a diagram illustrating an example of the configuration of a terminal. [Figure 4] FIG. 13 is a diagram showing a display example of a terminal. [Diagram 5] 5 is a flowchart showing a main process of the imaging apparatus. [Figure 6] FIG. 13 is a diagram illustrating switching of recognition processing. [Figure 7] FIG. 13 is a diagram showing a display example of a terminal. [Figure 8] FIG. 13 is a diagram illustrating a notification operation of the execution of recognition processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, the embodiments will be described with reference to the drawings. Note that the configurations in the embodiments described below are merely examples and are not limited to the configurations shown in the drawings.

[0015] (First embodiment) First, the first embodiment will be described.

[0016] Fig. 1 is a diagram showing an example of the configuration of an imaging system 160 according to the first embodiment. The imaging system 160 includes an imaging device 100 installed on a ceiling or a utility pole, a display for images and various information from the imaging device 100, a terminal (external device) 200 for remotely operating the imaging device 100, and a network (IP network) 300 for connecting them. Note that, although Fig. 1 shows one imaging device 100 and one terminal 200, there is no particular limit to the number of them.

[0017] The imaging device 100 has a wide-area camera 110 as a first imaging means, which is a fixed-angle camera that captures wide-angle images using a wide-angle lens, for example. The imaging device 100 also contains a PTZ camera 120 as a second imaging means that can capture only narrow-angle images but can change the imaging direction and optical zoom magnification to obtain high-definition images. The PTZ camera 120 can capture the imaging range of the wide-area camera 110 by moving the imaging range.

[0018] Although the wide area camera 110 is a fixed angle camera, the wide area camera 110 may be capable of zooming as long as it is within the range that the PTZ camera 120 can capture. The imaging device 100 has a function of transmitting captured images to the terminal 200 on the network 300. The terminal 200 has a function of receiving and displaying images from the imaging device 100 via the network 300, and issuing various commands to the imaging device 100.

[0019] Fig. 2 is a block diagram showing an example of the configuration of the imaging device 100 in Fig. 1. The imaging device 100 has a wide-area camera 110, a PTZ camera 120, a control unit 130 that controls these and processes captured images, a storage unit 140, and a communication unit 150 that communicates with a network 300.

[0020] The imaging speed of the wide-area camera 110 and the PTZ camera 120 is not particularly limited, but in this embodiment, it is assumed to be 15 frames / second. The imaging device 100 in this embodiment has a remote control mode and a monitoring mode as imaging modes. The imaging mode is switched according to a command transmitted from the terminal 200.

[0021] In the remote control mode, the control unit 130 performs pan, tilt, and zoom control of the PTZ camera 120 in accordance with a request command from the terminal 200, and transmits an image obtained by the PTZ camera 120 to the terminal 200. At this time, the control unit 130 also transmits a wide-range image (described later) obtained by the wide-area camera 110 to the terminal 200.

[0022] On the other hand, in the monitoring mode, the control unit 130 independently controls the pan, tilt, and zoom of the PTZ camera 120, regardless of the terminal 200. Here, the control unit 130 performs control so that the subject is at the center of the image according to the movement of the detected subject, and performs a tracking operation process to track the subject. Then, the control unit 130 transmits the image obtained by the PTZ camera 120 and the image obtained by the wide area camera 110 to the terminal 200. The remote control mode is not the main focus of this embodiment. Therefore, the monitoring mode will be described below.

[0023] The storage unit 140 serving as a storage means is, for example, a non-volatile memory such as a flash memory or a hard disk. The storage unit 140 stores model information of a "vehicle" for detecting a "vehicle" as a subject from a moving object, and feature information for recognizing (identifying) a vehicle (specific vehicle) to be searched for. There is no particular limit to the number of vehicles to be searched for.

[0024] The model information and the vehicle characteristic information are received from the terminal 200 together with a predetermined registration command. Then, when the control unit 130 receives this registration command, the accompanying received information is stored in the storage unit 140. The subject is not limited to a vehicle, but is intended to be an object with specific characteristics such as a person (face) or a human body (body type). Note that the detection of the vehicle and the recognition of the specific vehicle itself are performed using well-known technology, and detailed description thereof will be omitted here.

[0025] The communication unit 150 is an interface that communicates with the network 300. Note that communication between the communication unit 150 and the network 300 may be wired or wireless.

[0026] The PTZ camera 120 has an imaging optical system 121, an imaging element 122, an image processing unit 123, and a driving unit 124. The imaging optical system 121 is composed of a zoom lens, a focus lens, an aperture mechanism, etc. The imaging element 122 is composed of a two-dimensional image sensor such as a CCD or CMOS, and performs photoelectric conversion by capturing an image, and supplies the obtained electrical signal to the image processing unit 123.

[0027] The image processing unit 123 converts the electrical signal received from the imaging element 122 into digital data and performs processes to improve image quality, such as demosaic processing. The image processing unit 123 also performs gain processing to increase the acquired light amount as a signal level, and further performs processes such as outputting a reset signal to the imaging element 122 to clear the accumulated charge.

[0028] Furthermore, the image processing unit 123 performs exposure control by adjusting the amount of light incident on the image sensor 122 by changing the settings of the shutter speed, the aperture value, and the gain processing. Then, the image processing unit 123 supplies the image data after the above processing to the control unit 130.

[0029] The driving unit 124 drives various motors (not shown) for controlling the panning, tilting and zoom lens operations in accordance with control signals from the control unit 130. As a result of driving by the driving unit 124, the imaging direction (optical axis direction) and the optical zoom magnification of the PTZ camera 120 can be changed.

[0030] The wide-angle camera 110 has an imaging optical system 111 including a wide-angle lens for acquiring a wide-angle image, an imaging element 112, and an image processing unit 113. The imaging element 112 is similar to the imaging element 122. The image processing unit 113 receives an image signal from the imaging element 112, but the processing content thereof is similar to that of the image processing unit 123.

[0031] The control unit 130 includes a drive control unit 131, a camera determination unit 132, a subject detection unit 133, a recognition processing unit 134, a recognition switching unit 135, an encoding unit 136, a communication control unit 137, a command interpretation unit 138, and a frame generation unit 139. The control unit 130 can be configured, for example, with a CPU, a ROM that stores a program executed by the CPU, and a RAM used as a work area for the CPU. In this case, the processing units denoted by reference numerals 131 to 138 are realized by the CPU that executes the program. However, some of them may be realized by dedicated hardware.

[0032] The drive control unit 131 performs processing for generating drive signals for controlling the pan and tilt operations of the drive unit 124 of the PTZ camera 120 and the zoom magnification. Based on these drive signals, the drive unit 124 drives various motors that move the pan, tilt, and zoom lenses (not shown). The drive control unit 131 also controls the wide area camera 110 and the PTZ camera 120 in terms of focusing, aperture, and the like.

[0033] The camera determination unit 132 determines whether the acquired image, the detected subject information described below, and the result information of the recognition processing are images from the wide area camera 110 or the PTZ camera 120, and applies processing to the wide area camera 110 and the PTZ camera 120 according to the determination result.

[0034] The subject detection unit 133, which serves as a detection means for detecting a subject, detects the subject by comparing, in time series, images acquired at a specific frame rate interval from the wide area camera 110 and the PTZ camera 120, and judging that it is a moving object when there is a difference between the frames. The subject detection unit 133 sends to the recognition processing unit 134 size information (size of a circumscribing rectangle that contains the moving object) information, feature information, and position information of the subject detected from the difference information.

[0035] The recognition processing unit 134 as a subject recognition means first identifies the subject from the information from the subject detection unit 133, and determines, for example, whether the detected subject is a vehicle or a person. Furthermore, when one or more new subjects are detected by the subject detection unit 133, the recognition processing unit 134 selects one of them.

[0036] Here, when a subject is detected in each image of the wide-area camera 110 and the PTZ camera 120, recognition processing may be performed for each, but it is also possible to take a detailed image of the subject detected in the image of the wide-area camera 110 by the PTZ camera 120. In this case, the recognition processing unit 134 controls the drive control unit 131 to direct the optical axis direction of the PTZ camera 120 toward the selected subject, and controls the zoom magnification to obtain a high-precision image of the selected subject. Then, the recognition processing unit 134 extracts a feature amount of the selected subject from the image obtained by the PTZ camera 120. The recognition processing unit 134 obtains a degree of agreement between the extracted feature amount and a feature amount for recognizing a search target (for example, a vehicle) stored in the storage unit 140, performs a recognition process to determine whether the selected subject is a search target (whether it is the car being searched for), and outputs the recognition result.

[0037] For example, in the case of license plate recognition processing, the recognition processing unit 134 reads the license plate from the captured image, processes the image (contrast adjustment, binarization, etc.) from the read data, estimates the characters from the character outlines, and judges the characters from character size information. There are many known methods for judging these characters based on learning data, etc.

[0038] The recognition switching unit 135, which serves as a determination means for determining whether to switch the recognition process, determines whether to perform a recognition process different from that performed by the recognition processing unit 134, depending on the recognition result by the recognition processing unit 134, and performs a recognition process switching process depending on the determination. This switching process will be described later with specific examples.

[0039] The encoding unit 136 encodes the image data captured by the wide area camera 110 and the PTZ camera 120, obtained from the image processing units 113 and 123, to generate encoded data. Then, the communication control unit 137 transmits the encoded data generated by the encoding unit 136 to the terminal 200 via the communication unit 150.

[0040] At this time, the communication control unit 137 transmits the image obtained from the wide area camera 110 determined by the camera determination unit 132 and the image obtained by the PTZ camera 120 in an appropriate layout. The communication control unit 137 also generates, as metadata, data such as start-up information related to the image, and in the case of the PTZ camera 120, information such as pan, tilt, and zoom information and coordinate information of the subject. Furthermore, when the communication control unit 137 receives various request commands from the terminal 200 via the communication unit 150, it notifies the command interpretation unit 138 of the request commands.

[0041] The command interpretation unit 138 analyzes the request command notified from the communication control unit 137 and executes processing according to the request command. For example, when the command interpretation unit 138 receives a command to transition to a remote control mode, it transitions to the remote control mode. In the remote control mode, the PTZ camera 120 is controlled according to the pan angle, tilt angle, and zoom magnification according to the request command from the terminal 200, and the image captured at that time is transmitted to the terminal 200 together with the wide-range image.

[0042] On the other hand, when a command to transition to the monitoring mode is received, the command interpretation unit 138 performs the monitoring mode process described in this embodiment. In the monitoring mode, as described above, there is a registration command for registering model information for subject detection and feature amounts of search target subjects in the storage unit 140.

[0043] The frame generation unit 139, which serves as a display means for displaying the detection frame, generates a rectangular frame circumscribing the subject in accordance with the detection processing results of the subject detection unit 133 and the recognition processing results of the recognition processing unit 134, and superimposes the detected area on the captured image.

[0044] The configuration and each processing unit of the imaging device 100 in this embodiment have been described above. Next, the configuration and operation of the terminal 200 will be described.

[0045] 3 is a block diagram showing an example of the configuration of the terminal 200 in FIG. 1. The terminal 200 is an example of an information processing device represented by a personal computer. The terminal 200 has a CPU 201 that controls the entire terminal 200, a ROM 202 that stores a BIOS and a boot program, and a RAM 203 that stores an OS (operating system), a network camera application, etc., and functions as a work area. The terminal 200 also has a HDD 204 that stores the OS, the network camera application, etc., and a network I / F 205 for communicating with a network 300. The terminal 200 also has an operation unit 206 that is composed of a keyboard, a mouse, a touch panel, etc. for inputting instructions from a user. The terminal 200 also has a display control unit 207, a display device 208, and an input / output I / F 209.

[0046] In the above configuration, when the terminal 200 is powered on, the CPU 201 loads the OS from the HDD 204 to the RAM 203 in accordance with the boot program stored in the ROM 202, and executes it, so that the terminal 200 functions as an information processing device. As a result, the operation unit 206 and the display device 208 function as a user interface.

[0047] Then, the user operates the operation unit 206 to instruct execution of a camera application program. As a result, the camera application is loaded from the HDD 204 to the RAM 203 and executed, and the terminal 200 functions as a device that displays an image captured by the imaging device 100. The display control unit 207 responds to the operation of the operation unit 206, thereby realizing the operation intended by the user.

[0048] The display control unit 207 also sets the arrangement and layout of the captured image when it is displayed on the display device 208. It is also assumed that, for example, an image showing a specific vehicle or the face of a specific person (generally called a query image) is input from the input / output I / F 209 while the camera application program is being executed. In that case, the CPU 201 extracts features for recognizing an individual from the input image, and transmits the features together with a command to register them in the imaging device 100.

[0049] Furthermore, some of the functions of the control unit 130 in the imaging device 100 (for example, the subject detection unit 133 and the recognition processing unit 134) and the functions of the storage unit 140 can be performed by the CPU 201 and the HDD 204, respectively.

[0050] Here, the switching of the recognition process in this embodiment will be described with reference to Fig. 4 which is a display example of the display device 208, Fig. 5 which is a flowchart showing a series of steps in the recognition process, and Fig. 6(a) to (c) which show an image of the switching. In the figure, the image capturing device 100 detects a vehicle in a parking lot, performs a recognition process for the license plate, and performs a process according to the determination result. However, the present invention is not limited to this as long as it does not deviate from the process of switching between different recognition processes.

[0051] 4 is a diagram showing an example of a display screen 401 displayed by the terminal 200. The display screen 401 is a display screen of the display device 208 while an application is being executed. The display screen 401 has a display area 402 for displaying a wide-range image received from the wide-area camera 110, a display area 403 for displaying an image obtained from the PTZ camera 120, and an operation area 404 in which GUI buttons, sliders, etc. related to user operations are arranged.

[0052] The operation area 404 has a button for selecting either the monitoring mode or the remote control mode. Also arranged in the operation area 404 are buttons and sliders for specifying the pan angle, tilt angle, and zoom magnification in the remote control mode, buttons for setting search conditions in the monitoring mode, and the like.

[0053] In the case of Figure 4, the wide area camera 110 is in monitoring mode and shows a moving object 405 being detected in a display area 402, and the detected moving object 405 being tracked by the PTZ camera 120 is displayed in a display area 403.

[0054] The operation area 404 further has a function for setting the recognition process, and allows the user to select which recognition process to perform for each of the wide-area camera 110 and the PTZ camera 120. For example, when setting the recognition process for license plates, the user can select whether to detect vehicles and set which area the target has entered to be processed. When setting the recognition process for people, the user can select whether to detect people and set individual settings required for person recognition, such as the distance between the eyes. Furthermore, by having a function for inputting and outputting search target images and exchanging data with the storage unit 140, the user can update information.

[0055] FIG. 5 is a flowchart showing a method of controlling the imaging device 100. In this embodiment, the imaging device 100 automatically switches the implementation of recognition processing. A method of main processing in the switching will be described with reference to the flowchart shown in FIG. 5. Specifically, an example of switching from vehicle license plate recognition to person recognition will be described. In addition, the imaging device 100 in this embodiment will be described assuming that the surveillance mode is the initial imaging mode. Also, it is assumed that the recognition settings for license plate recognition and person recognition are set in advance in the operation area 404.

[0056] In step S501, the imaging device 100 starts operation and captures images using the wide area camera 110 and the PTZ camera 120. The wide area camera 110 is an imaging unit capable of capturing an image of a first area. The PTZ camera 120 is an imaging unit capable of capturing an image of a second area including the first area by controlling and changing the shooting direction. The subject detection unit 133 detects a moving object from within the first image acquired by the wide area camera 110. The recognition processing unit 134 detects whether the moving object detected by the subject detection unit 133 is a vehicle (detection target). When a vehicle is detected from within the image acquired by the wide area camera 110, the process proceeds to step S502.

[0057] In step S502, the control unit 130 controls the pan, tilt, and zoom of the PTZ camera 120 so that the PTZ camera 120 tracks the vehicle detected in the image of the wide area camera 110. That is, the control unit 130 controls the PTZ camera 120 so that the PTZ camera 120 tracks the vehicle detected by the recognition processing unit 134 by changing the shooting direction of the PTZ camera 120. The control unit 130 functions as a transmission unit, and transmits the image acquired by the wide area camera 110 and the image acquired by the PTZ camera 120 to the terminal 200 as shown in FIG. 4. The terminal 200 displays the image acquired by the wide area camera 110 in the display area 402, and displays the image acquired by the PTZ camera 120 in the display area 403.

[0058] In step S503, the control unit 130 determines whether or not the vehicle in the image acquired by the PTZ camera 120 has stopped in the parking space based on the tracking operation of the PTZ camera 120. If the vehicle has not stopped, the process returns to step S502, and the tracking operation of the PTZ camera 120 continues. If the vehicle has stopped, the process proceeds to step S504.

[0059] In step S504, the recognition processing unit 134 functions as a recognition unit, and performs a recognition process of the license plate number of the vehicle (first recognition target) in the second image acquired by the PTZ camera 120.

[0060] In step S505, the control unit 130 performs data matching between the data recognized in step S504 and the model information stored in the storage unit 140. Specifically, the control unit 130 stores suspicious vehicle information in the storage unit 140, and matches information related to the license plate number of the vehicle recognized by the recognition processing unit 134 with information on the license plate number of the suspicious vehicle. The license plate number of the suspicious vehicle is a number stored in the storage unit 140.

[0061] In step S506, the control unit 130 determines whether the recognized vehicle is a suspicious vehicle based on the comparison result in step S505. If the recognized vehicle is a suspicious vehicle, the process proceeds to step S507. If the recognized vehicle is not a suspicious vehicle, the control unit 130 ends the tracking operation of the PTZ camera 120, and the process proceeds to step S513.

[0062] In step S507, the control unit 130 causes the PTZ camera 120 to continue the tracking operation of the target vehicle.

[0063] Here, if a vehicle is determined to be suspicious, it is desired to photograph the person driving the suspicious vehicle, but if person recognition is to be performed, it becomes necessary to set up person recognition from the timing when vehicle recognition is completed and to switch manually. In order to prevent a situation in which person recognition cannot be performed due to time being taken up by the setting, in this embodiment, when a suspicious vehicle is detected, the recognition switching unit 135 switches from the number plate recognition setting to the person recognition setting that has been registered in advance. The switching will be described later separately using Figs. 6(a) to (c).

[0064] In step S508, the recognition processing unit 134 switches the recognition target from the license plate number to a person based on the person recognition setting switched by the recognition switching unit 135. Then, the recognition processing unit 134 performs recognition processing of a person (second recognition target) of a driver of a vehicle whose number is different from the license plate number of the vehicle in the second image acquired by the PTZ camera 120.

[0065] In step S509, control unit 130 performs data matching between the data recognized in step S508 and the model information stored in storage unit 140. Specifically, control unit 130 stores registered person information of the parking lot manager, system manager, and user in storage unit 140, and matches information about the person recognized by recognition processing unit 134 with information about the registered person. The registered person is a person stored in storage unit 140.

[0066] In step S510, the control unit 130 determines whether the recognized person is a registered person based on the result of the comparison in step S509. If the recognized person is a registered person, the process proceeds to step S511. If the recognized person is not a registered person, the process proceeds to step S514.

[0067] In step S511, since the control unit 130 determines that the driver of the suspicious vehicle determined in step S506 is not a suspicious person but a registered person, it updates the information registered as the suspicious vehicle in the storage unit 140 and performs a process of linking the vehicle and the person. In this information linking, a process of writing each other's information data on each image data is performed using metadata.

[0068] In step S512, when the update in step S511 is completed, the control unit 130 controls the PTZ camera 120 so as to end the tracking operation of the PTZ camera 120. After that, the process proceeds to step S513.

[0069] In step S514, since there is a possibility that the driver of the suspicious vehicle is a suspicious person, the control unit 130 performs a process of registering the recognized person as a suspicious person. The control unit 130 performs a process of registering the image information of the PTZ camera 120 as metadata.

[0070] In step S515, when the registration process in step S514 is completed, the control unit 130 controls the PTZ camera 120 to continue tracking and capture the movement of the suspicious person until the next target detection information is received. After that, the process proceeds to step S513.

[0071] In step S513, the control unit 130 shifts the camera operation to a state waiting for vehicle detection in the image of the wide area camera 110.

[0072] The recognition targets in steps S504 and S508 are, for example, a vehicle license plate number or a person, respectively.

[0073] Figures 6(a) to (c) are diagrams showing switching of recognition processing of an image of the PTZ camera 120 in step S507 in Figure 5. Figures 6(a) to (c) show an image of the PTZ camera 120 in the display area 403 described in Figure 4.

[0074] Fig. 6(a) is a diagram showing an image of the PTZ camera 120 in the display area 403 when the license plate recognition is performed in step S504. Fig. 6(b) is a diagram showing an image of the PTZ camera 120 in the display area 403 when the person recognition is performed in step S508. Figs. 6(a) and (b) show the state when a vehicle, which is a moving object, is parked in a parking lot facing the front of the PTZ camera 120.

[0075] As shown in FIG. 6A, in step S504, the control unit 130 detects a license plate area 601 from vehicle contour information and the like, and performs recognition processing of the license plate area 601, such as the license plate number.

[0076] 6(b), in step S508, after completing the license plate recognition, the control unit 130 switches the recognition target to an area 602 assumed to be the driver's seat in order to recognize a person, and performs moving object detection and person recognition processing in this area 602. The area 602 is, for example, the windshield area of ​​the vehicle.

[0077] Here, in Fig. 6(b), the recognition process is performed through the windshield in the area 602, so that the recognition process may not be performed correctly due to the influence of reflection from the glass, etc. In that case, as shown in Fig. 6(c), in step S508, the control unit 130 may perform person recognition process on objects that have moved in the detection areas 603 on both sides of the driver's seat and the passenger seat so that the recognition process can be performed when the person in the vehicle gets off. The detection area 603 is, for example, the area of ​​the door entrance of the vehicle.

[0078] In this embodiment, the process is described when the license plate is registered, but the process is not limited to this, and for example, a vehicle with the license plate removed may be considered. In this case, in step S504, the control unit 130 cannot recognize the license plate of the vehicle. In step S506, if the license plate of the vehicle cannot be recognized, the process proceeds to step S507.

[0079] Furthermore, in step S504, control unit 130 performs a number recognition process in license plate area 601, and if the number cannot be recognized in license plate area 601, performs a number recognition process in an area larger than license plate area 601. In step S506, if the number cannot be recognized even in the above-mentioned larger area, the process proceeds to step S507.

[0080] In this embodiment, the control unit 130 performs person recognition processing through the windshield when the vehicle faces the front of the PTZ camera 120, but cannot recognize people when the vehicle faces backwards when parking. Therefore, in step S508, the control unit 130 may determine the orientation of the vehicle from the image of the wide-area camera 110 and change the area in which person recognition is performed depending on the orientation of the vehicle. When the front of the vehicle is captured, the control unit 130 performs person recognition processing in area 602 in FIG. 6(b), and when the rear of the vehicle is captured, the control unit 130 performs person recognition processing in detection area 603 in FIG. 6(c).

[0081] As described above, according to this embodiment, the imaging device 100 can instantly perform recognition processing of a second recognition target (person) without the user having to reset the recognition target after performing recognition processing of a first recognition target (license plate number) by the PTZ camera 120. Since the user does not have to reset the settings, switching from the first recognition target to the second recognition target can be performed quickly, preventing the subject from being missed.

[0082] Second Embodiment In the configuration of the imaging device 100 according to the second embodiment, the description of the same parts as those shown in the first embodiment will be omitted. In the above-mentioned first embodiment, a series of switching operations of the recognition processing is described, but in the second embodiment, a display method at the time of switching the recognition will be described.

[0083] In the first embodiment, the moving object captured by the wide-area camera 110 was considered as one moving object, but when multiple moving objects are captured, it is not clear which moving object the recognition process should start with. Also, when selecting the object to be recognized from among the subjects captured by the wide-area camera 110 in the remote control mode, it is not clear which object has already completed the recognition process. Therefore, the second embodiment aims to make it easier to visually understand the object to be recognized and the state when the recognition process is switched.

[0084] Fig. 7 is a diagram showing a display example of the display area 402 of the terminal 200 according to the second embodiment. The operation of notifying whether or not the recognition process is to be executed will be described with reference to Fig. 7. Fig. 7 shows the display area 402 that displays an image captured by the wide area camera 110. In a situation where a vehicle that is the subject of the image is detected, vehicles for which the recognition process has not been completed are surrounded by solid line frames 701 and 702, respectively. Vehicles for which the recognition process has already been completed are surrounded by dotted line frames 703 and 704, respectively.

[0085] The control unit 130 transmits the image of the display area 402 in FIG. 7 to the terminal 200. The image of the display area 402 in FIG. 7 is an image of the wide area camera 110 on which the information in the dotted frames 703 and 704 and the information in the solid frame 701 and 702 are superimposed. The information in the dotted frames 703 and 704 is information indicating the vehicle detected in step S501 for which the recognition process of the license plate number and the person has been completed. The information in the solid frame 701 and 702 is information indicating the vehicle detected in step S501 for which the recognition process of the license plate number and the person has not been completed. The terminal 200 displays the image received from the imaging device 100 in the display area 402 as in FIG. 7.

[0086] An example will be described in which the control unit 130 detects the vehicle in the solid frame 701 first, and the vehicle in the solid frame 702 second. The order of detection is displayed, for example, by a circled number "1" in the solid frame 701 and a circled number "2" in the solid frame 702. Recognition processing on the images from the PTZ camera 120 is performed according to this order of detection. Here, the order of images from the PTZ camera 120 is described as an example of the order of detection, but a special priority order may be used, for example, prioritizing red cars, or other prioritization that takes into account characteristic information of the subject. In addition, the solid frame of a subject undergoing recognition processing may be differentiated, for example, by applying a different color.

[0087] The control unit 130 transmits the image of the display area 402 in Fig. 7 to the terminal 200. The image of the display area 402 in Fig. 7 is an image of the wide area camera 110 on which information indicating the order in which the vehicles in solid-line frames 701 and 702 were detected is superimposed.

[0088] 8(a) to (d) are diagrams showing display examples of the display area 403 of the terminal 200 according to the second embodiment. A notification operation of the execution of recognition processing on an image from the PTZ camera 120 will be described with reference to Figs. 8(a) to (d). Figs. 8(a) to (d) show the display area 403 of an image captured by the PTZ camera 120.

[0089] 8(a) shows the display area 403 when license plate recognition processing is being performed while the vehicle is parked in a parking lot. The license plate area 801 enclosed in a dotted line indicates that the recognition processing of the license plate area is in progress.

[0090] Fig. 8(b) shows the display area 403 when the recognition process of the license plate area is completed after the display of the display area 403 in Fig. 8(a). The license plate area 802 enclosed in a solid line frame indicates that the recognition process of the license plate area is completed.

[0091] Fig. 8(c) shows display area 403 when the recognition process is switched after displaying display area 403 in Fig. 8(b). License plate area 802 framed by a solid line indicates that recognition process of the license plate area is complete. Person recognition area 803 framed by a dotted line indicates that recognition process of the person recognition area is currently being performed. This display makes it easy to visually understand that the recognition process has shifted from license plate recognition to person recognition processing.

[0092] Fig. 8(d) shows display area 403 when recognition processing of the person recognition area is completed after displaying display area 403 in Fig. 8(c). License plate area 802 framed with a solid line indicates that recognition processing of the license plate area is complete. Person recognition area 804 framed with a solid line indicates that recognition processing of the person recognition area is complete. When person recognition processing is completed, the fact that person recognition has been completed is displayed in person recognition area 804 framed with a solid line.

[0093] In this embodiment, the execution or non-execution of the recognition process is expressed by the type of line of the recognition area frame, but this is not limited to this, and the difference may be made, for example, by changing the color or shape.

[0094] Control unit 130 transmits an image of PTZ camera 120 on which information indicating that the recognition process of the license plate number is in progress or the recognition process has been completed, or information indicating that the recognition process of the person is in progress or the recognition process has been completed, is superimposed, as shown in Figs. 8(a) to (d). Terminal 200 displays the image transmitted by imaging device 100 in display area 403, as shown in Figs. 8(a) to (d).

[0095] As described above, according to this embodiment, it is possible to visually make it easy to understand the target of the recognition process and how the recognition process is switched.

[0096] (Other embodiments) Although the first and second embodiments have been described in detail above with reference to the drawings, the present invention is not limited to these specific embodiments, and various forms are also included within the scope of the gist of the first and second embodiments. Parts of the above-described embodiments may be appropriately combined.

[0097] For example, the process of switching from license plate recognition to person recognition has been described, but the process of switching may be one that focuses on people, such as from person recognition to face recognition, or one that switches from person recognition to license plate recognition.

[0098] The present disclosure can also be realized by a process in which a program for implementing one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) for implementing one or more functions.

[0099] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) a first imaging unit capable of capturing an image of a first area; a second imaging unit capable of imaging a second area including the first area by controlling a change in an imaging direction; A detection means for detecting a detection target from within a first image acquired by the first imaging unit; A control means for controlling the second imaging unit so that the second imaging unit tracks the detection target detected by the detection means by changing an imaging direction of the second imaging unit; a first recognition means for performing a recognition process on a first recognition target in a second image acquired by the second imaging unit; a second recognition means for switching the recognition target from the first recognition target to a second recognition target according to a result of the recognition processing of the first recognition target, and performing a recognition processing of the second recognition target different from the first recognition target in the second image; An imaging device comprising: (Configuration 2) The first recognition means compares information about the first recognition target with information stored in a storage unit; The imaging device according to configuration 1, wherein the second recognition means switches the recognition target from the first recognition target to a second recognition target depending on a result of the matching by the first recognition means, and performs a recognition process of the second recognition target in the second image. (Configuration 3) The second recognition means compares information about the second recognition target with information stored in a storage unit; The imaging device according to configuration 1, wherein the control means controls the second imaging unit to continue or end the tracking depending on a result of the matching by the second recognition means. (Configuration 4) The first recognition means compares information about the first recognition target with information stored in a storage unit; the second recognition means switches a recognition target from the first recognition target to a second recognition target according to a result of the comparison by the first recognition means, performs a recognition process for the second recognition target in the second image, and compares information about the second recognition target with information stored in a storage unit; The imaging device according to configuration 1, wherein the control means controls the second imaging unit to continue or end the tracking depending on a result of the matching by the second recognition means. (Configuration 5) 5. The imaging device according to any one of configurations 1 to 4, wherein the first recognition target and the second recognition target are a vehicle license plate number or a person, respectively. (Configuration 6) the detection target is a vehicle, The first recognition target is a vehicle license plate number, The first recognition means verifies whether the number is a registered number, The imaging device described in configuration 1, characterized in that the second recognition means switches the recognition target from the number to a second recognition target when the number is a registered number or when the number cannot be recognized, and performs recognition processing of the second recognition target in the second image. (Configuration 7) The first recognition means performs a recognition process of the number in a license plate area, and when the number cannot be recognized in the license plate area, performs a recognition process of the number in an area larger than the license plate area; The imaging device described in configuration 6, characterized in that when the second recognition means cannot recognize the number even in the wide area, it switches the recognition target from the number to a second recognition target and performs recognition processing of the second recognition target in the second image. (Configuration 8) 8. The imaging device according to configuration 7, wherein the second recognition target is a person. (Configuration 9) 9. The imaging device according to configuration 8, wherein the second recognition means performs a recognition process for the person in a windshield area of ​​the vehicle or an entrance / exit area of ​​a door of the vehicle. (Configuration 10) The second recognition means verifies whether the person is a registered person stored in a storage unit, The imaging device described in configuration 8 or 9, wherein the control means controls the second imaging unit to end the tracking if the person is a registered person, and to continue the tracking if the person is not a registered person. (Configuration 11) the detection target is a vehicle, 11. The imaging device according to any one of configurations 1 to 10, wherein the first recognition means starts the recognition process after the vehicle photographed by the second imaging unit stops. (Configuration 12) 12. The imaging device according to any one of configurations 1 to 11, wherein the second recognition means performs recognition processing on different areas depending on the orientation of the detection target. (Configuration 13) The imaging device according to any one of configurations 8 to 10, wherein the second recognition means performs a recognition process of the person in a windshield area of ​​the vehicle when the front of the vehicle is photographed, and performs a recognition process of the person in an entrance / exit area of ​​the vehicle when the rear of the vehicle is photographed. (Configuration 14) 14. The imaging device according to any one of configurations 1 to 13, further comprising a transmission means for transmitting the first image and the second image. (Configuration 15) The imaging device according to any one of configurations 1 to 13, further comprising a transmission means for transmitting the second image on which information indicating that the recognition process of the first recognition target is in progress or the recognition process has been completed, or information indicating that the recognition process of the second recognition target is in progress or the recognition process has been completed, is superimposed. (Configuration 16) The imaging device according to any one of configurations 1 to 13, further comprising a transmission means for transmitting the first image on which information indicating a detection target detected by the detection means and for which recognition processing of the first recognition target and the second recognition target has been completed, and information indicating a detection target detected by the detection means and for which recognition processing of the first recognition target and the second recognition target has not been completed, are superimposed. (Configuration 17) The imaging device according to any one of configurations 1 to 13, further comprising a transmission means for transmitting the first image on which information indicating an order in which detection objects detected by the detection means and for which recognition processing of the first recognition object and the second recognition object has not been completed were detected is superimposed. (Method 1) a first imaging unit capable of capturing an image of a first area; and a second imaging unit capable of imaging a second area including the first area by controlling and changing an imaging direction, A detection step of detecting a detection target from within a first image acquired by the first imaging unit; a control step of controlling the second imaging unit so that the second imaging unit tracks the detection target detected in the detection step by changing an imaging direction of the second imaging unit; a first recognition step of performing a recognition process on a first recognition target in a second image acquired by the second imaging unit; a second recognition step of switching the recognition target from the first recognition target to a second recognition target according to a result of the recognition processing of the first recognition target, and performing a recognition processing of the second recognition target different from the first recognition target in the second image; 13. A method for controlling an imaging apparatus comprising: (Program 1) A program for causing a computer to function as the imaging device according to any one of configurations 1 to 17. [Explanation of symbols]

[0100] 100 imaging device, 200 terminal, 300 network, 110 wide-area camera, 120 PTZ camera, 130 control unit, 140 memory unit, 150 communication unit, 131 drive control unit, 132 camera determination unit, 133 subject detection unit, 134 recognition processing unit, 135 recognition switching unit, 136 encoding unit, 137 communication control unit, 138 command interpretation unit, 139 frame generation unit

Claims

1. a first imaging unit capable of imaging a first area; a second imaging unit capable of imaging a second area including the first area by controlling a change in an imaging direction; A detection means for detecting a detection target from within a first image acquired by the first imaging unit; a control means for controlling the second imaging unit so that the second imaging unit tracks the detection target detected by the detection means by changing an imaging direction of the second imaging unit; a first recognition means for performing a recognition process of a first recognition target in a second image acquired by the second imaging unit; a second recognition means for switching the recognition target from the first recognition target to a second recognition target according to a result of the recognition processing of the first recognition target, and performing a recognition processing of the second recognition target different from the first recognition target in the second image; An imaging device comprising:

2. The first recognition means compares information about the first recognition target with information stored in a storage unit, 2. The imaging device according to claim 1, wherein the second recognition means switches the recognition target from the first recognition target to a second recognition target depending on a result of the matching by the first recognition means, and performs a recognition process of the second recognition target in the second image.

3. The second recognition means compares information about the second recognition target with information stored in a storage unit, 2. The imaging device according to claim 1, wherein the control means controls the second imaging unit so as to continue or end the tracking depending on a result of the matching by the second recognition means.

4. The first recognition means compares information about the first recognition target with information stored in a storage unit, The second recognition means switches a recognition target from the first recognition target to a second recognition target according to a result of the comparison by the first recognition means, performs a recognition process for the second recognition target in the second image, and compares information about the second recognition target with information stored in a storage unit; 2. The imaging device according to claim 1, wherein the control means controls the second imaging unit so as to continue or end the tracking depending on a result of the matching by the second recognition means.

5. 2. The imaging device according to claim 1, wherein the first recognition target and the second recognition target are a vehicle license plate number or a person, respectively.

6. the detection target is a vehicle, the first recognition target is a vehicle license plate number, The first recognition means verifies whether the number is a registered number, The imaging device according to claim 1, characterized in that, when the number is a registered number or when the number cannot be recognized, the second recognition means switches the recognition target from the number to a second recognition target and performs recognition processing of the second recognition target within the second image.

7. The first recognition means performs a recognition process of the number in a license plate area, and when the number cannot be recognized in the license plate area, performs a recognition process of the number in an area larger than the license plate area; The imaging device according to claim 6, characterized in that, when the number cannot be recognized even in the wide area, the second recognition means switches the recognition target from the number to a second recognition target and performs recognition processing of the second recognition target within the second image.

8. 8. The imaging apparatus according to claim 7, wherein the second recognition target is a person.

9. 9. The imaging device according to claim 8, wherein the second recognition means performs the recognition process of the person in a windshield area of ​​the vehicle or an entrance / exit area of ​​a door of the vehicle.

10. The second recognition means verifies whether the person is a registered person stored in a storage unit, The imaging device according to claim 8, wherein the control means controls the second imaging unit to terminate the tracking when the person is a registered person, and to continue the tracking when the person is not a registered person.

11. the detection target is a vehicle, 2. The imaging device according to claim 1, wherein the first recognition means starts the recognition process after the vehicle photographed by the second imaging unit stops.

12. 2. The imaging device according to claim 1, wherein the second recognition means performs recognition processing on different areas depending on the orientation of the detection target.

13. The imaging device according to claim 8, characterized in that the second recognition means performs recognition processing of the person in a windshield area of ​​the vehicle when the front of the vehicle is photographed, and performs recognition processing of the person in an entrance / exit area of ​​the vehicle when the rear of the vehicle is photographed.

14. 2. The imaging device according to claim 1, further comprising a transmission means for transmitting the first image and the second image.

15. 2. The imaging device according to claim 1, further comprising a transmission means for transmitting the second image on which information indicating that recognition processing of the first recognition target is in progress or that recognition processing has been completed, or information indicating that recognition processing of the second recognition target is in progress or that recognition processing has been completed, is superimposed.

16. 2. The imaging device according to claim 1, further comprising a transmission means for transmitting the first image on which information indicating the detection targets detected by the detection means and for which recognition processing of the first recognition target and the second recognition target has been completed, and information indicating the detection targets detected by the detection means and for which recognition processing of the first recognition target and the second recognition target has not been completed, are superimposed.

17. 2. The imaging device according to claim 1, further comprising a transmission means for transmitting the first image on which information indicating an order in which detection objects detected by the detection means and for which recognition processing of the first recognition object and the second recognition object has not been completed were detected is superimposed.

18. a first imaging unit capable of imaging a first area; and a second imaging unit capable of imaging a second area including the first area by controlling and changing an imaging direction, A detection step of detecting a detection target from within a first image acquired by the first imaging unit; a control step of controlling the second imaging unit so that the second imaging unit tracks the detection target detected in the detection step by changing an imaging direction of the second imaging unit; a first recognition step of performing a recognition process of a first recognition target in a second image acquired by the second imaging unit; a second recognition step of switching the recognition target from the first recognition target to a second recognition target according to a result of the recognition processing of the first recognition target, and performing a recognition processing of the second recognition target different from the first recognition target in the second image; 13. A method for controlling an imaging apparatus comprising:

19. A program for causing a computer to function as the imaging device according to any one of claims 1 to 17.

Images