Information processing device, processing system, control method, and program

Abstract

To provide an information processing device for optimizing a processing speed as a whole system, and its control method.SOLUTION: In an image analysis system including an imaging device, a detachable device, and a server, the imaging device 110 includes an analysis unit for executing analysis processing to image data, a network communication unit 307 for communicating with a server which can execute analysis processing to the image data through a network, a server load state acquisition unit 313 for acquiring load information on a load state in the server, and a control unit 304 for determining whether the analysis processing is performed by the analysis unit or the server on the basis of the load information.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to distributed processing technology. [Background technology]

[0002] In recent years, solutions that perform facial recognition using images captured by surveillance cameras to manage access control have become widespread. Because image processing such as facial recognition requires a relatively high processing load, surveillance cameras have traditionally transferred captured images to a remote, high-performance computing device (such as a PC or server) where image processing is performed. In contrast, with the recent improvement in the processing power of mobile computing devices, it has become possible to perform image processing locally on the surveillance camera. The processing on the surveillance camera side can be performed, for example, by a computing device located on the camera itself. It can also be performed by a computing device in a removable device connected to the camera itself. Performing image processing on the surveillance camera side can reduce the image processing load on the server and the communication load on the communication network.

[0003] Patent Document 1 describes a technology that provides extended functions by attaching a removable device to an imaging device. Patent Document 2 describes a technology that attaches a removable device to an imaging device and switches between processing by the removable device and the imaging device based on an execution instruction. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 7-184098 [Patent Document 2] Japanese Patent Application Publication No. 2019-219804 Summary of the Invention [Problem to be solved by the invention]

[0005] However, while Patent Documents 1 and 2 describe the expansion of processing in removable devices, they do not consider efficient usage methods, including the distribution of processing with server processing. Generally, imaging devices and removable devices have lower hardware performance than PCs and servers. Therefore, the techniques described in Patent Documents 1 and 2 cannot optimize processing across the entire system.

[0006] The present invention has been made in view of the above problems, and aims to provide a technique that enables optimization of the processing speed of the entire system. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, an information processing device according to the present invention has the following arrangement. a processing means for performing an analysis process on the image data; a communication means for communicating with an external processing device capable of executing analysis processing on image data; a load acquisition means for acquiring load information relating to a load state of the external processing device; a determining means for determining whether the analysis processing is to be performed by the processing means or the external processing device based on the load information; With death, the determining means determines whether the analysis processing is to be performed by the processing means or the external processing device based on the load information and a determination table that defines a device that performs the analysis processing depending on at least a load state of the external processing device; The analysis process is composed of a plurality of sub-processes, The information processing device includes: a first capability acquisition means for acquiring a first image processing capability indicating which of the plurality of sub-processes the processing means is capable of executing; a second capability acquisition means for acquiring a second image processing capability indicating which of the plurality of sub-processes the external processing device is capable of executing; a correcting means for correcting the decision table based on the first image processing capability and the second image processing capability; Further having . [Effects of the Invention]

[0008] According to the present invention, it is possible to provide a technique that enables optimization of the processing speed of the entire system. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 illustrates an example of a system configuration. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of an imaging apparatus. [Figure 3] FIG. 2 is a diagram illustrating an example of a functional configuration of an imaging apparatus. [Figure 4] FIG. 2 illustrates an example of the hardware configuration of a removable device. [Figure 5] FIG. 1 illustrates an example of a functional configuration of a removable device. [Figure 6] FIG. 2 illustrates an example of a hardware configuration of a server. [Figure 7] FIG. 2 illustrates an example of a functional configuration of a server. [Figure 8] FIG. 10 is a diagram illustrating an example of a flow of processing executed in the system. [Figure 9] FIG. 10 is a diagram illustrating an example of the flow of face authentication processing and electronic lock unlocking processing. [Figure 10] FIG. 10 is a diagram illustrating an example of a table used in the process of determining the image processing execution location. [Figure 11] FIG. 10 is a diagram illustrating an example of the flow of a process for determining an image processing execution location. [Figure 12] FIG. 10 is a diagram illustrating an example of the load / processing time of each device. [Figure 13] FIG. 10 is a diagram illustrating another example of the flow of the image processing execution location determination process. [Figure 14] FIG. 10 is a diagram showing another example of a table that defines the capabilities of each device. [Figure 15] FIG. 10 is a diagram illustrating another example of the flow of the face authentication process and the electronic lock unlocking process. [Figure 16] FIG. 10 is a diagram illustrating another example of the flow of the face authentication process and the electronic lock unlocking process. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0011] (First embodiment) As a first embodiment of an information processing device according to the present invention, an image analysis system 101 including an imaging device 110, a removable device 100, and a server 130 will be described below as an example.

[0012] <System configuration> FIG. 1 shows an example of the configuration of an image analysis system 101 according to this embodiment. The following description will be given of an example in which the system is an access control system. However, the present invention is not limited to this, and the following discussion can be applied to any processing system that analyzes images and outputs predetermined information. The present system includes imaging devices 110a-110d, a network 120, a server 130 serving as an external processing device, and an I / O module 140. Each of the imaging devices 110a-110d has a slot into which a device capable of recording captured images can be attached or detached. The imaging devices 110a-110d are connected to the removable devices 100a-100d by inserting the removable devices 100a-100d into the slot. The I / O module 140 is also connected to an access controller 150, a card reader 160, and an electric lock 170. By holding an ID card or the like over the card reader 160, the card reader performs ID authentication. In the following description, the removable devices 100a to 100d will be referred to as "detachable devices 100," and the imaging devices 110a to 110d will be referred to as "imaging devices 110."

[0013] The removable device 100 is a computing device that is detachable from the imaging device 110. The removable device 100 is, for example, a device in which a predetermined processing circuit is mounted on an SD card. The removable device 100 is configured to be insertable into the imaging device 110 in its entirety, for example, in the form of an SD card, and thus can be connected to the imaging device 110 without any protruding parts. This prevents the removable device 100 from interfering with obstacles such as wiring, improving the convenience of using the device. Furthermore, many existing imaging devices 110, such as network cameras, are provided with an SD card slot, so that the removable device 100 can provide expanded functions to existing imaging devices 110. The removable device 100 may be configured to be attached to the imaging device 110 using any interface other than the SD card, which is used when attaching a storage device capable of storing at least images captured by the imaging device 110. For example, the removable device 100 may have a USB (Universal Serial Bus) interface and be configured to be attached to a USB socket of the imaging device 110. Furthermore, the predetermined processing circuit is implemented, for example, by an FPGA (Field Programmable Gate Array) programmed to execute predetermined processing, but may also be implemented in other forms.

[0014] The imaging device 110 is an imaging device such as a network camera. In this embodiment, the imaging device 110 is assumed to have a built-in arithmetic device (information processing device) capable of processing video, but is not limited to this. For example, an external computer such as an information processing device (PC: personal computer) connected to the imaging device 110 may exist, and a combination of these may be treated as the imaging device 110. In this embodiment, it is assumed that a removable device 100 is attached to all of the imaging devices 110. Note that while FIG. 1 shows four imaging devices 110 and the removable devices attached to each, the number of combinations of these devices may be three or less, or five or more. By attaching a removable device 100 having an image analysis processing function to the imaging device 110, it becomes possible for the imaging device 110 to perform video processing on its own, even if the imaging device 110 does not have the image analysis processing function. Furthermore, in a configuration in which an arithmetic unit for image processing is arranged in the imaging device 110 as in this embodiment, by attaching the removable device 100 in which the arithmetic unit is arranged to the imaging device 110, it is possible to diversify and advance the image processing that can be performed on the imaging device 110 side.

[0015] The server 130 is configured by a computer such as a PC, and has an image analysis processing function. Furthermore, the server 130 is a device that receives input from a user and outputs information to the user (for example, displays information).

[0016] The imaging device 110 and the server 130 are communicatively connected via a network 120. The network 120 includes a plurality of routers, switches, cables, and the like that comply with a communication standard such as Ethernet (registered trademark). In this embodiment, the network 120 may be any network that enables communication between the imaging device 110 and the server 130, and may be constructed with any scale, configuration, and conformance to any communication standard. For example, the network 120 may be the Internet, a wired local area network (LAN), a wireless LAN, a wide area network (WAN), or the like. The network 120 may be configured to enable communication using a communication protocol that complies with the ONVIF (Open Network Video Interface Forum) standard, for example. However, the present invention is not limited to this, and the network 120 may also be configured to enable communication using another communication protocol, such as a proprietary communication protocol.

[0017] The I / O module 140 is communicably connected to the imaging device 110 and the server 130 via the network 120. The I / O module 140 receives an HTTP event (e.g., an unlocking event) from the imaging device 110 or the server 130, and transmits an electrical control signal to the entrance / exit controller 150 via the electric wire 180.

[0018] The entrance / exit controller 150 is connected to the I / O module 140, card reader 160, and electric lock 170 via electric wires 180. The entrance / exit controller 150 has a card verification and discrimination function, and performs card verification and discrimination using card information read by the card reader 160, and controls the locking and unlocking of the electric lock 170. Furthermore, the entrance / exit controller 150 receives electric control signals from the I / O module 140 and controls the locking and unlocking of the electric lock 170. The card reader 160 is a reader of magnetic cards, contactless IC cards, etc., and transmits the read information to the entrance / exit controller 150. The electric lock 170 is a lock that is electrically controlled to open and close.

[0019] <Device configuration> (Configuration of imaging device) Next, a description will be given of the configuration of the imaging device 110. Fig. 2 is a diagram showing an example of the hardware configuration of the imaging device 110. The imaging device 110 includes, as its hardware configuration, for example, an imaging unit 201, an image processing unit 202, an arithmetic processing unit 203, a distribution unit 204, and an SD I / F unit 205. Note that I / F is an abbreviation for interface.

[0020] The imaging unit 201 includes a lens unit for focusing light into an image, and an imaging element for converting the focused light into an analog signal. The lens unit has a zoom function for adjusting the angle of view, an aperture function for adjusting the amount of light, and the imaging element has a gain function for adjusting the sensitivity when converting light into an analog signal. These functions are adjusted based on setting values ​​notified by the image processing unit 202. The analog signal acquired by the imaging unit 201 is converted into a digital signal by an analog-to-digital conversion circuit, and transferred to the image processing unit 202 as an image signal.

[0021] The image processing unit 202 is configured to include an image processing engine and its peripheral devices. The peripheral devices include, for example, a RAM (Random Access Memory) and drivers for each I / F. The image processing unit 202 generates image data by performing image processing such as development, filtering, sensor correction, and noise removal on the image signal acquired from the imaging unit 201. The image processing unit 202 can also transmit setting values ​​to the lens unit and the imaging element and perform exposure adjustment so that an appropriately exposed image can be acquired. The image data generated by the image processing unit 202 is transferred to the arithmetic processing unit 203.

[0022] The arithmetic processing unit 203 is composed of one or more processors such as a CPU or MPU, memories such as RAM or ROM, drivers for each I / F, etc. Note that CPU is an acronym for Central Processing Unit, MPU is an acronym for Micro Processing Unit, RAM is an acronym for Random Access Memory, and ROM is an acronym for Read Only Memory. In one example, the arithmetic processing unit 203 determines which of the imaging device 110, the removable device 100, and the server 130 will perform each part of the processing to be executed in the above-mentioned system, and executes the processing corresponding to the determined allocation. The content of this processing and the details of the allocation of processing will be described later. The image received from the image processing unit 202 or the processing results of the arithmetic processing unit 203 are transferred to the distribution unit 204 or the SD I / F unit 205.

[0023] The distribution unit 204 includes a network distribution engine and peripheral devices such as RAM and an ETH PHY module. The ETH PHY module executes processing at the physical (PHY) layer of Ethernet. The distribution unit 204 converts image data and processing result data acquired from the arithmetic processing unit 203 into a format that can be distributed to the network 120, and outputs the converted data to the network 120. The SD I / F unit 205 is an interface for connecting to the removable device 100 and includes, for example, a power supply and an attachment mechanism such as a detachable socket for attaching and detaching the removable device 100. Here, it is assumed that the SD I / F unit 205 is configured in accordance with the SD standard established by the SD Association. Communication between the removable device 100 and the imaging device 110, such as transferring images acquired from the arithmetic processing unit 203 to the removable device 100 and acquiring data from the removable device 100, is performed via the SD I / F unit 205.

[0024] Fig. 3(a) shows an example of the functional configuration of the imaging device 110. Fig. 3(b) shows a detailed configuration of the analysis unit 305. The imaging device 110 includes, as its functions, an imaging control unit 301, a signal processing unit 302, a storage unit 303, a control unit 304, an analysis unit 305, a device communication unit 306, and a network communication unit 307, for example.

[0025] The imaging control unit 301 executes control to capture an image of the surrounding environment via the imaging unit 201. The signal processing unit 302 performs predetermined processing on the image captured by the imaging control unit 301 to generate captured image data. Hereinafter, this captured image data will be simply referred to as a "captured image." The signal processing unit 302 encodes, for example, the image captured by the imaging control unit 301. The signal processing unit 302 encodes a still image using an encoding method such as JPEG (Joint Photographic Experts Group). The signal processing unit 302 also encodes a moving image using an encoding method such as H.264 / MPEG-4 AVC (hereinafter referred to as "H.264") or HEVC (High Efficiency Video Coding). The signal processing unit 302 may also encode an image using an encoding method selected by a user from a plurality of preset encoding methods, for example, via an operation unit (not shown) of the imaging device 110.

[0026] The storage unit 303 stores a list of analysis processes that can be executed by the analysis unit 305. The storage unit 303 also stores the results of the analysis processes described below. The control unit 304 controls the signal processing unit 302, the storage unit 303, the analysis unit 305, the device communication unit 306, and the network communication unit 307 so that each unit executes a predetermined process.

[0027] The analysis unit 305 selectively performs at least one of pre-analysis processing 308, face detection processing 309, facial feature extraction processing 310, facial feature matching 311, and post-analysis processing 312, which will be described later, on the captured image.

[0028] The pre-analysis process 308 is a process executed on the captured image before executing the face detection process described later. In the pre-analysis process 308 of this embodiment, as an example, the captured image is divided to create divided images.

[0029] The face detection process 309 is a process that performs face detection on the divided image obtained in the analysis pre-processing 308, and outputs face position area information that is the area of ​​the detected face, and face likelihood that is the likelihood of the face (likelihood of the face). The face detection process 309 can be a process configured to output the position of the face in the divided image using, for example, a machine learning model.

[0030] The facial feature extraction process 310 is a process for cropping a facial image area from the divided image and extracting facial features using the face position area information output by the face detection process 309. The extracted facial features are output as multidimensional vectors.

[0031] The facial feature matching process 311 is a process for matching a facial feature list for a plurality of facial images registered in an image registration unit (described later) previously acquired by the imaging device 110 from the server 130 with the facial features extracted by the facial feature extraction process 310. In the facial feature matching process 311 of this embodiment, as an example, the distance between the facial features (multidimensional vector) of the registered image and the extracted facial features (multidimensional vector) is calculated, and if the distance is equal to or greater than a predetermined threshold, the matching is successful. If the distance is less than the predetermined threshold, the matching is unsuccessful.

[0032] The post-analysis processing 312 is a processing for filtering the likelihood (face-likeliness) of faces detected by the face detection processing 309 using a certain threshold and integrating overlapping face position areas based on the face likelihood values. A list of combinations of face position area information and face likelihood after the post-analysis processing 312 is performed is hereinafter referred to as metadata.

[0033] The device communication unit 306 communicates with the removable device 100. The device communication unit 306 converts input data into a format that can be processed by the removable device 100 and transmits the data obtained by the conversion to the removable device 100. The device communication unit 306 also receives data from the removable device 100 and converts the received data into a format that can be processed by the imaging apparatus 110. In this embodiment, the device communication unit 306 performs conversion processing to convert decimal numbers between floating-point format and fixed-point format, but this is not limited to this, and other processing may be performed by the device communication unit 306. In this embodiment, the device communication unit 306 communicates with the removable device 100 by transmitting a command sequence defined in advance within the scope of the SD standard to the removable device 100 and receiving a response from the removable device 100. The network communication unit 307 communicates with the server 130 via the network 120.

[0034] The server load status acquisition unit 313 acquires (load acquisition) load information indicating the load status of the server from the server 130 through the network communication unit 307. The image processing control unit 314 determines whether each image process should be performed by the imaging device 100, the removable device 100, or the server 130, based on the server load status and the image processing capabilities of each device described below.

[0035] (Removable Device Configuration) 4 shows an example of the hardware configuration of the removable device 100. The removable device 100 includes, for example, an I / F unit 401, an FPGA 402, an SD controller 403, and a storage unit 404. The removable device 100 is formed in a shape that can be inserted into and removed from a removable socket of the SD I / F unit 205 of the imaging device 110, that is, in a shape that complies with the SD standard.

[0036] The I / F unit 401 is an interface section for connecting devices such as the imaging device 110 to the removable device 100. The I / F unit 401 is configured to include, for example, electrical contact terminals and the like that receive power from the imaging device 110 and generate and distribute power used within the removable device 100. Like the SD I / F unit 205 of the imaging device 110, the I / F unit 401 complies with items defined in the SD standard. Receiving images and setting data from the imaging device 110 and transmitting data from the FPGA 402 to the imaging device 110 are performed via the I / F unit 401.

[0037] The FPGA 402 includes an input / output control unit 410, a processing switching unit 411, and an arithmetic processing unit 412. The FPGA 402 is a type of semiconductor device that can repeatedly reconfigure its internal logic circuit structure. The processing performed by the FPGA 402 allows additional processing functions to be added to the device to which the removable device 100 is attached. Furthermore, the reconfiguration function of the FPGA 402 allows the logic circuit structure to be changed later. Therefore, for example, by attaching the removable device 100 to a device in a field where technology is rapidly evolving, the device can execute appropriate processing at the appropriate time. While this embodiment describes an example in which an FPGA is used, a general-purpose ASIC or a dedicated LSI may also be used as long as it can implement the processing described below. The FPGA 402 is activated when configuration data containing information about the logic circuit structure to be generated is written via a dedicated interface, or when the configuration data is read from the dedicated interface. In this embodiment, the configuration data is stored in the storage unit 404. When powered on, the FPGA 402 reads the configuration data from the storage unit 404, generates a logic circuit, and activates the FPGA 402. However, the present invention is not limited to this. For example, the imaging device 110 may write the setting data to the FPGA 402 via the I / F unit 401 by implementing a dedicated circuit in the removable device.

[0038] The input / output control unit 410 includes a circuit for transmitting and receiving images to and from the imaging device 110, a circuit for analyzing commands received from the imaging device 110, and a circuit for performing control based on the analysis results. The commands are defined in the SD standard, and the input / output control unit 410 can detect some of them. Details of the functions will be described later. The input / output control unit 410 controls the image to be sent to the SD controller 403 in the case of storage processing, and to be sent to the arithmetic processing unit 412 in the case of image analysis processing. Furthermore, upon receiving setting data for switching processing, the input / output control unit 410 transmits the setting data to the processing switching unit 411. The processing switching unit 411 includes a circuit for acquiring information about the image analysis processing function from the storage unit 404 based on the setting data received from the imaging device 110 and writing the information to the arithmetic processing unit 412. The information about the image analysis processing function is, for example, setting parameters indicating the order and type of calculations to be performed in the arithmetic processing unit 412, the coefficients of the calculations, and the like. The arithmetic processing unit 412 includes multiple arithmetic circuits required to execute the image analysis processing function. The arithmetic processing unit 412 executes each arithmetic processing based on the information on the image analysis processing function received from the processing switching unit 411, and transmits the processing results to the imaging device 110 and / or records the processing results in the storage unit 404. In this way, the FPGA 402 extracts setting data for the processing function to be executed, which is included in the setting data corresponding to the multiple processing functions stored in advance, and rewrites the processing content to be executed by the arithmetic processing unit 412 based on the extracted setting data. This allows the removable device 100 to selectively execute at least one of the multiple processing functions. Furthermore, by adding setting data for a newly added process as needed, the imaging device 110 can execute the latest process. Note that, hereinafter, having multiple setting data corresponding to multiple processing functions is referred to as having multiple processing functions. In other words, even if the FPGA 402 of the removable device 100 is configured to execute one processing function, if the processing content of the arithmetic processing unit 412 can be changed using setting data for another processing function, this is referred to as having multiple processing functions.

[0039] The SD controller 403 is a known control IC (integrated circuit) as defined in the SD standard, and controls slave operation of the SD protocol and reads and writes data from and to the storage unit 404. The storage unit 404 is configured, for example, with a NAND flash memory, and stores various information such as memory data written from the imaging device 110, information on the image analysis processing function written to the arithmetic processing unit 412, and setting data for the FPGA 402.

[0040] Fig. 5(a) shows an example of the functional configuration of the removable device 100. Fig. 5(b) shows a detailed configuration of the analysis unit 501. The functional configuration of the removable device 100 includes, for example, the analysis unit 501 and a communication unit 502. The analysis unit 501 performs analysis processing on an image.

[0041] The analysis unit 501 executes analysis processing on an image. In this embodiment, the analysis unit 501 executes face detection processing 503, facial feature extraction processing 504, and facial feature matching processing 505, but is not limited to this. Note that the face detection processing 503 is equivalent to the face detection processing 309 described above, the facial feature extraction processing 504 is equivalent to the facial feature extraction processing 310 described above, and the facial feature matching processing 505 is equivalent to the facial feature matching processing 311 described above, and therefore description thereof will be omitted. However, processing performance may differ depending on the hardware performance and the algorithm of the analysis processing being executed. The communication unit 502 communicates with the image capture device 110 via the I / F unit 401.

[0042] (Server configuration) Fig. 6 shows an example of the hardware configuration of the server 130. The server 130 is configured as a computer such as a general PC, and, for example, as shown in Fig. 6, includes a processor 601 such as a CPU, memory such as a RAM 602 and a ROM 603, a storage device such as an HDD 604, and a communication I / F 605. The server 130 can perform various functions by the processor 601 executing programs stored in the memory or storage device.

[0043] FIG. 7A shows an example of the functional configuration of the server 130 according to this embodiment. FIG. 7B shows a detailed configuration of the analysis unit 705. The server 130 includes, as its functional configuration, a network communication unit 701, a control unit 702, a display unit 703, and an operation unit 704. The network communication unit 701 is connected to, for example, the network 120 and communicates with an external device such as the image capture device 110 via the network 120. Note that this is merely an example, and the network communication unit 701 may be configured to establish a direct connection with the image capture device 110 and communicate with the image capture device 110 without going through the network 120 or another device. The control unit 702 controls the network communication unit 701, the display unit 703, and the operation unit 704 to perform their respective processes. The display unit 703 presents information to the user via, for example, a display. In this embodiment, information is presented to the user by displaying the results of rendering by the browser on the display. Note that information may also be presented by methods other than a screen display, such as sound or vibration. The operation unit 704 accepts operations from the user. In this embodiment, the operation unit 704 is a mouse or keyboard, which the user operates to input user operations into the browser. However, the operation unit 704 is not limited to this, and may be any device capable of detecting the intention of other users, such as a touch panel or a microphone.

[0044] The analysis unit 705 executes analysis processing on an image. In this embodiment, the analysis processing that can be executed is assumed to be, but is not limited to, pre-analysis processing 707, face detection processing 708, facial feature extraction processing 709, facial feature matching processing 710, and post-analysis processing 711. Note that the processes that can be executed by the analysis unit 705 are the same as the processes that can be executed by the analysis unit 305. However, the processing performance may differ depending on the performance of the hardware being executed and the algorithm of the analysis processing.

[0045] The image registration unit 706 manages face images for matching with face features analyzed from a captured image in the face feature matching process 710. The server load monitoring unit 707 monitors the load state of the server .

[0046] <Processing flow> Next, an example of the flow of processing executed within the system will be described. Of the following processes, the processing executed by the imaging device 110 is realized by the processor in the arithmetic processing unit 203 executing a program stored in memory or the like. The processing executed by the removable device 100 is realized by the processor in the arithmetic processing unit 412 executing a program stored in memory or the like. Furthermore, the processing executed by the server 130 is realized by the processor 601 executing a program stored in RAM 602. However, this is merely an example, and some or all of the processing described below may be realized by dedicated hardware.

[0047] (Overall flow) FIG. 8 is a diagram showing an outline of a series of steps in the image analysis process executed by the image analysis system 101. As shown in FIG.

[0048] In S801, the imaging device 110 detects that the user has attached the removable device 100. In S802, the imaging device 110 executes an initialization sequence for the removable device 100. In this initialization sequence, predetermined commands are transmitted and received between the imaging device 110 and the removable device 100. This enables the imaging device 110 to use the removable device 100.

[0049] In S803, the image analysis system 101 performs face recognition processing on the captured image acquired by the imaging device 110, and unlocks the electronic lock 170. Here, the face recognition processing is processing that is realized by using multiple sub-processes (face detection processing function, facial feature extraction processing function, facial feature matching processing function). The face recognition processing is executed by any of the imaging device 110, the removable device 100, and the server 130. Details of the allocation of this face recognition processing will be described later. Furthermore, the processing of S803 is repeatedly executed on the captured images acquired by the imaging device 110 (each frame image that constitutes a moving image).

[0050] (Flow of face recognition processing and electronic lock unlocking processing (S803)) 9 is a diagram showing an example of the flow of face authentication processing and electronic lock unlocking processing (S803). This shows the processing from when the image capturing device 110, the removable device 100, and the server 130 perform face authentication processing on an image captured by the image capturing device 110, to when the electric key 170 unlocks the door.

[0051] The processes shown in steps S901 to S907, S909 to S914, S916 to S919, and S920 to S922 are processes executed by the imaging apparatus 110. The processes shown in steps S908, S915, and S920 are processes executed by the removable device 100. The processes shown in steps S931 to S942 are processes executed by the server 130. The process shown in step S951 is processes executed by the entrance / exit controller 150.

[0052] In S901, the imaging control unit 301 captures an image of the surrounding environment. For example, the control unit 304 controls the signal processing unit 302 to process the image captured by the imaging control unit 301 and acquire the captured image.

[0053] In S902, the control unit 304 determines whether each process of the face recognition function group should be executed by the image capture device 110, the removable device 100, or the server 130. Details of this determination method will be described later. In this embodiment, it is assumed that the removable device 100 is capable of executing face detection processing 503, facial feature extraction processing 504, and facial feature matching processing 505. It is also assumed that the image capture device 110 is capable of executing pre-analysis processing 308, face detection processing 309, facial feature extraction processing 310, facial feature matching processing 311, and post-analysis processing 312. It is also assumed that the server 130 is capable of executing pre-analysis processing 707, face detection processing 708, facial feature extraction processing 709, facial feature matching processing 710, and post-analysis processing 711.

[0054] In S903, the control unit 304 transitions to S904 if the face detection process is to be executed locally (on the image capture device 110 or the removable device 100), or transitions to S905 if the face detection process is to be executed remotely (on the server 130).

[0055] In S904, the control unit 304 controls the analysis unit 305 to perform pre-analysis processing on the captured image input from the control unit 304, and acquires an image resulting from the pre-analysis processing.

[0056] In S905, the control unit 304 controls the network communication unit 307 to transmit the captured image acquired in S901 to the server 130. As an example, the imaging device 110 transmits information to the server 130 by transmitting a response message to the request message defined in the ONVIF standard to the server 130. However, the present invention is not limited to this, and information may be transmitted by other messages or the like.

[0057] In S906, the control unit 304 transitions to S907 if the face detection process is to be executed by the image capture device 110. On the other hand, if the face detection process is to be executed by a removable device, the control unit 304 transitions to S908.

[0058] In S907, the control unit 304 controls the analysis unit 305 to execute face detection processing 309 on the image resulting from the pre-analysis process, and acquires face position area information, which is the area of ​​the detected face, and the likelihood of the face.

[0059] In S908, the analysis unit 501 executes face detection processing 503 on the image resulting from the pre-analysis processing. Meanwhile, in S909, the control unit 304 controls the analysis unit 305 to execute post-analysis processing 312 and create metadata.

[0060] In S910, the control unit 304 determines whether or not there is a face detection result in the metadata. If there is a face detection result, the process proceeds to S908. If there is no face detection result, the flow ends.

[0061] In S911, the control unit 304 transitions to S912 if the facial feature extraction process is to be executed remotely (by the server 130), or to S913 if the process is to be executed locally (by the imaging device 110 or the removable device 100).

[0062] In S912, the control unit 304 controls the network communication unit 307 to transmit the captured image acquired in S901 and the metadata created in S909 to the server 130.

[0063] In S913, the control unit 304 transitions to S914 if the facial feature extraction process is to be executed by the image capture device 110. If the process is to be executed by a removable device, the control unit 304 transitions to S915.

[0064] In S914, the control unit 304 controls the analysis unit 305 to execute the facial feature extraction process 310 and extract the facial features of the detected face. Meanwhile, in S915, the analysis unit 501 executes the facial feature extraction process 504 and extracts the facial features of the detected face.

[0065] In S916, the control unit 304 transitions to S917 if the facial feature matching process is to be executed remotely (by the server 130), or transitions to S918 if the facial feature matching process is to be executed locally (by the imaging device 110 or the removable device 100).

[0066] In S917, the control unit 304 controls the network communication unit 307 to transmit the facial features extracted in S914 or S915 to the server 130.

[0067] In S918, the control unit 304 transitions to S919 if the facial feature matching process is to be executed by the imaging device 110. If the facial feature matching process is to be executed by a removable device, the control unit 304 transitions to S920.

[0068] In S919, the control unit 304 controls the analysis unit 305 to execute the facial feature matching process 311. On the other hand, in S920, the analysis unit 501 executes the facial feature matching process 311.

[0069] In S921, the control unit 304 transitions to S922 if the result of the facial feature matching process is matching OK. If the result of the facial feature matching process is matching NG, the flow ends. In S922, the control unit 304 controls the network communication unit 307 to send an unlock event to the I / O module 140.

[0070] In S931, the control unit 702 of the server 130 controls the network communication unit 701 to receive data from the imaging device 110. That is, the control unit 702 receives any one of a captured image (if transmitted in S905), a captured image and metadata (if transmitted in S912), or facial features (if transmitted in S917). In S932, the control unit 702 transitions to S939 if the received data is facial features. Otherwise, the control unit 702 transitions to S932. In S933, the control unit 702 transitions to S935 if the received data is a captured image and metadata. Otherwise (i.e., the received data is a captured image), the control unit 702 transitions to S934.

[0071] In S934, the control unit 702 controls the analysis unit 705 to execute the pre-analysis process 707 on the received captured image and obtain an image as a result of the pre-analysis process. On the other hand, in S935, the control unit 702 controls the analysis unit 705 to execute the pre-analysis process 707 on the received captured image and obtain an image as a result of the pre-analysis process.

[0072] In S936, the control unit 702 controls the analysis unit 705 to execute face detection processing 708 on the image resulting from the pre-analysis processing. In S937, the control unit 702 controls the analysis unit 705 to execute post-analysis processing 711 and create metadata. In S938, the control unit 702 determines whether or not there is a face detection result in the metadata. If there is a face detection result, the process proceeds to S939. If there is no face detection result, the flow ends.

[0073] In S939, the control unit 702 controls the analysis unit 705 to execute the facial feature extraction process 709. In S940, the control unit 702 controls the analysis unit 705 to execute the facial feature matching process 710 and obtains the matching result.

[0074] In S941, the control unit 702 determines whether the likelihood of a face exceeds a predetermined threshold. If the likelihood exceeds the threshold, matching is successful and the process proceeds to S942. If the likelihood does not exceed the threshold, matching is unsuccessful and the flow ends. In S942, the control unit 702 controls the network communication unit 701 to send an unlock event to the I / O module 140.

[0075] In S951, when the I / O module 140 receives an HTTP event (unlock event) from the imaging device 110 or the server 130, it transmits an electrical control signal to the entrance / exit controller 150 via the electric wire 180. The entrance / exit controller 150 that has received the electrical control signal unlocks the electric lock 170.

[0076] (Flow of image processing execution location determination process (S902)) 10 is a diagram showing an example of a table used for determining the image processing execution location. In the following description, image processing capability refers to an index indicating whether each image processing can be executed. On the other hand, hardware capability refers to an index of the processing speed of each image processing.

[0077] The server table 1001 indicates the image processing capabilities of the server 130, indicating whether each type of image processing is possible. The table indicates that the server 130 is capable of processing pre-analysis processing 707, face detection processing 708, facial feature extraction processing 709, facial feature matching processing 710, and post-analysis processing 711.

[0078] The image capture device table 1002 indicates the image processing capabilities of the image capture device 110. The table indicates that the image capture device 110 is capable of performing pre-analysis processing 308, face detection processing 309, facial feature extraction processing 310, facial feature matching processing 311, and post-analysis processing 312.

[0079] The removable device table 1003 indicates the image processing capabilities of the removable device 100. This indicates that the removable device 100 is capable of processing face detection processing 503, facial feature extraction processing 504, and facial feature matching processing 505.

[0080] The load state information 1004 indicates the load state of the server 130. The process of grasping the load state of the server is performed by the server load grasping unit 712. The load state of the server 130 may be, for example, a low load state 1005, a medium load state 1006, or a high load state 1007.

[0081] The low load state 1105 is a state in which the server 130 is simultaneously processing zero or very few analysis processes (less than a predetermined load), and the analysis processing speed of the server 130 is ideal for each analysis process. The medium load state 1006 is a state in which the server 130 is simultaneously executing multiple analysis processes (above a predetermined load), and the analysis processing speed is slower than in the ideal state. The high load state 1007 is a state in which the server 130 is simultaneously executing multiple analysis processes (above a predetermined load), and there are analysis processes in the waiting queue. When an analysis request is made in the high load state 1007, the process is queued and waits until the process previously queued is finished before being executed.

[0082] The metrics for understanding the server load are not limited to this, and may be calculated from, for example, the average analysis speed over a certain period of time, or from the number of connected cameras or the number of people passing by.

[0083] Hardware capability information 1008 indicates the hardware capability of the imaging device 110. The low-speed imaging device 1009 is slower at performing each image processing than the high-speed imaging device 1010. Conversely, the high-speed imaging device 1010 is faster at performing each image processing than the low-speed imaging device 1009.

[0084] Hardware capability information 1011 indicates the hardware capability of the removable device 100. Not inserted 1012 indicates a state in which the removable device 100 is not inserted into the imaging device and image processing cannot be performed by the removable device 100. The slow removable device 1013 is slower in the speed at which each image processing is performed than the high-speed removable device 1014. Conversely, the high-speed removable device 1014 is faster in the speed at which each image processing is performed than the slow removable device.

[0085] The determination table 1015 is a table that describes where each image processing is to be executed according to the load status information 1004, the hardware capability information 1008, and the hardware capability information 1011. That is, the determination table 1015 is a determination table that uniquely determines where the image processing is to be executed among the imaging device 110, the removable device 100, and the server 130. Note that the determination table 1015 specifies where each image processing is to be executed, depending on at least the load status information 1004. Furthermore, the determination table 1015 may be dynamically configured based on the server table 1001, the imaging device table 1002, and the removable device table 1003.

[0086] FIG. 11 is a diagram showing an example of the flow of the image processing execution location determination process (S902).

[0087] In S1101, the control unit 304 of the imaging device 110 controls the network communication unit 701 to acquire the server table 1001 from the server 130. In S1102, the control unit 304 acquires the imaging device table 1002. In S1103, the control unit 304 acquires the removable device table 1003. That is, image processing capability information of each of the server 130 and the imaging device 110 is acquired (capability acquisition).

[0088] In S1101 to S1103, the control unit 304 may confirm that there is no inconsistency between the acquired table and the judgment table 1015. That is, it may confirm that no unprocessable analysis process is included in the judgment table 1015. If the judgment table 1015 includes an unprocessable analysis process, an error is output and the process is terminated, or the judgment table 1015 is corrected so that no inconsistency occurs.

[0089] In S1104, the server load status acquisition unit 313 controls the network communication unit 701 to make a server load status acquisition request to the server 130. The server 130 performs server load recognition processing in the server load recognition unit 712 and returns load status information 1004 to the imaging device 110. As a result, the imaging device 110 acquires the load status information 1004 from the server 130.

[0090] In S1105, the control unit 304 acquires the hardware capability information 1008 of the imaging device 110. In S1106, the control unit 304 acquires the hardware capability information 1011 of the removable device 100.

[0091] In S1107, the image processing control unit 314 determines the device that will execute each image processing operation in accordance with the determination table 1015, based on the acquired load status information 1004, hardware capability information 1008, and hardware capability information 1011. That is, it determines whether each image processing operation that constitutes the face authentication processing will be performed by the imaging device 110, the removable device 100, or the server 130.

[0092] As described above, according to the first embodiment, in the image analysis system 101 including the imaging device 110, the removable device 100, and the server 130, the device (location) that will perform processing is determined based on at least the load state of the server 130. Furthermore, the device that will perform processing is determined based on the image processing capabilities and hardware capabilities of each device in addition to the load state of the server. This makes it possible to efficiently utilize the resources of the entire system and optimize the processing speed of the entire system.

[0093] (Second embodiment) In the second embodiment, information on the execution time of each process in the server 130, the imaging device 110, and the removable device 100 is acquired, and a process for determining where each process of the face recognition function group is to be performed is described. The system configuration and each device configuration (FIGS. 1 to 7) and the overall processing flow (FIG. 8) are the same as those in the first embodiment, and therefore a description thereof will be omitted. Below, differences from the first embodiment are described.

[0094] <Processing flow> (Flow of image processing execution location determination process (S902)) FIG. 12 is a diagram showing an example of the load / execution time of each device.

[0095] The load status information 1201 is information about the load status of the server 130. The process of grasping the load status information 1201 is performed by the server load grasping unit 712. The load status of the server 130 is indicated by the total execution time and processing wait time of each process (here, pre-analysis processing 707, face detection processing 708, facial feature extraction processing 709, facial feature matching processing 710, and post-analysis processing 711). Note that the server load grasping unit 712 determines the load status of the server 130 based on, for example, the total execution time and processing wait time of each process executed most recently.

[0096] Records 1202 to 1204 shown in the load state information 1201 show typical examples of execution times under three load states: low load, medium load, and high load. Record 1205 shows an example of the number of queued processes corresponding to a high load.

[0097] Record 1202 indicates that the execution time required for pre-analysis processing 707 was 10 ms, for face detection processing 708 was 50 ms, for facial feature extraction processing 709 was 30 ms, for facial feature matching processing 710 was 20 ms, and for post-analysis processing 711 was 5 ms. Note that the processing wait time is assumed to be 0. Record 1203 indicates that the execution time required for pre-analysis processing 707 was 20 ms, for face detection processing 708 was 100 ms, for facial feature extraction processing 709 was 60 ms, for facial feature matching processing 710 was 40 ms, and for post-analysis processing 711 was 10 ms. Note that the processing wait time is assumed to be 0.

[0098] Record 1204 indicates that the execution time required for pre-analysis processing 707 was 30 ms, for face detection processing 708 was 150 ms, for facial feature extraction processing 709 was 90 ms, for facial feature matching processing 710 was 60 ms, and for post-analysis processing 711 was 15 ms. Record 1205 indicates the number of processes waiting in the processing queue during high load. The high-load processing waiting time is calculated from records 1204 and 1205.

[0099] Records 1207 and 1208 shown in the imaging device information 1206 show typical examples of execution times corresponding to two processing performance levels: low speed and high speed.

[0100] Record 1207 indicates that the execution time required for pre-analysis processing 308 was 200 ms, for face detection processing 309 was 5000 ms, for facial feature extraction processing 310 was 3000 ms, for facial feature matching processing 311 was 2000 ms, and for post-analysis processing 312 was 100 ms. Record 1208 indicates that the execution time required for pre-analysis processing 308 was 20 ms, for face detection processing 309 was 1000 ms, for facial feature extraction processing 310 was 600 ms, for facial feature matching processing 311 was 400 ms, and for post-analysis processing 312 was 10 ms.

[0101] Records 1210 to 1212 shown in the removable device information 1209 show typical examples of execution times corresponding to three states: not inserted, low speed, and high speed. As shown in record 1210, the execution times when the removable device 100 is not inserted are all indicated by "x" (indicating unavailable).

[0102] Record 1211 indicates that the execution time required for the face detection process 503 was 200 ms, the execution time required for the facial feature extraction process 504 was 120 ms, and the execution time required for the facial feature matching process 505 was 80 ms. Record 1212 indicates that the execution time required for the face detection process 503 was 100 ms, the execution time required for the facial feature extraction process 504 was 60 ms, and the execution time required for the facial feature matching process 505 was 40 ms.

[0103] 13 is a diagram showing an example of the flow of the image processing execution location determination process (S902) in the second embodiment. The processes from S1104 onwards in the first embodiment (FIG. 11) are replaced with the processes from S1301 onwards.

[0104] In S1301, the imaging device 110 acquires load status information from the server 130. In this process, the control unit 304 of the imaging device 110 controls the network communication unit 701 to make a server load status acquisition request to the server 130. The server 130 controls the server load grasping unit 712 to perform server load grasping processing and responds with load status information 1201 to the imaging device 110. For example, if the server 130 is under a low load, the information indicated in record 1202 is sent as a response.

[0105] In S1302, the control unit 304 acquires imaging device information of the imaging device 110. For example, the control unit 304 acquires, as the imaging device information, the execution time of each process in the analysis unit 305 of the imaging device 110. For example, if the processing speed of the imaging device 110 is low, the information indicated in record 1207 is transmitted as a response.

[0106] In S1303, the control unit 304 acquires removable device information of the removable device 100. For example, the control unit 304 acquires the execution time of each process in the removable device 100 as the removable device information. For example, if the processing speed of the removable device 100 is low, the information indicated in record 1211 is transmitted as a response.

[0107] In S1304, the control unit 304 determines the device that will execute each image process based on the acquired load status information, imaging device information, and removable device information of the server 130. That is, it determines whether each image process that constitutes the face authentication process will be processed by the imaging device 110, the removable device 100, or the server 130. For example, it calculates the total execution time for each of multiple combinations of each image process that constitutes the face authentication process and the device (location) that executes it, and determines the combination that is the shortest (the combination that provides the fastest processing speed).

[0108] As described above, according to the second embodiment, in the image analysis system 101 including the imaging device 110, the removable device 100, and the server 130, the execution time of each process at least in the server 130 is acquired. Then, based on the acquired execution time of each process in the server 130, the device (location) that will perform the processing is determined. Furthermore, based on the execution time in the server 130, as well as the image processing capacity of each device and the execution time of each process in the imaging device 110 and the removable device 100, the device that will perform the processing is determined. This makes it possible to efficiently utilize the resources of the entire system, and to optimize the processing speed of the entire system.

[0109] (Third embodiment) In the third embodiment, the process of acquiring accuracy information (accuracy acquisition) of each process in the server 130, the imaging device 110, and the removable device 100 and determining where each process of the face recognition function group is to be performed will be described. Furthermore, the re-determination process performed according to the number of detections in the face detection results will also be described. The system configuration, the configuration of each device (FIGS. 1 to 7), and the overall processing flow (FIG. 8) are the same as those in the first and second embodiments, so their explanations will be omitted. Below, the parts that differ from the first and second embodiments will be described.

[0110] <Processing flow> (Flow of face recognition processing and electronic lock unlocking processing (S803)) Fig. 14 is a diagram showing an example of a table defining the capabilities of each device in the third embodiment. Tables 1401 to 1403 are used instead of tables 1001 to 1003 shown in Fig. 10 of the first embodiment. In the following description, image processing accuracy refers to an index indicating the degree of accuracy with which each image process can be executed.

[0111] The server table 1401 indicates the image processing capabilities and image processing accuracy of the server 130. It indicates that the server 130 is capable of processing pre-analysis processing 707, face detection processing 708, facial feature extraction processing 709, facial feature matching processing 710, and post-analysis processing 711. It also indicates that the server 130 processes face detection processing 708 with high accuracy.

[0112] The image capture device table 1402 indicates the image processing capabilities and image processing accuracy of the image capture device 110. It indicates that the image capture device 110 is capable of processing pre-analysis processing 308, face detection processing 309, facial feature extraction processing 310, facial feature matching processing 311, and post-analysis processing 312. It also indicates that the image capture device 110 processes the face detection precision processing 309 with low accuracy.

[0113] The removable device table 1403 indicates the image processing capabilities and image processing accuracy of the removable device 100. It indicates that the removable device 100 is capable of processing face detection processing 503, face feature extraction processing 504, and face feature matching processing 505. It also indicates that the removable device 100 processes face detection processing 503 with medium accuracy.

[0114] 15 is a diagram showing an example of the flow of face authentication processing and electronic lock unlocking processing (S803) in the third embodiment. Processing from S1501 to S1503 has been added to the first embodiment (FIG. 9).

[0115] In S1501, the control unit 304 of the imaging device 110 determines whether the likelihood of the face detection result in S910 is equal to or less than a threshold. If the likelihood is equal to or less than the threshold, the process proceeds to S905 to re-execute the face detection process 708 on the server 130, which is capable of performing face detection with higher accuracy. On the other hand, if the likelihood of the face detection result is greater than the threshold, the process proceeds to S1502.

[0116] In S1502, the control unit 304 determines whether the number of detected faces in the face detection result is multiple. If the number is multiple, the process proceeds to S1503. On the other hand, if the number of detected faces in the face detection result is not multiple (is one), the process proceeds to S911.

[0117] In S1503, the image processing control unit 314 determines that the first face detection result among the multiple face detection results will be processed at the image processing execution location determined in S902, while re-determining (re-determining) the image processing execution location for the other face detection results (the second and subsequent face detection results).

[0118] For example, if it is determined in S902 that the facial feature amount processing is to be performed by the removable device 100, the facial feature amount processing for the first face detection result is performed by the removable device 100, and the number of waiting processes for the facial feature amount processing for the removable device 100 is 1. Therefore, if multiple faces are detected in the face detection processing, the subsequent processing for the second and subsequent face detection results is performed by re-determining the location where image processing should be performed, taking into account the subsequent processing status for the first face detection result.

[0119] As described above, according to the third embodiment, the device (location) for performing processing is determined by taking image processing accuracy into consideration. This makes it possible to optimize the entire system, taking into account not only speed but also accuracy. Furthermore, if the number of faces detected in the face detection results is multiple, the image processing execution location is re-determined for the second and subsequent face detection results. This makes it possible to utilize the resources of the entire system more efficiently.

[0120] (Fourth embodiment) In the fourth embodiment, a form will be described in which the image processing execution location determination process is performed by the server 130. The system configuration and each device configuration (FIGS. 1 to 7) and the overall processing flow (FIG. 8) are the same as those of the first to third embodiments, so their explanation will be omitted. Below, the parts that differ from the first to third embodiments will be described.

[0121] <Processing flow> (Flow of face recognition processing and electronic lock unlocking processing (S803)) 16 is a diagram showing an example of the flow of face authentication processing and electronic lock unlocking processing (S803) in the fourth embodiment. As described above, the main difference is that the image processing execution location determination processing is performed by the server 130. Specifically, S1601 to S1604 are added instead of S902 and S1503 in the third embodiment.

[0122] In S1601, the control unit 304 of the imaging device 110 controls the network communication unit 701 to inquire of the server 130 about the location where image processing is to be performed.

[0123] In S1602, the server 130 performs server load grasping processing using the server load grasping unit 712, and further determines the image processing execution location from the status of each of the imaging devices 110a-d and removable devices a-d. That is, the server 130 differs from the first to third embodiments in that the image processing execution location is determined based on information about the multiple imaging devices (and removable devices) present in the system. The determination result is then transmitted to the imaging device 110. Note that the contents of the image processing execution location determination processing are the same as those in the first to third embodiments, and therefore description thereof will be omitted.

[0124] In S1603, the control unit 304 of the imaging device 110 controls the network communication unit 701 to again inquire of the server 130 about the image processing execution location.

[0125] In S1604, the server 130 performs server load grasping processing using the server load grasping unit 712, and further re-determines the image processing execution location based on the status of each of the image capturing devices 110a to 110d and the removable devices a to d. Then, the re-determination result is transmitted to the image capturing device 110.

[0126] As described above, according to the fourth embodiment, the image processing execution location determination process is performed by the server 130. This makes it possible to perform the image processing execution location determination process based on information on multiple image capture devices (and removable devices) present in the system. In other words, it becomes possible to perform the image processing execution location determination process corresponding to each individual image capture device (and removable device) while taking into account the balance of the entire system.

[0127] (Other Examples) Furthermore, in the above-described embodiment, image analysis processing has been described as an example of analysis processing, but the present invention can also be applied to audio analysis processing. For example, the present invention can be applied to processing for detecting audio patterns such as screams, gunshots, and the sound of glass breaking. For example, audio features are extracted using various audio data analysis methods such as spectral analysis, and the extracted features are compared with the detected audio pattern. Then, the degree of match is calculated, allowing a specific audio pattern to be detected.

[0128] Furthermore, when performing voice analysis processing, voice data is divided into voice data for a predetermined time, and voice analysis processing is performed using the voice data for the predetermined time as a unit. Furthermore, this predetermined time varies as appropriate depending on the voice pattern to be detected. Therefore, voice data for each time corresponding to the voice pattern to be detected is input to removable device 100. Removable device 100 has a function to analyze the input voice data and a function to store the input voice data.

[0129] Furthermore, in the above-described embodiment, the removable device 100 capable of non-temporarily storing data input from the imaging device 110 has been described as an example. However, in some embodiments, a removable device 100 that cannot non-temporarily store data input from the imaging device 110 may be used. That is, the removable device 100 may only perform analysis processing on the data input from the imaging device 110 and may not have the function of non-temporarily storing the data. In other words, the removable device 100 may have only the function of analysis processing, without being intended to be used for storing data like a normal SD card.

[0130] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0131] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0132] 100a to 100d: detachable devices, 110a to 110d: imaging devices, 120: network, 130: server, 201: imaging unit, 202: image processing unit, 203: arithmetic processing unit, 204: distribution unit, 205: SD I / F unit, 301: imaging control unit, 302: signal processing unit, 303: storage unit, 304: control unit, 305: analysis unit, 306: device communication unit, 307: network communication unit, 313: server load status acquisition unit, 314: image processing control unit

Claims

1. An information processing device, a processing means for performing an analysis process on the image data; a communication means for communicating with an external processing device capable of executing analysis processing on image data; a load acquisition means for acquiring load information relating to a load state of the external processing device; a determining means for determining whether the analysis processing is to be performed by the processing means or the external processing device based on the load information; and the determining means determines whether the analysis processing is to be performed by the processing means or the external processing device based on the load information and a determination table that defines a device that performs the analysis processing depending on at least a load state of the external processing device; The analysis process is composed of a plurality of sub-processes, The information processing device includes: a first capability acquisition means for acquiring a first image processing capability indicating which of the plurality of sub-processes the processing means is capable of executing; a second capability acquisition means for acquiring a second image processing capability indicating which of the plurality of sub-processes the external processing device is capable of executing; a correcting means for correcting the decision table based on the first image processing capability and the second image processing capability; Further having 1. An information processing device comprising:

2. The load state indicates one of a low load state in which the load state of the external processing device is less than a predetermined load, a medium load state in which the load is equal to or greater than the predetermined load but there is no waiting process, and a high load state in which the load is equal to or greater than the predetermined load and there is a waiting process.

2. The information processing apparatus according to claim 1, wherein:

3. the load state indicates a first execution time required for the analysis processing by the external processing device; the information processing device further includes information acquisition means for acquiring a second execution time required for the analysis process in the processing means, The determining means determines whether the analysis process is to be performed by the processing means or the external processing device based on the first execution time and the second execution time.

2. The information processing apparatus according to claim 1, wherein:

4. a first accuracy acquisition means for acquiring first accuracy information relating to the processing accuracy of the analysis processing in the external processing device; second accuracy acquisition means for acquiring second accuracy information relating to the processing accuracy of the analysis processing in the processing means; and The determining means determines whether the analysis process is to be performed by the processing means or the external processing device, further based on the first accuracy information and the second accuracy information.

4. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

5. The analysis process is a face recognition process consisting of a plurality of sub-processes, The determining means determines whether each of the plurality of sub-processes is to be performed by the processing means or the external processing device.

5. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

6. the plurality of sub-processes includes a face detection process for detecting a face of a person included in the image data; The determining means determines again whether the sub-processing subsequent to the face detection process should be performed by the processing means or the external processing device, based on the number of faces detected by the face detection process.

6. The information processing apparatus according to claim 5,

7. the plurality of sub-processes includes a face detection process for detecting a face of a person included in the image data; The determining means determines again whether a sub-processing subsequent to the face detection process is to be performed by the processing means or the external processing device, based on the likelihood of a face detection result obtained by the face detection process.

6. The information processing apparatus according to claim 5,

8. the information processing device further includes an imaging means, The processing means and the external processing device perform analysis processing on the image data obtained by the imaging means.

8. The information processing device according to claim 1, wherein the information processing device is a computer.

9. the processing means includes a first processing means included in the information processing device and a second processing means included in a removable device attached to the information processing device; The determining means determines whether the analysis process is to be performed by the first processing means, the second processing means, or the external processing device.

9. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

10. A processing system including an imaging device and a processing device connected to each other in a communicable manner, the imaging device has a first processing means for performing an analysis process on image data obtained by imaging, the processing device has a second processing means for executing a plurality of processes including an analysis process on image data obtained by imaging with the imaging device, one of the imaging device and the processing device has a determination means for determining whether analysis processing is to be performed by the first processing means or the second processing means based on a load state of the second processing means; the determination means determines whether the analysis processing is to be performed by the first processing means or the second processing means based on the load state and a determination table that defines a device that performs the analysis processing depending at least on the load state; The analysis process is composed of a plurality of sub-processes, Either the imaging device or the processing device a first capability acquisition means for acquiring a first image processing capability indicating which of the plurality of sub-processes the first processing means is capable of executing; a second capability acquisition means for acquiring a second image processing capability indicating which of the plurality of sub-processes the second processing means is capable of executing; a correcting means for correcting the decision table based on the first image processing capability and the second image processing capability; Further having A processing system comprising:

11. the processing system includes a plurality of imaging devices including the imaging device; The second processing means performs an analysis process on image data obtained by imaging with each of the plurality of imaging devices. The processing system of claim 10 .

12. A control method for an information processing device, comprising: The information processing device includes: a processing means for performing an analysis process on the image data; a communication means for communicating with an external processing device capable of executing analysis processing on image data; and The control method includes: a load acquisition step of acquiring load information relating to a load state of the external processing device; a determining step of determining whether the analysis processing is to be performed by the processing means or the external processing device based on the load information; Including, In the determination step, it is determined whether the analysis processing is to be performed by the processing means or the external processing device based on the load information and a determination table that defines a device that performs the analysis processing depending on at least a load state of the external processing device; The analysis process is composed of a plurality of sub-processes, The control method includes: a first capability acquisition step of acquiring a first image processing capability indicating which of the plurality of sub-processes the processing means is capable of executing; a second capability acquisition step of acquiring a second image processing capability indicating which of the plurality of sub-processes the external processing device is capable of executing; a modifying step of modifying the decision table based on the first image processing capability and the second image processing capability; Further includes A control method comprising:

13. A program for causing a computer to execute the control method according to claim 12.

Images