Imaging device, imaging control method, and program

Abstract

To simply identify a malfunction of the auxiliary processing device mounted on an imaging apparatus.SOLUTION: An imaging control device stores in advance a first signal information obtained by processing a captured first image. The imaging control device transmits the first image and processes the stored first image. The imaging control device determines whether or not the imaging control device is abnormal on the basis of a match between the first signal information and second signal information obtained by processing the first image.SELECTED DRAWING: Figure 25

Description

[Technical field]

[0001] The present invention relates to an imaging control device, an imaging system, an imaging control method, and a program. [Background technology]

[0002] In recent years, in various scenes, image analysis is being performed to detect, track, and estimate the attributes of objects using images captured by a surveillance camera. In addition, image processing such as estimating the number of objects is performed based on the image analysis results. Conventionally, high-performance computing devices such as PCs and servers perform image processing on surveillance camera images. Meanwhile, with the recent improvement in the processing power of mobile computing devices, surveillance cameras are now able to perform image processing on images. Image processing by a surveillance camera is performed, for example, by a computing device mounted on the camera body. Also, an auxiliary processing device such as a USB can be equipped with a computing device. A surveillance camera can be equipped with the above-mentioned auxiliary processing device, and a part of the image processing performed by the computing device of the surveillance camera can be executed by the computing device of the auxiliary processing device. However, when the auxiliary processing device malfunctions, a diagnosis is required to identify the cause of the malfunction, since the countermeasures differ depending on the cause of the malfunction. Meanwhile, there is a limit to the circuit size that can be implemented in an FPGA (Field Programmable Gate Array) of the auxiliary processing device. Patent Document 1 describes a technique for testing the internal memory of a microprocessor circuit and identifying the cause of a malfunction, in which a removable memory card is equipped with a test program and a memory space that serves as a substitute for the internal memory. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 10-55319 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there is a problem in that the user must insert a memory card into the imaging device in order to identify the malfunction of the FPGA.

[0005] An object of the present invention is to easily identify malfunctions of an auxiliary processing device mounted on an imaging device. [Means for solving the problem]

[0006] In order to achieve the object of the present invention, an imaging control device according to an embodiment of the present invention comprises the following configuration: an imaging device having an attachment part to which the device can be attached and detached, storing means for storing first signal information obtained by processing a captured first image, transmitting means for transmitting the first image to the device attached to the attachment part, and a transmission means for transmitting the first image to the device. but The first image analysis process This is the result of The device is characterized by comprising a receiving means for receiving second signal information, and a determining means for determining whether or not the device is abnormal based on whether or not the first signal information and the second signal information match. Effect of the Invention

[0007] According to the present invention, a malfunction of an auxiliary processing device mounted on an imaging device can be easily identified. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a configuration diagram of an imaging system according to an embodiment of the present invention. [Diagram 2] FIG. 1 is a diagram showing the configuration of an imaging apparatus according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a block diagram showing functions of the imaging apparatus according to the embodiment. [Figure 4] FIG. 2 is a diagram showing the configuration of an auxiliary processing device according to the embodiment. [Diagram 5] FIG. 2 is a block diagram showing the functions of the auxiliary processing device according to the embodiment; [Figure 6]FIG. 2 is a configuration diagram of an input / output device according to the embodiment. [Figure 7] FIG. 2 is a block diagram showing the functions of the input / output device according to the embodiment. [Figure 8] 4 is a flowchart of processing of the imaging system according to the present embodiment. [Figure 9] 4 is a flowchart of an analysis process according to the embodiment. [Figure 10] 6 is a flowchart for determining the content of an analysis process according to the embodiment. [Figure 11] 5 is a flowchart for executing an analysis process according to the embodiment. [Figure 12] 5 is a flowchart of post-processing according to the embodiment. [Figure 13] FIG. 4 is a diagram showing the configuration of a command and a response according to the embodiment. [Figure 14] 5A and 5B are diagrams showing data in an address storing information on processing functions according to the embodiment; [Figure 15] 5A and 5B are diagrams showing an example of information acquired by the imaging device according to the embodiment. [Figure 16] 6 is a flowchart showing a process for switching between a storage process and an image analysis process according to the embodiment. [Figure 17] 6 is a flowchart showing a process for switching between a storage process and an image analysis process according to the embodiment. [Figure 18] 6 is a flowchart showing a process for switching between a storage process and an image analysis process according to the embodiment. [Figure 19] 3 shows a user interface according to the present embodiment. [Figure 20] 4 is a user interface that displays the processing results according to the present embodiment. [Figure 21] 5A and 5B are diagrams showing an image analysis processing group for face recognition processing according to the embodiment. [Figure 22] 6 is a flowchart showing a selection process of a processing function according to the embodiment. [Figure 23] 6 is a flowchart showing a selection process of a processing function according to the embodiment. [Figure 24] 6 is a flowchart showing a selection process of a processing function according to the embodiment. [Diagram 25] 3 is a flowchart of a fault diagnosis process according to the embodiment. [Figure 26] 6 is a diagram showing a fault diagnosis result according to the embodiment displayed on a user interface. FIG. [Figure 27] FIG. 2 is a diagram showing the configuration of an auxiliary processing device according to the embodiment. [Figure 28] 3 is a flowchart of a fault diagnosis process according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0010] (First embodiment) The imaging system 10 according to this embodiment will be described below.

[0011] FIG. 1 shows an example of the configuration of an imaging system 10 of this embodiment. A case will be described where the imaging system 10 is a specific person tracking system. However, the imaging system 10 is not limited to the above, and may be applied to any system that outputs predetermined information by analyzing an image. The imaging system 10 includes an imaging device 110a, an imaging device 110b, an imaging device 110c, an imaging device 110d, a network 120, and an information processing device 130. The imaging system 10 also includes an auxiliary processing device 100a, an auxiliary processing device 100b, an auxiliary processing device 100c, and an auxiliary processing device 100d. Note that the imaging devices 110a to 110d each have a slot that allows a device (e.g., an SD card) capable of recording a captured image to be attached and detached. The auxiliary processing devices 100a to 100d are connected to each other by inserting the auxiliary processing devices 100a to 100d into the slots of the imaging devices. In the following, the auxiliary processing device 100a is used as a representative example of the auxiliary processing device, and the imaging device 110a is used as a representative example of the imaging device.

[0012] The auxiliary processing device 100a is a computing device that is detachable from the imaging device 110a. As an example, the auxiliary processing device 100a is a device in which a predetermined processing circuit is mounted on an SD card. The auxiliary processing device 100a is configured so that the entirety of the auxiliary processing device 100a can be inserted into the imaging device 110a according to the form of the SD card, for example. As a result, the auxiliary processing device 100a is connected to the imaging device 110 without any part protruding from the imaging device 110a. Since the auxiliary processing device 100a does not interfere with obstacles such as wiring around the imaging device 110a, the convenience of arranging the imaging device 110a is improved. In addition, since the imaging device 110a including a normal network camera has an SD card slot, the auxiliary processing device 100a can provide an extended function to the imaging device 110a. Note that the auxiliary processing device 100a may be a storage device capable of storing images captured by the imaging device 110a, other than the form of an SD card. For example, the auxiliary processing device 100a may have a USB interface, which may be configured to be attached to a USB socket of the imaging device 110a. Also, the predetermined processing circuit may be, for example, an FPGA programmed to execute a predetermined process, but may be in another form.

[0013] The imaging device 110a is an imaging device such as a network camera. In this embodiment, the imaging device 110a has a built-in arithmetic device capable of image processing, but is not limited to this. For example, there may be an information processing device (not shown) such as a PC connected to the imaging device 110a, and the combination of these may be the imaging device 110a. In this embodiment, each auxiliary processing device is installed for each imaging device. In FIG. 1, an auxiliary processing device is installed in each of four imaging devices, but the combination of imaging devices and auxiliary processing devices may be, for example, three or less, or five or more. By installing the auxiliary processing device 100a having an image analysis processing function in the imaging device 110a, the imaging device 110a can perform image processing even if the imaging device 110a does not have an image analysis processing function. In addition, when the imaging device 110a is equipped with an arithmetic device for image processing as in this embodiment, the auxiliary processing device 100a equipped with the arithmetic device is installed in the imaging device 110a, and the image processing that can be performed by the imaging device 110a can be diversified and advanced.

[0014] The information processing device 130 is a device that receives input from a user and outputs information to the user (for example, displays an image). In this embodiment, for example, the information processing device 130 is a computer such as a PC, and information is input and output by a browser or native application installed on the computer. Note that the information processing device 130 may be a portable information terminal in which a display unit and an input unit are integrated, such as a smartphone, tablet, or PDA.

[0015] All the imaging devices and the information processing device 130 are connected to each other so as to be able to communicate with each other via a network 120. The network 120 includes a plurality of routers, switches, cables, etc. that meet a communication standard such as Ethernet (registered trademark). In this embodiment, the network 120 may be any network that enables communication between all the imaging devices and the information processing device 130, and may be configured with any scale and communication standard. For example, the network 120 may be the Internet, a wired LAN, a wireless LAN, a WAN, etc. In addition, the network 120 may be configured to enable communication using a communication protocol that complies with the ONVIF (Open Network Video Interface Forum) standard, for example. However, the present invention is not limited to this, and the network 120 may be configured to enable communication using, for example, a unique communication protocol and other communication protocols.

[0016] FIG. 2 is a diagram showing an example of the configuration of the imaging device 110a. The imaging device 110a includes an imaging unit 201, an image processing unit 202, an arithmetic processing unit 203, a distribution unit 204, and an I / F unit 205. The imaging devices 110b to 110c include the same configuration as the imaging device 110a. The imaging unit 201 includes a lens for forming an image of light, and an imaging element for converting an analog signal according to the light formed by the lens. The lens has a zoom function for adjusting the angle of view, an aperture function for adjusting the amount of light, and the like. The imaging element has a gain function for adjusting the sensitivity when converting light into an analog signal. These functions are adjusted based on the setting values ​​notified by the image processing unit 202. The analog signal acquired by the imaging unit 201 is converted into a digital signal by an analog-digital conversion circuit, and transferred to the image processing unit 202 as an image signal.

[0017] The image processing unit 202 includes an image processing engine and its peripheral devices. The peripheral devices include, for example, a RAM and drivers for each I / F. The image processing unit 202 generates image data by performing image processing such as development processing, filter processing, sensor correction, and noise removal on the image signal acquired by the imaging unit 201. The image processing unit 202 also performs exposure adjustment by transmitting setting values ​​to the lens and the imaging element so that an appropriate exposure image can be acquired. The image data generated by the image processing unit 202 is transferred to the arithmetic processing unit 203.

[0018] The arithmetic processing unit 203 includes one or more processors such as a CPU and an MPU (Micro Processing Unit), memories such as a RAM and a ROM, and drivers for each I / F. The arithmetic processing unit 203 determines the share of processing to be performed by the imaging system 10 between the imaging device 110a and the auxiliary processing device 100a. The arithmetic processing unit 203 performs processing based on the determined share. Details of the processing content and the share of processing will be described later. The image data transmitted by the image processing unit 202 is transferred to the distribution unit 204 or the I / F unit 205. Data of the processing result by the arithmetic processing unit 203 is transferred to the distribution unit 204.

[0019] The distribution unit 204 includes a network distribution engine and peripheral devices such as a RAM and an ETH PHY module. The ETH PHY module is a module that executes processing of the physical (PHY) layer of Ethernet. The distribution unit 204 converts the image data and the processing result data acquired from the arithmetic processing unit 203 into a format that can be distributed to the network 120, and outputs the converted data to the network 120. The I / F unit 205 is an interface unit for connecting to the auxiliary processing device 100a. The I / F unit 205 includes, for example, a power source and an attachment mechanism such as a detachable socket for attaching and detaching the auxiliary processing device 100a. The I / F unit 205 may be configured according to the SD standard established by the SD Association. The I / F unit 205 can transfer image data acquired by the arithmetic processing unit 203 to the auxiliary processing device 100a, and can transfer data from the auxiliary processing device 100 to the imaging device 110a. That is, communication between the auxiliary processing device 100a and the image capture device 110a is performed via the I / F unit 205.

[0020] 3 is a block diagram showing the functions of the imaging device 110a. The imaging device 110a includes an imaging control unit 301, a signal processing unit 302, a storage unit 303, a control unit 304, an analysis unit 305, and a communication unit 306. The imaging devices 110b to 110c each include the same configuration as the imaging device 110a.

[0021] The imaging control unit 301 executes control to capture the environment around the imaging device 110a via the imaging unit 201. The signal processing unit 302 performs a predetermined process on the image captured by the imaging control unit 301 to generate data of the captured image. The signal processing unit 302, for example, encodes the image captured by the imaging control unit 301. The signal processing unit 302 encodes a still image using an encoding method such as JPEG. The signal processing unit 302 also encodes a moving image using an encoding method such as H.264 / MPEG-4 AVC (hereinafter, H.264) or HEVC (High Efficiency Video Coding). The signal processing unit 302 may also encode an image using an encoding method selected by a user from a plurality of encoding methods set in advance, for example, via an operation unit (not shown) of the imaging device 110a.

[0022] The storage unit 303 stores a list of analysis processes executable by the analysis unit 305 (hereinafter referred to as a first processing list) and a list of post-processing for the results of the analysis processes. The storage unit 303 also stores the results of the analysis processes described below. Note that in this embodiment, any process may be executed in addition to the analysis processes. The storage unit 303 stores a first processing list and a post-processing list for processes related to the any process. The control unit 304 performs overall control of the imaging device 110a, and controls the signal processing unit 302, the storage unit 303, the analysis unit 305, and the communication unit 306.

[0023] The analysis unit 305 performs analysis on the captured image by selecting at least one of pre-analysis processing, analysis processing, and post-analysis processing, which will be described later. The pre-analysis processing is processing performed on the captured image before performing analysis processing on the captured image. The pre-analysis processing is processing for creating divided images by dividing the captured image. The analysis processing is processing for outputting information obtained by analyzing the input divided images. The analysis processing performs at least one of human body detection processing, face detection processing, and vehicle detection processing, using the divided images obtained by the pre-analysis processing as input. The analysis processing outputs the analysis processing results related to the above detection processing.

[0024] The analysis process may be a process of outputting the position of an object in a divided image using a machine learning model that has been trained to detect an object included in an image, as in Patent Document 1, for example. The post-analysis process is a process that is executed after the analysis process is executed. The post-analysis process is a process that outputs the total number of objects detected in each divided image as a processing result based on the analysis process result for each divided image. The analysis process may detect an object in an image by performing pattern matching based on the learning data, and may output the position where the object is detected.

[0025] The communication unit 306 communicates with the auxiliary processing device 100a. The communication unit 306 converts input image data into a format that can be processed by the auxiliary processing device 100a, and transmits the image data obtained by the conversion to the auxiliary processing device 100. The communication unit 306 also receives data from the auxiliary processing device 100a, and converts the received data into a format that can be processed by the imaging device 110a. In this embodiment, the communication unit 306 executes a process of converting decimals from a floating-point format to a fixed-point format as a conversion process, but is not limited to this. Other conversion processes may be executed by the communication unit 306. The communication unit 306 can also transmit a command sequence that is predefined within the range of the SD standard to the auxiliary processing device 100a, and receive a response from the auxiliary processing device 100. This allows the communication unit 306 to communicate between the imaging device 110a and the auxiliary processing device 100a. The communication unit 306 can also communicate with the information processing device 130 via the network 120.

[0026] 4 is a diagram showing the configuration of auxiliary processing device 100a. Auxiliary processing device 100a includes an I / F unit 401, a circuit 402 (FPGA), a controller 403, and a storage unit 404. Auxiliary processing devices 100b to 100d include the same configuration as auxiliary processing device 100a. Auxiliary processing device 100a has a shape that complies with, for example, the SD standard, so that it can be inserted into and removed from a detachable socket of I / F unit 205 of imaging device 110a.

[0027] The I / F unit 401 is an interface section for connecting devices such as the imaging device 110a to the auxiliary processing device 100a. The I / F unit 401 is an electrical contact terminal or the like, and receives power from the imaging device 110a, for example, and generates and distributes power used in the auxiliary processing device 100a. The I / F unit 401 complies with the SD standard, similar to the I / F unit 205 of the imaging device 110a. Receiving images and setting data from the imaging device 110a, and transmitting data from the circuit 402 to the imaging device 110a are performed via the I / F unit 401.

[0028] The circuit 402 includes a control unit 410, a switching unit 411, and a processing unit 412. The circuit 402 is an FPGA, which is a type of semiconductor device that can repeatedly reconfigure the internal logic circuit structure. The imaging device 110a equipped with the auxiliary processing device 100a has a processing function added thereto by the processing function of the circuit 402. The reconfiguration function of the circuit 402 can change the logic circuit structure even after the logic structure is set. Therefore, for example, by installing the auxiliary processing device 100a in a device (not shown) in a field where technology is rapidly advancing, the auxiliary processing device 100a can execute processing suitable for the device. Note that, in this embodiment, an example in which an FPGA is used will be described, but as long as the same processing as that of an FPGA can be realized, for example, a general-purpose ASIC and a dedicated LSI may be used.

[0029] The circuit 402 is started by writing setting data including information on the logic circuit structure to be generated from a dedicated I / F, or by reading the setting data from the dedicated I / F. In this embodiment, the setting data is stored in the storage unit 404. When the auxiliary processing device 100a is powered on, the circuit 402 reads the setting data from the storage unit 404, generates a logic circuit, and starts up. The method of starting up is not limited to this, and for example, a dedicated circuit may be implemented in the auxiliary processing device 100a, and the imaging device 110a may write the setting data to the circuit 402 via the I / F unit 401.

[0030] The control unit 410 includes a circuit for transmitting and receiving images between the imaging device 110a and the auxiliary processing device 100a, a circuit for analyzing commands received from the imaging device 110a, and a circuit for performing control based on the analysis results. The commands are compliant with the SD standard, and the control unit 410 can detect some of these commands. The control unit 410 performs control so as to transmit an image to the controller 403 when performing storage processing, and transmit an image to the processing unit 412 when performing image analysis processing. In addition, when the control unit 410 receives setting data for switching processing, it transmits the setting data to the switching unit 411. The switching unit 411 includes a circuit for acquiring information on the image analysis processing function from the storage unit 404 based on the setting data received from the imaging device 110a, and writing the information to the processing unit 412. The information on the image analysis processing function is, for example, a setting parameter including the order, type, and calculation coefficient of the calculation to be processed in the processing unit 412.

[0031] The processing unit 412 includes a plurality of arithmetic circuits required to execute the image analysis processing function. The processing unit 412 executes each arithmetic processing based on the information of the image analysis processing function received from the switching unit 411. The processing unit 412 transmits the processing result to the imaging device 110a, and records the processing result in the storage unit 404 as necessary. In this manner, the circuit 402 extracts setting data of the processing function executed by the processing unit 412 from setting data corresponding to the plurality of processing functions previously stored. The circuit 402 rewrites the processing contents executed by the processing unit 412 based on the extracted setting data. This allows the auxiliary processing device 100a to selectively execute at least one of the plurality of processing functions. In addition, by adding setting data of a newly added process as needed, the imaging device 110a can cause the auxiliary processing device 100a to execute the latest process.

[0032] Hereinafter, having multiple setting data corresponding to multiple processing functions will be expressed as having multiple processing functions. In other words, even if the circuit 402 of the auxiliary processing device 100a is configured to execute one processing function, if the processing content of the processing unit 412 can be changed by setting data including other processing functions, it is said to have multiple processing functions.

[0033] The controller 403 is a known control IC (integrated circuit) that complies with the SD standard, and controls the slave operation of the SD protocol and the reading and writing of data from and to the storage unit 404. The storage unit 404 is configured, for example, with a NAND type flash memory, and stores various information such as memory data written from the imaging device 110a, information on the image analysis processing function written to the processing unit 412, setting data for the circuit 402, and the like.

[0034] FIG. 5 shows an example of the functions of the auxiliary processing device 100a. The auxiliary processing device 100a includes an analysis unit 501 and a communication unit 502. The analysis unit 501 executes an analysis process for an image. For example, when an analysis process setting request is input, the analysis unit 501 executes settings for making the input analysis process executable. Also, when an image is input to the auxiliary processing device 100a, the analysis unit 501 executes the analysis process set to an executable state for the input image. In this embodiment, the executable analysis processes are human body detection processing and face detection processing, but are not limited to these. For example, the analysis process may be face recognition processing, which is a process for determining whether or not a person stored in advance is included in an image. For example, the analysis unit 501 calculates the degree of agreement between the image feature amount of a person stored in advance and the image feature amount of a person detected from the input image, and determines that the person is the person stored in advance when the degree of agreement is equal to or greater than a threshold value.

[0035] The analysis process may be a process of superimposing a predetermined mask image or performing a mosaic process on a person detected from an input image for the purpose of privacy protection. The analysis process may be a process of detecting whether or not a person in an image is performing a specific action using a learning model that has learned a specific action of the person by machine learning. The analysis process may be a process of determining what kind of area an area in an image is. For example, the analysis process may be a process of determining what kind of area an area in an image is using a learning model that has learned buildings, roads, people, and the sky by machine learning. As described above, the executable analysis process can be applied to image analysis process using machine learning and image analysis process not using machine learning. The above analysis processes may be executed in cooperation with the imaging device 110a, rather than being executed by the auxiliary processing device 100a alone. The communication unit 502 communicates with the imaging device 110a via the I / F unit 401.

[0036] Fig. 6 shows an example of the configuration of the information processing device 130. The information processing device 130 is a general computer such as a PC, and includes, for example, a CPU 601, a RAM 602, a ROM 603, a HDD 604, and an I / F 605 as shown in Fig. 6. The information processing device 130 can execute various functions by the CPU 601 executing programs stored in the memory and the storage device.

[0037] 7 shows an example of functions of the information processing device 130 according to this embodiment. The information processing device 130 includes a communication unit 701, a control unit 702, a display unit 703, and an operation unit 704. The communication unit 701 is connected to the network 120, and executes communication with an external device (not shown) such as the imaging device 110 via the network 120. This is not limiting, and for example, the communication unit 701 may secure a direct connection to the imaging device 110a and communicate with the imaging device 110 without going through the network 120 or other devices. The control unit 702 controls the communication unit 701, the display unit 703, and the operation unit 704 so that they can execute their respective processes.

[0038] The display unit 703 is a display including, for example, an LCD and an organic EL, and can present various information to the user. The display unit 703 displays the results (for example, an image) rendered by the browser on the display, and the results are presented to the user. Note that, as a method other than display, information may be transmitted to the user by, for example, sound and vibration. The operation unit 704 accepts operations from the user. The operation unit 704 is, for example, a mouse, a keyboard, and the like, and the user can input user operations to the browser by operating these. The operation unit 704 is not limited to the above, and may be, for example, a touch panel, a microphone, or any device capable of detecting the user's intention.

[0039] The flow of processing executed in the imaging system 10 will be described below. Among the various processes, the processing executed by the imaging device 110a is realized by the CPU in the arithmetic processing unit 203 executing a program stored in a memory or the like. However, this is merely an example, and some or all of the processing described below may be realized by dedicated hardware. Furthermore, the processing executed by each of the auxiliary processing device 100a and the information processing device 130 may be realized by the CPU of each device executing a program stored in a memory or the like. Alternatively, some or all of the above processing may be realized by dedicated hardware.

[0040] FIG. 8 shows a flowchart of the image analysis process executed by the imaging system 10. In FIG. 8, the user attaches the auxiliary processing device 100a to the imaging device 110a (S801). Next, the imaging device 110a executes an initialization sequence of the auxiliary processing device 100a (S802). The initialization sequence is to make the auxiliary processing device 100a available by transmitting a predetermined command from the imaging device 110a to the auxiliary processing device 100a. After that, the imaging device 110a determines the process that can be executed by the auxiliary processing device 100a. The process determination may be performed by only the imaging device 110a or by a combination of the imaging device 110a and the auxiliary processing device 100a (S803). Note that the auxiliary processing device 100a is configured to be able to execute any process, but the any process does not have to include a process unrelated to the process to be executed by the imaging device 110a.

[0041] For example, the imaging device 110a may hold a list of processes executable by the auxiliary processing device 100a, which is acquired in advance from the information processing device 130. In this case, the imaging device 110a can determine the processes executable by the auxiliary processing device 100a based on the list of processes executable by the auxiliary processing device 100a. The imaging device 110a decides the processes to be executed by the auxiliary processing device 100a, and sets the decided processes in the auxiliary processing device 100a as necessary (S804). In other words, when it is decided that at least some of the executable processes will be executed by the auxiliary processing device 100a, settings for executing the part of the processes are executed in the auxiliary processing device 100a.

[0042] This setting includes, for example, reconfiguring the circuit 402 with setting data corresponding to a part of the processing to be executed. After the reconfiguration of the circuit 402 is completed, the image capture device 110a or the auxiliary processing device 100a executes a predetermined analysis processing (S805). Next, the image capture device 110a executes post-processing on the analysis result (S806). The processing of S805 and S806 is executed by repeating a predetermined number of times. The series of processing in FIG. 8 is executed when the auxiliary processing device 100a is attached to the image capture device 110a. For example, when the auxiliary processing device 100a is detached from the image capture device 110a, the processing of S803 may be executed again, and at least a part of the processing in FIG. 8 may be executed repeatedly.

[0043] Fig. 9 shows a flowchart of a process for determining a process executable by the imaging device 110a. This process corresponds to the process of S803 in Fig. 8, and is executed when the auxiliary processing device 100a or the like is attached to or detached from the imaging device 110a, and when the imaging device 110a is powered on. In this process, the imaging device 110a reads out a process executable by the auxiliary processing device 100a, integrates the process with an analysis process executable by the imaging device 110a, and determines an analysis process executable by the imaging device 110a. An outline of this process will be described below.

[0044] First, the control unit 304 of the imaging device 110a reads out a first process list, which is a list of executable processes stored in the storage unit 303 (S901). Next, the control unit 304 judges whether the attached device is, for example, a device having only a conventional storage function, or a predetermined device having a specific processing function such as the auxiliary processing device 100a (S902). For example, the control unit 304 controls the communication unit 306 to issue a request (read command) to the attached device that allows reading to a specific address. The communication unit 306 reads out flag data stored in a specific address of the attached device. Hereinafter, this specific address will be referred to as "address A". Details of the data stored in address A will be described later. Then, the control unit 304 judges whether the attached device is the auxiliary processing device 100a having a specific processing function based on the read flag data. However, this is merely an example, and whether the attached device is the auxiliary processing device 100a may be judged by other methods.

[0045] If the attached device is auxiliary processing device 100a (YES in S902), control unit 304 executes a process to determine processes executable by auxiliary processing device 100a, and the process proceeds to S903. Control unit 304 controls communication unit 306 to communicate with auxiliary processing device 100a, and acquires a second process list, which is a list of processes executable by auxiliary processing device 100a (S903). Control unit 304 acquires the second process list by, for example, reading data stored in address A in the same way as when determining whether the attached device is auxiliary processing device 100a.

[0046] The second process list is stored in the same address A as the flag data for determining whether or not the processing device is the auxiliary processing device 100a. In this case, the imaging device 110a simultaneously acquires the flag data and the second process list by accessing the address A, and simultaneously executes the process of S902 and the process of S903. The flag data and the second process list may be stored in an address different from the address A. Thereafter, the control unit 304 creates an integrated process list in which the first process list of processes executable by the imaging device 110a read from the storage unit 303 and the second process list acquired from the auxiliary processing device 100a are integrated (S904), and the process ends.

[0047] The integrated process list is a list indicating processes that can be executed locally by the imaging device 110a without processing by a device such as a server on the network 120. In this embodiment, the integrated process list is a list obtained by a union of the processes in the first process list and the second process list. The integrated process list is a list of processes included in at least one of the first process list and the second process list. This is not limited to the above, and for example, when a different process becomes executable by combining a process in the first process list and a process in the second process list, the different executable process may be added to the integrated process list.

[0048] That is, when a new analysis process can be executed by executing at least a part of the process in the first process list and at least a part of the process in the second process list together, information on the analysis process may be added to the integrated process list. For example, face recognition process is realized by a face detection process function, a face feature extraction process function, and a face feature matching process function. In this case, when the first process list includes the face detection process function and the face feature extraction process function, and the second process list includes the face feature matching process function, the face recognition process may be included in the integrated process list.

[0049] Returning to S902, if the attached device is not the auxiliary processing device 100a (NO in S902), the control unit 304 determines that there is no processing that can be executed by the attached device. Therefore, the control unit 304 determines that the first processing list, which is processing that can be executed by the imaging device 110a and read from the storage unit 303, is the integrated processing list (S905), and ends the processing. Note that when the attached device is removed from the imaging device 110a and the processing of Fig. 9 is executed, the specified device does not exist, so the first processing list is determined to be the integrated processing list.

[0050] This makes it possible to list processes that can be locally executed by the imaging device 110, based on whether or not the auxiliary processing device 100a capable of executing a specific process is attached to the imaging device 110a. Furthermore, the display unit 703 of the information processing device 130 presents the integrated process list to the user, so that the user can select a process that can be locally executed by the imaging device 110a. Note that, although the case where the integrated process list is generated has been shown in this embodiment, the first process list and the second process list may be managed separately, and the integrated process list may not be generated. In other words, by managing the processes that can be executed by the auxiliary processing device 100a and the processes that can be executed by the imaging device 110a in a manner that allows them to be distinguished, it is possible to omit the generation of the integrated process list.

[0051] On the other hand, even when the first process list and the second process list are managed separately, an integrated process list may be generated. For example, when a new process becomes executable by using a process in the first process list and a process in the second process list together, the new process is not included in the processes in the first process list and the second process list, but may be included in the integrated process list. When the integrated process list is output, information is output to distinguish whether the process included in the integrated process list is included in the first process list or the second process list. This allows the user to recognize whether the process presented on the display unit 703 can be executed without the auxiliary processing device 100a.

[0052] The integrated process list is presented on the display unit 703 of the information processing device 130, but is not limited thereto. For example, if the imaging device 110a has a display, the integrated process list may be displayed on the display. If the imaging device 110a has an audio output function, the integrated process list may be output via the function. By the imaging device 110a presenting the integrated process list to the user, the user can quickly recognize that the auxiliary processing device 100a having an unintended function has been mistakenly attached to the imaging device 110a. In this way, the imaging device 110a can output information based on the first process list indicating processes that the imaging device 110a can execute and the second process list indicating processes that the auxiliary processing device 100a can execute in any format.

[0053] Furthermore, when the auxiliary processing device 100a is removed, the imaging device 110a can update the integrated processing list by executing the processing of FIG. 9 again. At this time, the imaging device 110a discards the second processing list related to the removed auxiliary processing device 100a. This is not limited to the above, and the imaging device 110a can output the second processing list even when the auxiliary processing device 100a is not attached by storing the second processing list related to the auxiliary processing device 100a in the storage unit 303. That is, the imaging device 110a may output the second processing list for the auxiliary processing device 100a that was previously attached to the imaging device 110a and has been removed. Furthermore, the imaging device 110a may output a process that can be executed by combining the process of the second processing list and the process of the first processing list. That is, the imaging device 110a can output information on a process that cannot be executed by the imaging device 110a alone. At this time, the user may be notified that there is an auxiliary processing device 100a that can handle the process that cannot be executed by the imaging device 110a alone.

[0054] Furthermore, the imaging device 110a may output a second process list for the auxiliary processing device 100a that has not been attached to the imaging device 110a in the past and can be attached to the imaging device 110a. Such auxiliary processing device 100a and information indicating the analysis processes that it can execute may be acquired, for example, by the imaging device 110a from an external server (not shown) via a network. Also, information indicating the analysis processes that it can execute may be stored in advance in the storage unit 303 of the imaging device 110a. Also, the imaging device 110a may output a process that can be executed by combining the process of the second process list for the auxiliary processing device 100a that is not yet attached with the process of the first process list. That is, the imaging device 110a can output information on a process that it cannot execute by itself. At this time, the user may be notified that there is an auxiliary processing device 100a that can handle the process that the imaging device 110a cannot execute by itself.

[0055] When storing the second processing list for the auxiliary processing device 100a that is not yet attached to the imaging device 110a, the imaging device 110a can also store information capable of identifying the device, such as the model number of the auxiliary processing device 100a. When outputting the second processing list, the imaging device 110a can also output information capable of identifying the auxiliary processing device 100a. This allows the user to easily recognize which auxiliary processing device 100a should be attached to the imaging device 110a in order to use the processing function presented on the display unit 703.

[0056] 10 shows a flowchart of a process in which the imaging device 110a determines the content of an analysis process. In this process, the information processing device 130 presents to the user analysis processes that can be locally executed by the imaging device 110a, and the information processing device 130 accepts the user's selection and transmits the accepted information to the imaging device 110a. The imaging device 110a determines the analysis process to be executed according to the information of the user selection received from the information processing device 130. An outline of this process will be described below.

[0057] In FIG. 10, the control unit 702 of the information processing device 130 requests the imaging device 110a for a captured image, an integrated processing list, and a post-processing list via the communication unit 701 (S1001). At this time, the information processing device 130 requests information from the imaging device 110a by transmitting a request message conforming to the ONVIF standard, for example, to the imaging device 110a. The information transmission request may be made by other request messages or the like. Based on this request, the imaging device 110a captures the surrounding environment via the imaging control unit 301, and the control unit 304 controls the signal processing unit 302 to process the captured image, thereby acquiring the captured image (S1002). Note that the imaging device 110a may capture the surrounding environment and acquire the captured image regardless of the presence or absence of a request from the information processing device 130.

[0058] The imaging device 110a may store the captured image locally in a storage unit 303 or may store the captured image in an external device such as a network server. Next, the control unit 304 reads out the post-processing list stored in the storage unit 303. In this embodiment, the post-processing list is a display process and a storage process, but is not limited to this. The control unit 304 transmits the post-processing list, the integrated process list acquired in the process of FIG. 9, and the captured image acquired in S1002 to the information processing device 130 (S1003). The imaging device 110a transmits a response message corresponding to the request message conforming to the ONVIF standard to the information processing device 130. This allows the information processing device 130 to receive information.

[0059] The control unit 702 receives the captured image, the integration processing list, and the post-processing list from the imaging device 110a. The control unit 702 presents the integration processing list and the post-processing list to the user on the screen of the display unit 703 or the like (S1004). At this time, the control unit 702 may display the captured image on the screen together with the above lists. The user checks the integration processing list and the post-processing list displayed on the display unit 703. The user selects an arbitrary analysis process from the integration processing list via the operation unit 704 (S1005). In addition, the user selects an arbitrary post-processing for the analysis processing result via the operation unit 704 (S1006). The operation unit 704 transmits the analysis processing and post-processing selection results by the user to the control unit 702. The control unit 702 transmits information indicating the analysis processing and post-processing received from the operation unit 704 to the imaging device 110a (S1007).

[0060] When the control unit 304 of the imaging device 110a receives information indicating the analysis process selected by the user from the information processing device 130, it determines whether or not the analysis process is included in the second process list (S1008). Next, when the control unit 304 determines that the user-selected analysis process is not included in the second process list (NO in S1008), the control unit 304 executes the analysis process in the imaging device 110a. As a result, the control unit 304 ends the process without notifying the auxiliary processing device 100a. On the other hand, when the control unit 304 determines that the user-selected analysis process is included in the second process list (YES in S1008), the process proceeds to S1009. The control unit 304 transmits a setting request for the user-selected analysis process to the auxiliary processing device 100a (S1009).

[0061] The communication unit 502 of the auxiliary processing device 100a receives a setting request for the user-selected analysis process from the imaging device 110a. At this time, the communication unit 502 judges the setting request for the user-selected analysis process based on the amount of data written by the imaging device 110a and the type of the write command. The method of judging the setting request will be described later in detail. The communication unit 502 transmits the setting request for the user-selected analysis process received from the imaging device 110a to the analysis unit 501. Based on the setting request for the user-selected analysis process received from the communication unit 502, the analysis unit 501 executes settings so that the auxiliary processing device 100a is in a state where the processing can be executed (S1010). After completing the setting of the processing, the communication unit 502 transmits a setting completion notification to the imaging device 110a (S1011). Note that the communication unit 502 notifies information for preventing the imaging device 110a from writing data to the auxiliary processing device 100a when the processing setting of the auxiliary processing device 100a is not completed. The communication unit 502 may notify the imaging device 110a of information on the scheduled completion of the setting before the processing setting is completed. The control unit 304 of the imaging device 110a controls the communication unit 306 to receive a processing setting completion notification from the auxiliary processing device 100a.

[0062] The auxiliary processing device 100a may notify the imaging device 110a of the completion of the process setting by using, for example, any of the following three methods. In the first notification method, the communication unit 502 outputs a BUSY signal when the imaging device 110a is in the process of writing the first block of data and the setting of the user-selected analysis process is not completed. The BUSY signal is output by, for example, driving a DATA signal line conforming to the SD standard to a low state. In this case, the imaging device 110a can determine whether the setting of the user-selected analysis process is completed by checking the BUSY signal.

[0063] In the second notification method, the time until the setting of the analysis process selected by the user is completed is stored in advance in the above-mentioned specific address, and the imaging device 110a reads out the time until the setting is completed. After the time until the setting of the analysis process selected by the user has elapsed, the imaging device 110a issues a write command as an output of write data. This allows the imaging device 110a to transmit the data of the captured image to the auxiliary processing device 100a after the setting of the analysis process selected by the user is completed. In the third notification method, when the analysis unit 501 completes the setting of the analysis process selected by the user, the analysis unit 501 writes a setting completion flag in the second specific address of the auxiliary processing device 100a. The imaging device 110a can determine whether the setting of the analysis process selected by the user is completed by reading out the data in this second specific address. Note that the information of the second specific address in which the setting completion flag is written may be stored in the above-mentioned specific address, or may be stored in another address.

[0064] 10 uses an integrated process list to appropriately determine the analysis process selected by the user while taking into consideration the state of the imaging device 110a. Furthermore, when the analysis process selected by the user includes a process executed by the auxiliary processing device 100a, the auxiliary processing device 100a is automatically configured, eliminating the need for a user to perform a configuration operation. Furthermore, the auxiliary processing device 100a automatically prepares to execute the analysis process selected by the user. When the analysis process selected by the user does not include a process executed by the auxiliary processing device 100a, the imaging device 110a can execute the process without changing the settings of the auxiliary processing device 100a.

[0065] FIG. 11 shows a flowchart of the process when the imaging device 110a executes the analysis process. In FIG. 11, the imaging control unit 301 captures the surrounding environment (S1101). The control unit 304 controls the signal processing unit 302 to process the image captured by the imaging control unit 301 and acquire the captured image. Thereafter, the control unit 304 controls the analysis unit 305 to execute pre-analysis processing on the captured image input from the control unit 304 and acquire an image that is a result of the pre-analysis processing (S1102). The control unit 304 determines whether the analysis processing selected by the user is included in the second processing list (S1103). When the control unit 304 determines that the analysis processing selected by the user is not included in the second processing list (NO in S1103), it executes the analysis processing on the image that is a result of the pre-analysis processing in the imaging device 110a (S1104). The control unit 304 controls the analysis unit 305 to execute post-analysis processing on the analysis processing result (S1108) and ends the process.

[0066] When the control unit 304 determines that the analysis process selected by the user is included in the second process list (YES in S1103), it transmits the image, which is the result of the process before analysis, to the auxiliary processing device 100a (S1105). For example, the control unit 304 issues a write request (write command) for the result of the process before analysis, and transmits the image, which is the result of the process before analysis, to the auxiliary processing device 100a. The communication unit 502 of the auxiliary processing device 100a receives the image, which is the result of the process before analysis, from the imaging device 110a, and transmits the image to the analysis unit 501. The analysis unit 501 executes the analysis process set in S1010 for the image received from the communication unit 502 (S1106). The communication unit 502 transmits the result of the analysis process of the image by the analysis unit 501 to the imaging device 110a (S1107). The control unit 304 controls the communication unit 306 to receive the result of the analysis process of the image from the auxiliary processing device 100a. The control unit 304 controls the analysis unit 305 to execute post-analysis processing on the analysis processing results (S1108).

[0067] The transmission of the analysis processing result from the auxiliary processing device 100a to the imaging device 110a is performed as follows. The analysis unit 501 stores the analysis processing result in a storage address of the analysis processing result assigned for each analysis processing selected by the user. The imaging device 110a reads out the storage address of the analysis processing result stored in address A together with the second processing list, for example, and transmits a read request (read command) for the storage address to the auxiliary processing device 100a. The auxiliary processing device 100a receives the read request for the storage address of the analysis processing result via the communication unit 502, and transmits the analysis processing result to the imaging device 110a. Note that the imaging device 110a issues a read request for the storage address of the analysis processing result after the estimated processing time stored in address A has elapsed, for example.

[0068] The auxiliary processing device 100a may output a BUSY signal from the write request for the last block of the processing result before analysis to the end of the analysis process selected by the user. At this time, when the imaging device 110a no longer receives the BUSY signal from the auxiliary processing device 100a, it may issue a read request to the storage address of the analysis process result. This allows the imaging device 110a to obtain the processing result after the analysis process selected by the user by the auxiliary processing device 100a is completed. According to this process, the imaging device 110 can determine whether to transfer the captured image to the auxiliary processing device 100a according to the analysis process selected by the user. This allows the user to perform the analysis process of the captured image without being aware of whether the analysis process is performed by the imaging device 110a or the auxiliary processing device 100a.

[0069] FIG. 12 shows a flowchart when the imaging device 110a executes post-processing. In FIG. 12, the control unit 304 of the imaging device 110a determines whether or not the post-processing to be executed includes "display" (S1201). When the control unit 304 determines that the post-processing to be executed includes display (YES in S1201), the process proceeds to S1202. The control unit 304 controls the communication unit 306 to transmit the result of the analysis process to the information processing device 130 (S1202). When the control unit 702 of the information processing device 130 receives the result of the analysis process from the imaging device 110a, the control unit 702 presents the result of the analysis process to the user on the display unit 703 (S1203). On the other hand, when the control unit 304 determines that the post-processing to be executed does not include display (NO in S1201), the process proceeds to S1204 without executing the processes of S1202 and S1203.

[0070] The control unit 304 determines whether the post-processing to be executed includes "save" (S1204). The determination of S1204 may be executed before S1201 or in parallel with S1201. When the control unit 304 determines that the post-processing to be executed includes save (YES in S1204), the process proceeds to S1205. The control unit 304 stores the result of the analysis process in the storage unit 303 and ends the process (S1205). On the other hand, when the control unit 304 determines that the post-processing to be executed does not include save (NO in S1204), the process of S1205 is not executed and the process ends. In this way, the imaging device 110a can transmit the result of the analysis process to the information processing device 130 and store the result in the storage unit 303 according to the selected post-processing without any additional setting operation by the user. According to this process, the convenience of the user regarding the post-processing can be improved.

[0071] The communication between the imaging device 110a and the auxiliary processing device 100a will be described below. The arithmetic processing unit 203 and the controller 403 are connected by a power line, a GND line, a clock line, a command line, and a data line via a device insertion socket of the I / F unit 205. The clock line, the command line, and the data line are connected to pass through the circuit 402. The clock line transmits a clock for synchronization output from the arithmetic processing unit 203. The command line transmits and receives an operation request command from the arithmetic processing unit 203 to the controller 403, and a response from the controller 403 to the arithmetic processing unit 203. The data line transmits and receives write data from the arithmetic processing unit 203 and read data from the auxiliary processing device 100a.

[0072] Furthermore, the arithmetic processing unit 203 judges whether the device detect signal of the device insertion socket of the I / F unit 205 is High or Low. This allows the arithmetic processing unit 203 to recognize whether the auxiliary processing device 100a is inserted or not. After supplying power to the controller 403, the arithmetic processing unit 203 issues an operation request command to the controller 403 via a command line. Then, the arithmetic processing unit 203 sets the voltage and communication speed (clock frequency) for data communication, etc., in response to the response from the controller 403 and the reception of output data indicating device information as an SD card.

[0073] FIG. 13 shows an example of the configuration of commands and responses communicated on a command line. The commands and responses have a configuration conforming to the SD standard. A command 1301 in FIG. 13(A) is a command requested from the arithmetic processing unit 203 to the controller 403. The command 1301 includes a start bit 1302, a direction bit 1303, and a command number section 1304. The command 1301 further includes a command argument section 1305, an error correction data section 1306, and an end bit 1307. The start bit 1302 is disposed as the first bit of the command 1301 to indicate the start position of the command. The direction bit 1303 is a signal transmitted from the imaging device 110a to the auxiliary processing device 100a.

[0074] In the command number section 1304, a value indicating the type of command is written. For example, when the value "23" is stored in the command number section 1304, the command 1301 is a block number designation command that designates the number of data blocks. Also, when the value "25" is stored in the command number section 1304, the command 1301 is a multi-write command. When the value "12" is stored in the command number section 1304, the command 1301 is a data transfer stop command. The command argument section 1305 has information such as the number of transfer data blocks, memory write / read addresses, etc. according to the type of command. The end bit 1307 is placed as the last bit of the command 1301 to indicate the end position of the command.

[0075] A response 1311 in FIG. 13B is a command transmitted from the controller 403 to the arithmetic processing unit 203 in response to the command 1301. The response 1311 includes a start bit 1312, a direction bit 1313, and a response number section 1314. The response 1311 further includes a response argument section 1315, an error correction data section 1316, and an end bit 1317. The start bit 1312 is the first bit of the response 1311 and indicates the start position of the response. The direction bit 1313 indicates a signal output from the auxiliary processing device 100a to the imaging device 110a. The response number section 1314 indicates which command the response corresponds to. The response argument section 1315 has information such as the status of the SD card depending on the command type. The end bit 1317 is the last bit of the response 1311 and indicates the end position of the response.

[0076] The method of transmitting and receiving data between the arithmetic processing unit 203 and the auxiliary processing device 100a will be described below. The I / F unit 205 transfers data in blocks for writing and reading data. The arithmetic processing unit 203 transfers a plurality of blocks of data to the auxiliary processing device 100a in the following manner. In the first transfer method, the number of blocks is specified by a block number specification command for the transfer data, and then data corresponding to the number of blocks specified by the multiwrite command is transferred. The block number specification command specifies the number of blocks of the write data in the command argument section 1305. The multiwrite command specifies the address of the storage unit 404 to which the data is written in the command argument section 1305. In the second transfer method, the data transfer is started when the block number specification command is not issued and the multiwrite command is issued. When the data transfer is completed, a transfer stop command is issued, and the process is completed. At this time, only the address of the storage unit 404 to which the data is written is specified in the command argument section 1305 of the multiwrite command. The arithmetic processing unit 203 can arbitrarily switch between the above data transfer methods.

[0077] When, for example, a storage process is performed in the analysis process, the circuit 402 inputs a command and data transmitted from the arithmetic processing unit 203 to the controller 403. The controller 403 stores the received data at an address in the storage unit 404 specified by the command. When, for example, an image analysis process is performed in the analysis process, the circuit 402 executes the analysis process on the data transmitted from the arithmetic processing unit 203. The circuit 402 transmits the data of the processing result and information specifying a predetermined address in the storage unit 404 to the controller 403. The controller 403 stores the processing result in the specified address in the storage unit 404.

[0078] The arithmetic processing unit 203 reads a plurality of blocks of data from the auxiliary processing device 100a. At this time, data is read in the following two ways. In the first read method, when the number of blocks is specified by the block number specification command, a multi-read command is issued. In this method, only the specified number of blocks of data are read based on the specification command. In the block number specification command, the number of blocks of the read data is specified in the command argument part 1305. In the command argument part 1305 of the multi-read command, the address of the memory from which the data is read is specified. In the second read method, the block number specification command is not issued and the multi-read command is issued, thereby starting the reading of data. In this method, the process is terminated by issuing a transfer stop command. The arithmetic processing unit 203 can arbitrarily switch between the two read methods.

[0079] When the write data and read data are one block, a single write command and a single read command are issued. This allows data to be written and read without issuing a block count designation command and a transfer stop command. In the single write command and single read command, the address of the storage unit 404 to be accessed is specified in the command argument section 1305. The arithmetic processing unit 203 can transmit data to be stored or analyzed by writing data to the auxiliary processing device 100a. The arithmetic processing unit 203 can obtain image data stored in the storage unit 404, the results of image analysis, and information on the processing functions of the auxiliary processing device 100a by reading data from the auxiliary processing device 100a.

[0080] The auxiliary processing device 100a stores information about the processing functions of the auxiliary processing device 100a at a specific address A in the storage unit 404. The calculation processing unit 203 of the imaging device 110a can determine the information about the processing functions of the auxiliary processing device 100a by issuing a multi-read command or a single-read command to this address A. The information about the processing functions includes whether or not the processing function is possessed, the time required from execution to completion of the processing, the data size of the processing result, and address information where the processing result is stored.

[0081] FIG. 14 shows an example of information on processing functions. The processing function flag 1401 indicates that the auxiliary processing device 100a has an image analysis processing function. The imaging device 110a checks the processing function flag 1401 to determine whether the auxiliary processing device 100a has an image analysis processing function. The processing classification 1402 indicates the analysis processing that the auxiliary processing device 100a has. The input data size 1403 and the number of input data 1404 indicate information on the input specifications of data for each processing function. Furthermore, the estimated processing time 1405 indicates the time required from data input to processing result output. The number of processing result data 1406 indicates the number of data of the processing result. The analysis result storage address 1407 indicates where in the storage unit 404 the processing result is stored. The arithmetic processing unit 203 can obtain the processing function table of FIG. 15 by reading out the data at address A of the storage unit 404 shown in FIG. 14. The reference symbols in FIG. 15 correspond to the reference symbols in FIG. 14.

[0082] Furthermore, if the processor 203 has not issued a read command for address A, the auxiliary processing device 100a determines that the device to which the auxiliary processing device 100a is attached is a device that does not use the image analysis processing function. In this case, the auxiliary processing device 100a only stores the transferred data in the storage unit 404. As a result, the auxiliary processing device 100a functions only as a memory device for devices that do not require the image analysis processing function. Here, a method of storing information on the processing function in a specific address A of the storage unit 404 has been shown, but this is not limiting. For example, information on the processing function may be added to the response argument section 1315 of the response to a command used during the initial setup of the auxiliary processing device 100a.

[0083] It should be noted that the imaging device 110a executes reading of address A in the storage unit 404 after, for example, completing initialization of the auxiliary processing device 100a. When the auxiliary processing device 100a is no longer detected in the socket of the I / F unit 205, the imaging device 110a discards the information that has been read so far. If the auxiliary processing device 100a is inserted into the socket after discarding the information, the imaging device 110a reads the value of address A after completing initialization of the auxiliary processing device 100a. In this way, when a different auxiliary processing device a is inserted, the imaging device 110a can read information on the processing functions of that auxiliary processing device a and set processing for it.

[0084] The process in which the auxiliary processing device 100a automatically switches between the storage process and the image analysis process will be described. This process is a process in which the auxiliary processing device 100a determines whether to store the image data received from the imaging device 110a as is or to perform image analysis process on the image data. The imaging device 110a can control whether to make the auxiliary processing device 100a store the transmitted image data or to perform image analysis process on the image data by transmitting a special command to the auxiliary processing device 100a, for example. However, it is not easy to define such a special command, since the standard to which the auxiliary processing device 100a conforms must be taken into consideration. In this embodiment, the process to be executed in the auxiliary processing device 100a is switched by the following method without defining a special command. In the following process example, communication between the imaging device 110a and the auxiliary processing device 100a is performed by a method conforming to the SD standard, but is not limited to this. In other words, any method may be used as long as the process described below can be executed by a command conforming to the standard to which the auxiliary processing device 100a conforms.

[0085] Fig. 16 shows a flow of control for automatically switching between storage processing and image analysis processing depending on the number of data blocks transferred to the auxiliary processing device 100a. In Fig. 16, the arithmetic processing unit 203 of the imaging device 110a issues a write command conforming to the SD standard to the auxiliary processing device 100a, and the auxiliary processing device 100a transfers data (S1601). The circuit 402 of the auxiliary processing device 100a determines whether the number of blocks of data written by the arithmetic processing unit 203 matches the amount of data when the image analysis processing is executed (S1602). The circuit 402 determines the number of data blocks described in the command argument section 1305 of the block number designation command. Furthermore, if the block number designation command is not issued, the circuit 402 may count the number of blocks transferred until a data transfer stop command is issued, and determine the number of data blocks based on the count.

[0086] When the circuit 402 determines that the number of data blocks written thereto by the arithmetic processing unit 203 matches the amount of data when the image analysis process is executed (YES in S1602), the process proceeds to S1603. The circuit 402 executes the image analysis process on the transferred data (S1603). The circuit 402 acquires the image analysis process result (S1604) and issues a write command to the controller 403. The circuit 402 stores the acquired image analysis process result in the analysis result storage address 1407 in the storage unit 404 corresponding to the classification of the analysis process (S1605). On the other hand, when the circuit 402 determines that the number of data blocks written thereto by the arithmetic processing unit 203 does not match the amount of data when the image analysis process is executed (NO in S1602), the process proceeds to S1606. The circuit 402 stores the transferred data as it is in the storage unit 404 (S1606). For example, the circuit 402 issues a command similar to the write command issued by the arithmetic processing unit 203 to the controller 403, and transfers the transfer data as is to the storage unit 404. The controller 403 stores the transfer data at an address in the storage unit 404 specified by the write command.

[0087] The auxiliary processing device 100a holds the information shown in Fig. 14 in a specific address A of the storage unit 404, and determines that the number of input data 1404 when the analysis process A of the process classification 1402 is executed is 20 blocks, for example. Therefore, when the circuit 402 determines that the number of data blocks written thereto by the arithmetic processing unit 203 is 20 blocks, it executes the analysis process A. When the circuit 402 determines that the number of written data blocks is other than 20 blocks, it does not execute the analysis process A. Note that when the circuit 402 determines that the number of data blocks written thereto by the arithmetic processing unit 203 is 40 blocks, for example, it executes the analysis process C. In this way, the analysis process executed by the circuit 402 may be changed according to the number of data blocks input thereto.

[0088] 17 shows an example of a flow of control for switching between storage processing and image analysis processing based on a write address specified in the command argument portion 1305 of a write command. In this process, the arithmetic processing unit 203 of the imaging device 110a issues a write command to the controller 403 (S1701). The circuit 402 determines whether the write address specified in the command argument portion 1305 matches the analysis result storage address 1407 in FIG. 14 (S1702). When the circuit 402 determines that the write address specified in the command argument portion 1305 matches the analysis result storage address 1407 (YES in S1702), the process proceeds to S1703. The circuit 402 executes image analysis processing corresponding to the analysis result storage address 1407 for the transferred data (S1703).

[0089] The circuit 402 acquires the image analysis processing result (S1704) and issues a write command to the controller 403. The circuit 402 stores the acquired image analysis processing result in the analysis result storage address 1407 of the storage unit 404 (S1705). On the other hand, when the circuit 402 determines that the write address specified in the command argument part 1305 does not match the analysis result storage address 1407 (NO in S1702), the process proceeds to S1706. The circuit 402 stores the transfer data as it is in the storage unit 404 (S1706). For example, the circuit 402 issues a command similar to the write command issued by the arithmetic processing unit 203 to the controller 403, and transfers the transfer data to the storage unit 404 as it is. The controller 403 stores the transfer data in the address of the storage unit 404 specified by the write command.

[0090] The auxiliary processing device 100a holds the information of FIG. 14 in a specific address A of the storage unit 404, and determines that the analysis result storage address 1407 is 0xFFFFFFFF when, for example, the analysis process A of the process classification 1402 of FIG. 15 is executed. Therefore, the circuit 402 executes the analysis process A when the analysis result storage address 1407 specified by the command acquired from the arithmetic processing unit 203 is 0xFFFFFFFF. The circuit 402 does not execute the analysis process A when the analysis result storage address 1407 is other than 0xFFFFFFFF. Note that, for example, when the analysis result storage address 1407 specified by the command acquired from the arithmetic processing unit 203 is 0xEEEEEEEE, the circuit 402 executes the analysis process C. In this way, the circuit 402 may change the analysis process to be executed according to the analysis result storage address 1407 specified by the command.

[0091] In this way, the auxiliary processing device 100a determines whether to store the transferred data or perform image analysis processing on it, depending on the number of data blocks and the analysis result storage address written thereto by the arithmetic processing unit 203. Alternatively, the circuit 402 may perform image analysis processing when it determines that both the number of data blocks and the analysis result storage address match the number of input data 1404 and the analysis result storage address 1407, respectively. Also, the circuit 402 may perform storage processing when it determines that at least one of the number of data blocks and the analysis result storage address does not match the number of input data 1404 and the analysis result storage address 1407.

[0092] According to the above-mentioned process, the auxiliary processing device 100a can perform image analysis on the data for which image analysis is to be performed without providing a special control command. Also, the auxiliary processing device 100a can store the data to be stored in the storage unit 404 without performing image analysis on the data. This can prevent the system from becoming complicated, and there is no need to provide additional procedures. According to this process, the image analysis process can be started quickly. The process of FIG. 17 may be executed in combination with the process of FIG. 16. That is, the image analysis process may be executed by associating the number of blocks of the image data and the analysis result storage address with the image analysis process.

[0093] When performing image analysis processing, the transfer data that was the subject of the analysis processing may be stored together with the processing result in an area of ​​the storage unit 404 that is different from the analysis result storage address 1407. In addition, in the above-mentioned control, when the auxiliary processing device 100a has multiple image analysis processing functions, the type of image analysis processing to be executed may be determined according to the number of blocks to which data is written and the analysis result storage address. For example, when the number of blocks of data and the analysis result storage address match the number of input data 1404 and the analysis result storage address 1407 of any image analysis processing among the multiple image analysis processing functions, that image analysis processing may be executed.

[0094] FIG. 18 shows an example of a control flow for switching between a command-based storage process and an image analysis process. In the SD standard, a first protocol is used as a protocol for writing data, which writes data after issuing a block number designation command. In addition, a second protocol is used for writing data without issuing a block number designation command in the first protocol. In addition, in the second protocol, a data transfer stop command is issued when data writing is completed. In this process, image analysis is performed based on data transmission via the first protocol. When data is transmitted using the second protocol, a storage process is performed to store image data in the storage unit 404 without performing image analysis. In addition, the circuit 402 of the auxiliary processing device 100a determines whether to perform image analysis based on whether a block number designation command has been issued for transmitting image data.

[0095] In this process, first, the arithmetic processing unit 203 of the imaging device 110a issues a write command to the auxiliary processing device 100a and transfers data (S1801). Here, the circuit 402 of the auxiliary processing device 100a judges whether or not a block number designation command has been issued (S1802). If the block number designation command has been issued (YES in S1802), the process proceeds to S1803. The circuit 402 executes image analysis processing on the transferred data (S1803) and acquires the processing result (S1804). The circuit 402 issues a write command to the controller 403, specifying a predetermined analysis result storage address 1407 according to the classification of the analysis processing shown in FIG. 14. As a result, the circuit 402 stores the data of the processing result in the storage unit 404 (S1805). If the block number designation command has not been issued (NO in S1802), the process proceeds to S1806. The circuit 402 issues to the controller 403 a write command similar to the command issued by the arithmetic processing unit 203. Then, the circuit 402 transmits the transferred data as is to the controller 403. Then, the controller 403 stores the data at the address of the storage unit 404 specified by the write command from the circuit 402 (S1806).

[0096] The block number designation command may be another predetermined command. That is, a predetermined command that triggers the execution of the image analysis process may be set in advance. The circuit 402 executes the image analysis process on the input image data based on the reception of the predetermined command. In addition, other identifiable information may be used as the protocol to be used. In addition, when the circuit 402 receives a predetermined command, for example, it may execute the process of FIG. 16 or FIG. 17 to determine whether or not to execute the image analysis process on the input image data. In this way, a command such as the block number designation command indicates which process of the image analysis process should be performed. This allows the imaging device 110a to instruct the auxiliary processing device 100a to execute the process within the range of the protocol conforming to the SD standard.

[0097] At least a part of the above-mentioned process determines whether or not a command conforming to the SD standard for transmitting image data includes a value associated with an image analysis process executable by the circuit 402. As a result, a part of the above-mentioned process can determine whether or not to execute the image analysis process. For example, the process of FIG. 16 executes the image analysis process when "23" is stored in the command number section 1304 and a value indicating a predetermined number of blocks is stored in the command argument section 1305. Also, the process of FIG. 17 executes the image analysis process when a value indicating a processing result storage address is stored in the command argument section 1305. The process of FIG. 18 executes the image analysis process when "23" is stored in the command number section 1304. In this way, the content of the command when image data is transmitted can be set to a predetermined value associated with the image analysis process. As a result, it is possible to flexibly control whether the circuit 402 executes the image analysis process or the storage process using a command conforming to the SD standard.

[0098] A method for the imaging device 110a to read out the image analysis processing results stored in the auxiliary processing device 100a will be described. The arithmetic processing unit 203 specifies the analysis result storage address 1407 shown in FIG. 14 to the auxiliary processing device 100a. The arithmetic processing unit 203 issues a read command to read out the processing result data number 1406 for each analysis process. The controller 403 receives the read command via the circuit 402. The controller 403 outputs the processing result data stored in the specified address of the storage unit 404 to the arithmetic processing unit 203 of the imaging device 110a. This allows the imaging device 110a to obtain the image analysis processing results.

[0099] An example of a method of presenting a captured image, an integrated processing list, and a post-processing list to a user and accepting a user selection will be described. FIG. 19 shows an example of a screen display of a captured image, an integrated processing list, and a post-processing list via the display unit 703. This display screen displays, for example, a user interface 1901. The user interface 1901 includes a captured image display area 1902, an integrated processing list display area 1903, and a post-processing list display area 1904. The captured image display area 1902 displays an image captured by the imaging device 110a, and displays, for example, three people in FIG. 19. The integrated processing list display area 1903 displays the analysis processing contents, for example, face detection, human body detection, and vehicle detection processing. The post-processing list display area 1904 displays, for example, display and save as post-processing contents. The user can determine the contents of the captured image, the integrated processing list, and the post-processing list by checking these areas.

[0100] The list displayed in the integrated process list display area 1903 is not limited to the integrated process list. For example, the imaging device 110a stores a second process list for a certain auxiliary processing device 100a in the storage unit 303. Even if the auxiliary processing device 100a is not attached to the imaging device 110a, the imaging device 110a can transmit the second process list in the storage unit 303 to the information processing device 130. That is, the imaging device 110a may output the second process list for the auxiliary processing device 100a that was previously attached to the imaging device 110a. In this case, the information processing device 130 can gray out the analysis processes that are included in the second process list and are not included in the integrated process list as analysis processes that are possible when the auxiliary processing device 100a is attached. This makes it possible to present to the user that the auxiliary processing device 100a should be attached to the imaging device 110a in order to make the grayed-out process executable. Also, for example, when the imaging device 110a and the auxiliary processing device 100a have the same processing function, they may be integrated and displayed as one process. In this case, the image capture device 110a can determine whether the processing should be executed by itself or the auxiliary processing device 100a. The method of this determination will be described later.

[0101] The information processing device 130 may display the analysis process and post-processing displayed to the user so that the user can identify whether the analysis process and post-processing are to be performed by the imaging device 110a or the auxiliary processing device 100a. For example, when the imaging device 110a creates an integrated process list, the imaging device 110a includes information indicating whether each analysis process included in the integrated process list is included in the first process list or the second process list in the integrated process list. The information processing device 130 can display each analysis process in a different text color according to the above information. This allows the user to confirm whether each process is executable even if the auxiliary processing device 100a is removed. If the imaging device 110a and the auxiliary processing device 100a can execute the same process and both are displayed in an integrated manner, the following display is possible. Since the process is executable even if the auxiliary processing device 100a is removed, the above process is displayed in a text color corresponding to the imaging device 110a. However, the present invention is not limited to this, and the process may be displayed in a text color indicating that the process is executable by either the imaging device 110a or the auxiliary processing device 100a.

[0102] Furthermore, when a process that can be executed by the imaging device 110a and the auxiliary processing device 100a in cooperation is included in the integrated process list, the integrated process list may include information indicating that the process requires cooperation. In this case, the process that is realized by the imaging device 110a and the auxiliary processing device 100a in cooperation may be displayed in a different text color. For example, face authentication processing is realized by a functional group of a face detection processing function, a face feature extraction processing function, and a face feature matching processing function. The imaging device 110a has a face detection processing function and a face feature extraction processing function, and the auxiliary processing device 100a has a face feature matching processing function. In this case, the user interface 1901 displays, for example, the face detection processing and the face feature extraction processing in blue text, the face feature matching processing in red text, and the face authentication processing in green text.

[0103] Note that changing the text color is merely one aspect for distinguishing whether each function is executed by the imaging device 110a or the auxiliary processing device 100a, or whether it is executed by cooperation between them. Such distinguishing display may be performed in other aspects. For example, the execution subject of the process may be clearly indicated by changing the background color of each process. Also, the difference between the execution subjects may be indicated by character strings. For example, a character string representing the imaging device 110a is added after a character string indicating a process executed by the imaging device 110a. A character string representing the auxiliary processing device 100a is added after a character string indicating a process executed by the auxiliary processing device 100a. Also, a character string indicating the cooperation between the imaging device 110a and the auxiliary processing device 100a may be added to a character string indicating a process realized by cooperation between the imaging device 110a and the auxiliary processing device 100a.

[0104] In this way, the imaging device 110a provides information that distinguishes between the processes included in the first process list and the processes included in the second process list to the information processing device 130. This allows the imaging device 110a to display the executing entity of each process in a distinguishable manner on the display unit 703 of the information processing device 130. Also, even if the imaging device 110a has a display unit, by preparing information that distinguishes between the processes included in the first process list and the processes included in the second process list, it is possible to display the executing entity of each process in a distinguishable manner. In other words, the imaging device 110a can display the executing entity of each process on any display device by outputting information that distinguishes between the processes included in the first process list and the processes included in the second process list.

[0105] The user can select an analysis process from the integrated process list displayed in the integrated process list display area 1903 in the user interface 1901 via the operation unit 704. Also, the user can select a post-process from the process list displayed in the post-process list display area 1904 via the operation unit 704. For example, FIG. 19 shows an example in which the user selects the "face detection" process as the analysis process and selects "display" and "save" as the post-process. Note that, in this embodiment, an example in which only one analysis process is selected is shown, but this is not limited to this. A configuration in which the user can select multiple analysis processes may be used. For example, in addition to "face detection", at least one of "human body detection" and "vehicle detection" may be selected. Also, when one process is selected, other processes may not be selected. As an example, in the integrated process list display area 1903 in FIG. 19, when "human body detection" is selected while "face detection" is selected, the selection of "face detection" may be cancelled. Also, FIG. 19 shows an example in which both of two post-processes are selected, but only one of them may be selectable.

[0106] In S1007 of Fig. 10, the image capture device 110a is notified of the selection result based on the user's selection of analysis processing and post-processing. The control unit 702 may check the user selection state at predetermined intervals and notify the image capture device 110a of the processing to be executed depending on which processing is selected as the processing to be executed. That is, the processing from S1005 to S1007 of Fig. 10 may be executed periodically. The selections of S1005 and S1006 may be constantly monitored, and the processing of S1007 may be executed when the selection state changes.

[0107] Fig. 20 shows an example of a method of displaying information in S1203 when "face detection" is selected as the analysis process to be executed and "display" is selected as the post-processing. In Fig. 20, the number of people 2001 detected by the face detection process is displayed as a result of the analysis process on the screen of the user interface 1901 in Fig. 19. Note that Fig. 20 is merely an example, and the results of the process may be displayed separately from the user interface 1901, or may be displayed in another area in the user interface 1901.

[0108] Further, a priority may be set for each of the processes and post-processing to be executed selected by the user. For example, when the control unit 304 of the imaging device 110a sets multiple priorities for the processes to be executed, the control unit 304 executes the processes from S1103 to S1107 in FIG. 11 for each process to be executed in order of priority. Note that the computational resources and network resources of the imaging device 110a may be allocated based on the priority of the processes. For example, a process with a high priority may be executed for every first predetermined number of frames of the video. A process with a low priority may be executed for every second predetermined number of frames, which is greater than the first predetermined number of frames, of the video. That is, the frequency at which the processes are executed may be determined by the priority. Also, the frequency at which the results of the processes with a high priority are transmitted to the information processing device 130 may be higher than the frequency at which the results of the processes with a low priority are transmitted to the information processing device 130.

[0109] A specific process may be executable by combining a plurality of processes. For example, face recognition process may be executable by combining face detection process, face feature extraction process, and face feature matching process. Here, if the image capture device 110a and the auxiliary processing device 100a are capable of executing at least one of these three processes, the processes may be shared and executed between these devices. In addition, the image capture device 110a and the auxiliary processing device 100a may provide different processing functions suitable for the acquisition conditions (e.g., shooting conditions) of the data to be processed and the conditions of the analysis target, for at least one of the above-mentioned three processes. For example, separate processing functions may be provided for the process for images captured during the day and the process for images captured at night.

[0110] FIG. 21 shows a configuration in which the imaging device 110a and the auxiliary processing device 100a have a face detection processing function, a facial feature extraction processing function, and a facial feature matching processing function, and each can perform face authentication processing. Note that even if the functions of the imaging device 110a and the auxiliary processing device 100a are similar, the conditions suitable for using the functions are different. The imaging device processing list 2101 includes face detection A, facial feature detection A, and facial feature matching A. The auxiliary processing device processing list 2102 includes face detection B, facial feature extraction B, facial feature extraction C, and facial feature matching B. In addition, the imaging device 110a and the auxiliary processing device 100a may have multiple processing functions capable of performing similar processing, such as the auxiliary processing device 100a having two facial feature extraction processing functions (facial feature extraction B and C). In this way, by appropriately sharing the processing between the imaging device 110a and the auxiliary processing device 100a, processing suitable for various conditions can be performed.

[0111] Even when performing the same processing, the imaging device 110a and the auxiliary processing device 100a have different configurations, and therefore have advantages and disadvantages when sharing the processing. For example, with regard to calculation accuracy, the calculation processing unit 203 of the imaging device 110a has a large bit width for data and high calculation accuracy. With regard to calculation speed, the circuit 402 of the auxiliary processing device 100a performs calculations using a logic circuit, so the calculation speed is high. Also, when there are multiple processing functions capable of executing the same processing, it is advantageous to select an appropriate processing function based on the shooting environment of the imaging device 110a. In consideration of the above, when the auxiliary processing device 100a has a processing function, it is important to appropriately determine whether or not to use the processing function and appropriately select the processing function to be used.

[0112] In the following, a method for automatically selecting which of the auxiliary processing device 100a and the imaging device 110a should execute a process, and whether the imaging device 110a and the auxiliary processing device 100a should cooperate to execute a process, will be described. In addition, when the imaging device 110a or the auxiliary processing device 100a has multiple processing functions capable of executing the same process, for example, a method for automatically selecting which of the multiple processing functions should be used and which processing function should be used will be described. Note that, although three processing examples will be described individually below, these processes may be used in combination.

[0113] Fig. 22 describes a first processing example for selecting a processing function to be used. In Fig. 22, a processing function to be used is selected from the processing functions possessed by the image capture device 110a and the auxiliary processing device 100a so as to satisfy the performance required for image analysis processing. For example, this processing is executed when there is a condition that the processing needs to be performed at a frame rate equal to or higher than a certain level, and when both the image capture device 110a and the auxiliary processing device 100a are capable of executing the same processing.

[0114] 22, the user selects an analysis process to be executed via the user interface 1901 (S2201). The control unit 702 transmits an execution instruction command for the analysis process to be executed to the image capture device 110a based on the user selection. The control unit 304 acquires the execution instruction command from the control unit 702. Note that if the image capture device 110a has an information presentation function that presents executable processes and an operation reception function that allows the user to make a selection, the following aspect may be adopted. The user may directly operate the image capture device 110a to instruct the control unit 304 of the image capture device 110a on the analysis process to be executed.

[0115] The control unit 304 of the imaging device 110a checks the processing performance required to execute the analysis process selected by the user (S2202). The processing performance may be set in advance for each process. When selecting an analysis process, a target value may be set by the user. Then, the control unit 304 executes the analysis process selected in S2201 in the imaging device 110a (S2203). This process may be executed in parallel with the shooting. Furthermore, among the processing functions used in executing the selected process, functions that exist only in the auxiliary processing device 100a are executed by the auxiliary processing device 100a, and are not executed by the imaging device 110a.

[0116] The control unit 304 judges whether the executed processing satisfies the processing performance set in S2202 during processing or after completing a certain amount of data processing (S2204). If the control unit 304 judges that the processing performance is satisfied (YES in S2204), the processing returns to S2203 to continue the processing. On the other hand, if the control unit 304 judges that the processing performance is not satisfied (NO in S2204), the processing proceeds to S2205 to change the processing allocation to one that satisfies the processing performance.

[0117] In S2205, the execution subject of a part of the processing performed by the imaging device 110a that can also be executed by the auxiliary processing device 100a is changed to the auxiliary processing device 100a. The control unit 304 has already determined the processing that can be executed by the auxiliary processing device 100a. The control unit 304 selects the processing to be shared by the auxiliary processing device 100a from the list of the processing (second processing list) and changes the execution subject of the processing. After the change processing is completed, the control unit 304 and the analysis unit 501 share and execute the processing selected in S2201 (S2206). Thereafter, the control unit 304 determines whether or not to return the processing function of the auxiliary processing device 100a to the imaging device 110a (S2207). By returning the processing function to the imaging device 110a, the processing can be executed with high calculation accuracy.

[0118] For example, if the reason for determining that the processing performance is not satisfied in S2204 is a temporary high-load state, and if the state is resolved, the control unit 304 determines to return the processing function to the imaging device 110a. That is, the control unit 304 determines whether the imaging device 110a or the auxiliary processing device 100a will execute the processing function according to the processing load of the imaging device 110a. Note that, as described above, it is not limited to making the auxiliary processing device 100a execute the processing when the processing load of the imaging device 110a is high. It is also possible to make the imaging device 110a execute the processing when the processing load of the auxiliary processing device 100a is high. That is, it may be determined whether the imaging device 110a or the auxiliary processing device 100a will execute the processing based on the processing load of the auxiliary processing device 100a.

[0119] Furthermore, when the target value of the processing performance is lowered by the user, the control unit 304 determines whether to return the processing to the imaging device 110a. When the control unit 304 determines to return the processing to the imaging device 110a (YES in S2207), the processing proceeds to S2208. The execution subject of part of the processing executed by the auxiliary processing device 100a is changed to the imaging device 110a (S2208). Note that the processing whose execution subject is returned to the imaging device 110a in S2208 may be part or all of the processing whose execution subject is changed to the auxiliary processing device 100a in S2205. After the execution subject of at least part of the processing is returned to the imaging device 110a, the processing ends. On the other hand, when the control unit 304 determines not to return the processing function to the imaging device 110a (NO in S2207), the processing returns to S2206 and continues the processing without changing the processing allocation.

[0120] In addition, when the auxiliary processing device 100a has multiple processing functions that can execute the same process, the following aspect may be implemented. After the execution subject of part of the process is changed to the auxiliary processing device 100a, if it is determined that the processing performance is not satisfied, it may be switched to another processing function that executes the same function. That is, instead of switching the execution subject of the process in S2207, the execution subject may remain the auxiliary processing device 100a, and only the processing function to be used may be changed.

[0121] In addition, after the execution subject of a part of the process is changed to the auxiliary processing device 100a, if the processing performance confirmed in S2202 is not satisfied, the control unit 304 may return the execution subject of the process to the imaging device 110a. At this time, the control unit 304 stores information indicating the processing performance as information that cannot be satisfied by the currently attached auxiliary processing device 100a. Then, if the same processing performance or a stricter processing performance is required, the control unit 304 may not cause the auxiliary processing device 100a to execute the process. Similarly, if the processing performance cannot be satisfied even in a situation where the processing load of the imaging device 110a is sufficiently small, for example, the information on the processing performance may be stored. Here, in the case where the processing performance stored in S2202 or a stricter processing performance is confirmed in the later process, the control unit 304 executes the following mode. The control unit 304 changes the execution subject of a part of the process to the auxiliary processing device 100a without executing the process of S2203.

[0122] According to the first processing example, the processing functions of the image capture device 110a and the auxiliary processing device 100a are selected so as to satisfy the required processing performance. The selected processing functions are shared between these devices and processing is executed. This allows appropriate sharing of processing depending on, for example, the state of the image capture device 110a, making it possible to maintain good processing performance.

[0123] FIG. 23 describes a second processing example for appropriately selecting a processing function. This processing is executed to select a processing function suitable for use when the auxiliary processing device 100a has multiple processing functions capable of executing the same processing. Note that this processing is executed when it is determined in the first processing example that the auxiliary processing device 100a is to execute a part of the processing. That is, this processing determines which of one or more executable processing functions is to be used when the auxiliary processing device 100a executes the processing. This is only one example, and the processing allocation between the imaging device 110a and the auxiliary processing device 100a may be determined by this processing example. For example, this processing is applied when the imaging device 110a and the auxiliary processing device 100a have multiple processing functions capable of executing the same processing in the integrated processing list. In that case, this processing may be used to determine which processing function is to be used. That is, this processing determines which processing function of which device should be used to execute the processing when the imaging device 110a and the auxiliary processing device 100a each have one or more processing functions capable of executing the same processing.

[0124] In FIG. 22, similarly to S2201 in FIG. 22, the user selects an analysis process via the information processing device 130. The control unit 304 of the imaging device 110a acquires information indicating the selected process from the information processing device 130 (S2301). The control unit 304 determines a list of processes that the auxiliary processing device 100a can execute (second process list). The control unit 304 determines whether or not the auxiliary processing device 100a has multiple processing functions that can execute the same analysis process (S2302). If the control unit 304 determines that there is one processing function that can execute the analysis process (NO in S2302), the process proceeds to S2303. The control unit 304 executes the process using that processing function (S2303). On the other hand, if the control unit 304 determines that there are multiple processing functions that can execute the analysis process (YES in S2302), the process proceeds to S2304. A method of determining which of the multiple processing functions to use for processing will be described below.

[0125] In S2304, the control unit 304 determines the characteristics of the multiple processing functions capable of executing the same process determined in S2302. Here, the characteristic determination is, for example, a determination that the first processing function is suitable for processing images with relatively high brightness during the daytime with respect to facial feature extraction. Similarly, the control unit determines the characteristics that the second processing function is suitable for processing images with relatively low brightness during the night. After determining the difference in the characteristics of each processing function, the control unit 304 determines the environment in which the imaging device 110a is currently capturing images (S2305). The control unit 304 selects a processing function to be used for analysis processing based on the characteristics of each processing function acquired in S2304 and the information on the shooting environment acquired in S2305 (S2306). The control unit 304 executes the analysis processing using the selected processing function (S2307).

[0126] Here, the shooting environment may be determined based on, for example, the internal clock of the imaging device 110a and the distribution of the luminance values ​​of the image captured by the imaging device 110a. For example, when the internal clock indicates a nighttime time period, the control unit 304 selects a processing function suitable for processing an image with a relatively low luminance value. Also, when the luminance values ​​of the captured image are biased toward the low side, the control unit 304 selects a processing function suitable for image processing of an image with a relatively low luminance value. Also, a distribution of evaluation values ​​of detection accuracy for luminance values ​​may be prepared for each processing function. For example, a processing function having the best sum of values ​​obtained by multiplying the frequency of each luminance value of the captured image by a value indicating the detection accuracy for that luminance value and adding them may be selected. Also, the shooting environment may be determined based on, for example, information on the angle of view (pan, tilt, zoom) at the time of shooting of the imaging device 110a.

[0127] For example, which processing function is used is selected depending on whether a dark area in a room is being photographed or a bright area by a window. The characteristics of the processing functions may be defined by an index other than the brightness value. For example, various characteristics, such as high accuracy of face extraction in an image containing a specific object such as a window, high accuracy of detection of a fast-moving object, etc., may be used as a criterion for selecting the processing function to be used. Also, for example, each processing function may have a characteristic of high speed processing but low accuracy, or a characteristic of relatively slow processing but high accuracy, and an appropriate processing function may be selected depending on the processing conditions.

[0128] The control unit 304 determines whether the shooting environment has changed during the process (S2308). When the control unit 304 determines that the shooting environment has changed (YES in S2308), the process proceeds to S2306. The control unit 304 executes again the process of selecting a processing function suitable for the shooting environment after the change in environment (S2306). The control unit 304 executes the analysis process with the selected processing function (S2307). On the other hand, when the control unit 304 determines that the shooting environment has not changed (NO in S2308), it continues the analysis process without changing the processing function, and ends the process. According to this process, it becomes possible to select and use a processing function suitable for the environment from among a plurality of processing functions capable of executing the same process. This makes it possible to selectively use a processing function suitable for each shooting environment from the viewpoint of processing accuracy, etc.

[0129] 24 illustrates a third example of a process for determining the allocation of processing between the imaging device 110a and the auxiliary processing device 100a. In this process, the allocation of processing is determined depending on whether the processing can be completed using only a combination of the processing functions possessed by the auxiliary processing device 100a.

[0130] In FIG. 24, similarly to S2201 in FIG. 22 and S2301 in FIG. 23, the user selects an analysis process via the information processing device 130. The control unit 304 of the imaging device 110a acquires information indicating the selected process from the information processing device 130 (S2401). The control unit 304 determines whether the selected process can be executed only by the auxiliary processing device 100a (S2402). Note that the control unit 304 determines, for example, whether all the functions of the selected process can be fulfilled by a combination of the processing functions possessed by the auxiliary processing device 100a. The control unit 304 may also make the determination in S2402 based on whether the processing results may be stored in the auxiliary processing device 100a, etc. The control unit 304 determines that the processing can be executed only by the auxiliary processing device 100 if all the functions of the selected process can be fulfilled by a combination of the processing functions possessed by the auxiliary processing device 100a and the processing results may be stored therein.

[0131] If the control unit 304 determines that the selected process cannot be performed by the auxiliary processing device 100a alone (NO in S2402), the process proceeds to S2403. The control unit 304 allocates the process to the image capture device 110a and the auxiliary processing device 100a (S2403). At this time, the allocation of the process is determined as in the first and second process examples. Note that the control unit 304 does not need to use the processing functions of the auxiliary processing device 100a so that all of the process is performed by the image capture device 110a. On the other hand, if the control unit 304 determines that the selected process can be performed by the auxiliary processing device 100a alone (YES in S2402), the process proceeds to S2404. The control unit 304 selects which processing function of the auxiliary processing device 100a to use (S2404).

[0132] If the auxiliary processing device 100a has a plurality of processing functions capable of executing the same process, one of the above processing functions to be used is selected as in the second processing example. After that, the control unit 304 executes a process to cause the auxiliary processing device 100a to execute image analysis processing using the selected processing function (S2405). The control unit 304 also executes a process to store the results of the image analysis processing in S2405 in the auxiliary processing device 100a (S2406), and the process ends. These processes are executed using, for example, commands of the SD standard. In addition, in S2406, the image processing analysis results may be stored in the storage unit 404, and if the circuit 402 includes a RAM, the results may be stored in the RAM.

[0133] In this process, when the auxiliary processing device 100a can execute the process, the auxiliary processing device 100a executes the process. As a result, the process executed by the imaging device 110a is only the transmission of an image to the auxiliary processing device 100a, and the processing load of the imaging device 110a is significantly reduced. As described above, the auxiliary processing device 100a is used to increase the processing functions executable by the imaging device 110a, thereby enhancing the processing functions of the imaging system 10. For example, by implementing the latest processing functions in the auxiliary processing device 100a, it becomes unnecessary to replace the imaging device 110a. In addition, the imaging device 110a can execute image analysis processing using the latest processing functions. As a result, the imaging system 10 can be flexibly operated, and the convenience of the user can be improved.

[0134] FIG. 25 shows an example of a flow of processing in which the imaging device 110a performs fault diagnosis of the auxiliary processing device 100a. This processing is performed after the processing of S802 in FIG. 8. The storage unit 303 has setting parameters and a test image to be executed in the processing unit 412 in order to perform fault diagnosis (test processing) of the circuit 402. The storage unit 303 also has an expected value (signal information) which is an output value when processing is performed in the processing unit 412 using the test image and the setting parameters. Here, the input of the test processing is a 128x128 RGB image, and the output is 1000 8-bit data strings. Furthermore, the test processing performs a convolution operation used in deep learning and a ReLU operation once each. The test image is a gray image with 128 of 256 levels for each RGB value. The expected value is 0.5 for all 1000 8-bit data strings.

[0135] In FIG. 25, the imaging device 110a writes the setting parameters of the storage unit 303 to the storage unit 404 of the auxiliary processing device 100a. The auxiliary processing device 100a loads the setting parameters to the circuit 402 (S2501). The imaging device 110a may reconfigure the circuit 402 by transferring logic circuit data corresponding to the setting parameters to the circuit 402 as necessary. The arithmetic processing unit 203 of the imaging device 110a transfers the test image of the storage unit 303 to the storage unit 404 of the auxiliary processing device 100a (S2502). After the transfer of the test image is completed, when the storage unit 404 of the auxiliary processing device 100a stores the test image, the processing unit 412 of the circuit 402 executes test processing on the test image using the setting parameters (S2503). After the test processing, the imaging device 110a acquires the test processing result from the auxiliary processing device 100a (S2504). The test processing result is also called signal information.

[0136] The arithmetic processing unit 203 of the imaging device 110a compares the test processing result with the expected value corresponding to the test image in the storage unit 303. If the comparison result is a perfect match, the arithmetic processing unit 203 ends the process, and if the comparison result is not a match, the process proceeds to S2507 (S2505). At this time, the arithmetic processing unit 203 judges whether or not all 1000 values ​​of the test processing result and the expected value match. That is, the arithmetic processing unit 203 judges whether or not all 1000 values ​​of the test processing result are 0.5. If the arithmetic processing unit 203 judges that the comparison result does not satisfy the condition (No in S2505), the process proceeds to S2507. The arithmetic processing unit 203 judges that the auxiliary processing device 100a is broken (abnormal), and displays a message dialog on the display unit 703 (S2507). Here, it is judged that at least one of the processing unit 412 and the storage unit 404 included in the auxiliary processing device 100a is abnormal (broken). An example of the message dialog is shown in Fig. 26. In Fig. 26, screen S2601 displays "An error occurred during calculation processing on the device. Please contact the support center." When the user presses the OK button in Fig. 26, screen S2601 may be configured to disappear.

[0137] In addition, the test process is not limited to a test image, and may be any process as long as the output changes according to the input. Also, the test process may be a process that performs an operation according to a circuit implemented in the circuit 402. For example, when a Pooling operation used in deep learning is executed, the Pooling operation may be implemented in the circuit 402. Furthermore, there are cases where the amount of calculation of the calculation processing unit 203 is large, and the storage unit 404 is used as a storage area for temporary data during the calculation. The test process may be a small-scale process that does not use the storage unit 404. This makes it possible to accurately determine which of the processing unit 412 and the storage unit 404 of the circuit 402 is faulty.

[0138] The test image may be generated by the imaging device 110a. For example, an image in which all RGB values ​​are 0.5 may be generated by calculation, and the image may be transferred to the auxiliary processing device 100a. The setting parameters, the test image, and the expected value may be stored in any location. For example, the setting parameters and the expected value may be stored in advance in the storage unit 404 of the auxiliary processing device 100a, and they and the test processing result may be transferred to the imaging device 110. In addition, the message dialog displayed when the comparison results are different may be in any format as long as it notifies the user of a malfunction. Furthermore, when the comparison results are the same, a message may be displayed to notify the user that the auxiliary processing device 100a is normal. In addition, the notification may be in the form of output to a console, an event notification to another server connected to the imaging device 110a, or a message displayed to a client connected to the server.

[0139] Next, an example in which two types of storage units are installed in the auxiliary processing device 100a will be described. This process is performed at any timing by the user after S804 in FIG. 8, but here, an example in which it is performed after the process of S806 will be described. FIG. 27 shows an example of the hardware configuration of the auxiliary processing device 100a. Explanation of the same components as in FIG. 4 will be omitted. The auxiliary processing device 100a includes a storage unit A405 and a storage unit B406 instead of the storage unit 404 in FIG. 4. The storage unit A405 is a non-volatile memory (ROM) in which information is stored even when the power is turned off, and stores setting parameters to be processed in the processing unit 412 of the circuit 402. The storage unit B406 is a volatile memory (RAM) in which information is erased when the power is turned off, and stores input data required for processing in the processing unit 412.

[0140] The setting parameters in the test process are the same as the setting parameters used in the image analysis process. Here, a case where face detection is performed as the image analysis process will be described in particular. At this time, as in the first embodiment, the input is a 128x128 RGB image, and the output is 1000 8-bit data strings. The test image is a gray image with 128 of 256 RGB values, and the expected value of all 1000 8-bit data strings is 0.5. The setting parameters used in the image analysis process are stored in the storage unit A405 and the storage unit 303 of the imaging device 110a, and the test image and the expected value are stored in the storage unit 303. In addition, the storage unit 303 holds a reinstalled flag indicating that the setting parameters used in the image analysis process have been reinstalled in the storage unit A405. The storage unit 303 holds a restarted flag indicating that the auxiliary processing device 100a has been restarted in the test process. Before the test process is performed, all of the above flags are in the OFF state.

[0141] FIG. 28(a) shows an example of the failure determination process of the auxiliary processing device 100a. In FIG. 28(a), the arithmetic processing unit 203 of the imaging device 110a transfers the test image in the storage unit 303 to the storage unit A405 (S2801). After the transfer of the test image is completed, the auxiliary processing device 100a causes the circuit 402 (processing unit 412) to execute analysis processing on the test image in the storage unit A405 with the setting parameters for image analysis processing (S2802). After the analysis processing by the circuit 402 (processing unit 412), the imaging device 110a obtains the analysis result A (signal information) from the auxiliary processing device 100a (S2803). Next, the arithmetic processing unit 203 transfers the test image in the storage unit 303 to the storage unit B406 (S2804). After the transfer of the test image is completed, the auxiliary processing device 100a causes the circuit 402 (processing device 412) to execute analysis processing on the test image in the memory unit B 406 using the set parameters for image analysis processing (S2805). After the analysis processing by the circuit 402 (processing device 412), the imaging device 110a obtains the analysis result B (signal information) from the auxiliary processing device 100a (S2806).

[0142] The arithmetic processing unit 203 compares the analysis result A with the analysis result B and the expected value in the storage unit 303. When the arithmetic processing unit 203 determines that the analysis result A and the expected value and the analysis result B and the expected value completely match (Yes in S2807), the process ends. When the arithmetic processing unit 203 determines that either or both of the analysis result A and the expected value and the analysis result B and the expected value do not match (No in S2807), the process proceeds to S2808. Here, the arithmetic processing unit 203 determines whether or not the values ​​of the 1000 8-bit data strings that are the outputs of the analysis result A and the analysis result B all match the expected values. That is, the arithmetic processing unit 203 determines whether or not all of the 1000 values ​​of the analysis result A and the analysis result B are 0.5. Here, it is determined that at least one of the processing unit 412, the storage unit A405, and the storage unit B406 included in the auxiliary processing device 100a is abnormal (broken). The calculation processing unit 203 performs detailed fault determination processing based on the determination result (S2808).

[0143] Next, the detailed fault determination process will be described. The comparison result between each analysis result (A, B) and the expected value, and the corresponding relationship of the fault in this embodiment will be described. In this embodiment, a fault in the memory unit B406 and a fault in the processing unit 412 are determined. Note that the memory unit A405 is in a normal state when S802 is normally completed as the initial sequence and this sequence is executed. FIG. 28(b) shows an example of the detailed fault determination process of the auxiliary processing device 100a.

[0144] In FIG. 28(b), the calculation processing unit 203 calculates the analysis result A and the expected value, and the analysis result B and the expected value Both Further, the calculation processing unit 203 judges whether the display result of the reinstallation completed flag is OFF or not (S2810). That is, the calculation processing unit 203 judges whether both of the two conditions are satisfied, and if both conditions are satisfied, the process proceeds to S2811. If the analysis result A and the expected value, and the analysis result B and the expected value do not match, and the reinstallation completed flag is OFF (S2810), the calculation processing unit 203 Yes ), the processing unit 412 determines that an abnormality has occurred. No ), processing proceeds to S2812.

[0145] The arithmetic processing unit 203 reinstalls the setting parameters for image analysis processing in the storage unit 303 to the storage unit A 405, and turns the reinstalled flag ON (S2811). At this time, a message dialog stating "Setting parameters will be reinstalled to the device" is displayed on the display unit 703. Thereafter, the test processing from S2801 onwards is resumed (S2814), and the processing ends. Returning to S2812, the arithmetic processing unit 203 determines whether analysis result A matches the expected value and analysis result B does not match the expected value, and determines whether the restarted flag is OFF (S2812). If the arithmetic processing unit 203 determines that analysis result A matches the expected value and analysis result B does not match the expected value, and the restarted flag is OFF (S2812), Yes ), the memory unit B 406 is determined to be abnormal. Furthermore, the calculation processing unit 203 determines to restart the auxiliary processing device 100a, and the process proceeds to S2813. If the calculation processing unit 203 does not satisfy the condition of S2812, that is, if analysis result A does not match the expected value and analysis result B matches the expected value and the restart completion flag is ON (S2812 No ), processing proceeds to S2815.

[0146] In S2813, the arithmetic processing unit 203 turns the restarted flag ON and restarts the auxiliary processing device 100a after one minute (S2813). At this time, a message dialog stating "Restarting will be performed in one minute" is displayed on the display unit 703. After that, the test process from S2801 onwards is executed again (S2814), and the process ends. If it is determined that the processing unit 412 is abnormal and the reinstalled flag is ON, the arithmetic processing unit 203 determines that the processing unit 412 is faulty (abnormal). Alternatively, if it is determined that the storage unit B406 is abnormal and the restarted flag is ON, the arithmetic processing unit 203 determines that the storage unit B406 is faulty. The arithmetic processing unit 203 displays the fault target in a message dialog on the display unit 703 (S2815). For example, after the message "An error occurred during arithmetic processing in the device. Please contact the support center", the following message is displayed. The display unit 703 displays a message such as "The arithmetic processing unit is abnormal" or "The memory is abnormal".

[0147] When the setting parameters are reinstalled in S2811, logic circuit data corresponding to the setting parameters may be transferred to the circuit 402 as necessary, and may be reconfigured. Alternatively, only a message prompting the user to reinstall the setting parameters may be displayed, and the user may reinstall the setting parameters. In the determination condition in S2812, a restarted flag indicating whether or not the restart has been performed is used, but instead, a counter that counts the number of restarts may be used. For example, the restart may be performed until the restart of the processing unit 412 reaches a threshold, and when the threshold is exceeded, it may be determined that the storage unit B406 is faulty. The waiting time until the auxiliary processing device 100a is restarted in S2813 may be changed according to the state of the auxiliary processing device 100a. For example, when the temperature of the auxiliary processing device 100a is lower than a threshold, the auxiliary processing device 100a may be restarted. Alternatively, only a message prompting the user to restart the auxiliary processing device 100a may be displayed, and the user may restart the auxiliary processing device 100a.

[0148] In the above embodiment, the analysis process is an image analysis process, but the present invention is also applicable to voice analysis. Specifically, the present invention is applicable to a process for detecting voice patterns such as screams, gunshots, and glass breaking sounds. For example, various voice data analysis methods such as spectrum analysis are used to extract voice features, which are then compared with the detected voice pattern. A specific voice pattern can be detected based on the degree of agreement between the features and the detected voice pattern. When performing voice analysis, voice data is divided into voice data for a predetermined time, and the voice analysis process is performed with the voice data for the predetermined time as a unit. The predetermined time is changed according to the voice pattern to be detected. Therefore, voice data for each time corresponding to the voice pattern to be detected is input to the auxiliary processing device 100a. The auxiliary processing device 100a has a means for analyzing the input voice data and a means for holding the input voice data.

[0149] In the above embodiment, the auxiliary processing device 100a capable of non-temporarily storing data input from the imaging device 110a has been described as an example. However, in some embodiments, the auxiliary processing device 100a that cannot non-temporarily store data input from the imaging device 110a may be used. That is, the auxiliary processing device 100a may only perform analysis processing on the data input from the imaging device 110a, and may not non-temporarily store the data. That is, the auxiliary processing device 100a may not have a data holding function like a normal SD card, but may have a function specialized for analysis processing.

[0150] (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0151] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0152] 10: imaging system, 100a to 100d: auxiliary processing devices, 110a to 110d: imaging devices, 120: network, 130: information processing device

Claims

1. An imaging device having an attachment part to which the device can be attached and detached, a storage means for storing first signal information obtained by processing a captured first image; a transmitting means for transmitting the first image to the device attached to the attachment portion; a receiving means for receiving second signal information resulting from the analysis and processing of the first image by the device; a determination unit that determines whether or not the device is abnormal based on whether or not the first signal information and the second signal information match, 1. An imaging device comprising:

2. the second signal information is obtained by processing the first image stored in a storage means of the device by a processing means of the device; The determination means determines that at least one of the storage means of the device and the processing means of the device is abnormal based on whether the first signal information and the second signal information match.

2. The imaging device according to claim 1 .

3. The determination means determines that the processing means of the device is abnormal when the first signal information and the second signal information do not match.

3. The imaging device according to claim 2.

4. When the first signal information and the second signal information match, the determination means determines that the processing means and the storage means of the device are not abnormal.

4. The imaging device according to claim 2, wherein the first and second lenses are arranged in a first direction.

5. a notification means for notifying a user whether or not the device is abnormal; 5. The imaging device according to claim 1, wherein the first and second lenses are arranged in a first direction.

6. a storage means for storing in advance parameters for the device to process the first image; 6. The imaging device according to claim 1, wherein the first and second lenses are arranged in a first direction.

7. when the determination means determines that the device is abnormal, reinstalling parameters for processing the first image in a storage means of the device; 7. The imaging device according to claim 6.

8. When the determination means determines that the storage means of the device is abnormal, the device is restarted.

2. The imaging device according to claim 1 .

9. The determination means determines whether or not the storage means of the device is abnormal based on the number of times the device has been restarted.

9. The imaging device according to claim 8.

10. the receiving means receives third signal information obtained by a processing means of the device that processes the first image stored in another storage means of the device; the determination means determines whether or not either the processing means of the device or the storage means of the device is abnormal based on whether or not the first signal information matches the second signal information and whether or not the first signal information matches the third signal information.

3. The imaging device according to claim 2.

11. the determination means determines that the processing means of the device is abnormal when the first signal information does not match the second signal information and the first signal information does not match the third signal information; 11. The imaging device according to claim 10.

12. the determination means determines that the storage means of the device is abnormal when the first signal information and the second signal information match and the first signal information and the third signal information do not match.

12. The imaging device according to claim 10 or 11.

13. a first flag indicating whether parameters for processing the first image have been reinstalled in the storage means of the device, and a second flag indicating whether the processing means of the device has been rebooted; the determination means further determines whether or not the processing means of the device is abnormal and whether or not the storage means of the device is abnormal, based on the first flag and the second flag.

13. The imaging device according to claim 10, wherein the imaging device is a single lens.

14. the determination means determines that the processing means of the device is abnormal when the first signal information does not match the second signal information and when the first signal information does not match the third signal information, and when the first flag indicates that the parameter has been reinstalled in the storage means of the device.

14. The imaging device according to claim 13.

15. the determination means determines that the storage means of the device is abnormal when the first signal information and the second signal information match and the first signal information and the third signal information do not match, and when the second flag indicates that the processing means of the device has been restarted.

15. The imaging device according to claim 13 or 14.

16. The storage means of the device is a volatile storage means.

2. The imaging device according to claim 1 .

17. The other storage means of the device is a non-volatile storage means.

11. The imaging device according to claim 10.

18. The device is an SD card.

18. The imaging device according to claim 1,

19. An imaging control method executed by an imaging device having an attachment part to which the device can be detached, comprising: a storage step of storing first signal information of the captured first image; a transmitting step of transmitting the first image to the device attached to the mounting portion; a receiving step of receiving second signal information resulting from analysis and processing of the first image by the device; a determination step of determining whether or not the device is abnormal based on whether or not the first signal information and the second signal information match each other.

4. An imaging control method comprising:

20. A program for causing a computer to function as each of the means of the imaging apparatus according to any one of claims 1 to 18.

Images