Imaging apparatus, method for controlling imaging apparatus, device, and method for controlling device

Abstract

To cause a device mounted to an imaging apparatus detachably to operate in a range of a temperature lower than a fixed value.SOLUTION: Temperature information indicative of a temperature of the device mounted to the imaging apparatus detachably is acquired from the device and the execution frequency of a processing by the device is controlled based on the temperature information.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an operation control technology for a device detachably attached to an imaging device.

Background Art

[0002] In recent years, image analysis such as detecting an object and estimating an attribute using an image captured by a surveillance camera has been performed. Conventionally, the video of the surveillance camera has been transferred to a high-performance computing device such as a PC or a server, and the computing device has performed image analysis on the video. However, with the improvement of the processing power of mobile computing devices, image analysis is increasingly being performed on the surveillance camera side. As an implementation form, in addition to arranging a computing device in the surveillance camera main body, a form of connecting a computing device to the surveillance camera via USB or the like has also been proposed.

[0003] In the latter form, the computing device is a "detachable device detachably connected to the surveillance camera". The "detachable device detachably connected to the surveillance camera" is often small, and such a detachable device often uses a computing device capable of performing advanced arithmetic processing, so the power consumption tends to be high with respect to the size of the housing. Furthermore, in recent years, the multifunctionality of surveillance cameras has advanced, and the power consumption of the surveillance cameras themselves has also increased. Therefore, depending on the usage environment, there is a possibility that the operating guaranteed temperature of the components used in the surveillance camera and the detachable device may be exceeded. When the surveillance camera is not designed assuming the attachment of the detachable device, the temperature near the connector to which the detachable device is attached cannot be obtained, and normal operation may be performed without detecting that the operating guaranteed temperature of the components has been exceeded, which may cause the device to break down.

[0004] Patent Document 1 discloses a method of reducing the power consumption and maintaining the body temperature by reducing the frequency of the drive clock of the in-vehicle device and slowing down the processing speed if the acquired temperature is equal to or higher than a threshold value.

Prior Art Documents

Patent Documents

[0005] [Patent Document 1] Japanese Patent Publication No. 2019-183650 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] However, the technology described in Patent Document 1 does not consider the exchange of temperature information between devices, and therefore cannot address the case where a detachable device is attached to an imaging device. The present invention provides a technology for operating a device that is detachably attached to an imaging device within a range where the temperature of the device does not exceed a certain level. [Means for solving the problem]

[0007] One embodiment of the present invention is an imaging device, characterized by comprising: an acquisition means for acquiring temperature information indicating the temperature of a device detachably attached to the imaging device from the device; and a control means for controlling the execution frequency of a process performed by the device based on the temperature information. [Effects of the Invention]

[0008] According to the configuration of the present invention, a device that is detachably attached to an imaging device can be operated within a range where the temperature of the device does not exceed a certain level. [Brief explanation of the drawing]

[0009] [Figure 1] A block diagram showing an example of the system configuration. [Figure 2] A block diagram showing an example of the hardware configuration of the imaging device 110. [Figure 3] A block diagram showing an example of the functional configuration of the imaging device 110. [Figure 4] A block diagram showing an example hardware configuration of the detachable device 100. [Figure 5]A block diagram showing an example of the functional configuration of the detachable device 100. [Figure 6] A block diagram showing an example of the hardware configuration of the input / output device 130. [Figure 7] A block diagram showing an example of the functional configuration of the input / output device 130. [Figure 8] A flowchart illustrating the system's operation. [Figure 9] A flowchart of the part relating to the process for controlling the frequency of execution of a process in the detachable device 100 according to the temperature of the detachable device 100. [Figure 10] A block diagram showing an example hardware configuration of the detachable device 100. [Figure 11] A flowchart of the part relating to the process for controlling the frequency of execution of a process in the detachable device 100 according to the temperature of the detachable device 100. [Modes for carrying out the invention]

[0010] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention to the claims. While the embodiments describe multiple features, not all of these features are essential to the invention, and the features may be combined in any way. Furthermore, in the attached drawings, the same or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0011] [First Embodiment] <System Configuration> First, an example of the system configuration according to this embodiment will be explained using the block diagram in Figure 1. As shown in Figure 1, the system according to this embodiment includes detachable devices 100a to 100d, imaging devices 110a to 110d, and an input / output device 130. The imaging devices 110a to 110d and the input / output device 130 are configured to communicate with each other via a network 120.

[0012] First, imaging devices 110a to 110d will be described. The imaging devices 110a to 110d are imaging devices such as network cameras, which capture moving images or capture still images periodically or irregularly. When the imaging devices 110a to 110d capture still images periodically or irregularly, they output the captured still images as captured images. On the other hand, when the imaging devices 110a to 110d capture moving images, they output the images of each frame in the moving images as captured images.

[0013] In the following description, unless otherwise distinguished, the imaging devices 110a to 110d are referred to as imaging device 110. That is, the following description regarding imaging device 110 is equally applicable to each of the imaging devices 110a to 110d.

[0014] In this embodiment, it is assumed that the imaging device 110 has a built-in arithmetic unit capable of processing captured images, but it is not limited to this. For example, an external computer such as a PC (personal computer) may be connected to the imaging device 110 as an arithmetic unit, and these combinations may be treated as the imaging device 110.

[0015] Next, removable devices 100a to 100d will be described. The removable device 100x (x = a to d) is a removable arithmetic unit for the imaging device 110x, stores the captured images by the imaging device 110x, and performs arithmetic processing including analysis processing on the captured images. For example, when the imaging device 110x has a "slot for detachable device capable of recording captured images", the removable device 100x can be connected to the imaging device 110x by inserting the removable device 100x into the slot.

[0016] In the following description, unless otherwise distinguished, the removable devices 100a to 100d are referred to as removable device 100. That is, the removable device 100 is a device removable from the imaging device 110, and the following description regarding the removable device 100 is equally applicable to each of the removable devices 100a to 100d.

[0017] For such a detachable device 100, for example, a device having a configuration in which a predetermined processing circuit is mounted on an SD card can be applied. The detachable device 100 can be configured to be entirely insertable into the imaging device 110 in the form of, for example, an SD card, and thereby can be configured to be connectable to the imaging device 110 without any part protruding from the imaging device 110. This can prevent the detachable device 100 from interfering with obstacles such as wiring, and can enhance the convenience during use of the device. Also, since many existing imaging devices 110 such as network cameras have SD card slots, an extended function can be provided to the existing imaging devices 110 by the detachable device 100. In the present embodiment, the detachable device 100 will be described as a device having a configuration in which a predetermined processing circuit is mounted on an SD card. However, the detachable device 100 may be configured to be mounted on the imaging device 110 by any interface used when at least a storage device capable of storing the captured image captured by the imaging device 110 is attached, other than the form of an SD card. For example, the detachable device 100 may have a USB interface and may be configured to be mounted on the USB socket of the imaging device 110. Also, the predetermined processing circuit is implemented, for example, by an FPGA (Field Programmable Gate Array) programmed to execute a predetermined process, but may be implemented in other forms. The clock of the FPGA is generated by a crystal oscillator configured within the detachable device 100. Alternatively, the clock for SD communication may be used.

[0018] In the present embodiment, it is assumed that the detachable device 100 is mounted on all (four in FIG. 1) of the imaging devices 110. However, the system may include an imaging device 110 to which the detachable device 100 is not connected, not limited to the case where the detachable device 100 is mounted on all of the imaging devices 110.

[0019] By attaching a detachable device 100 having a function for performing calculation processing (calculation processing function) to the imaging device 110, it is possible to obtain calculation processing results for the captured image by the imaging device 110, even if the imaging device 110 does not have a calculation processing function.

[0020] Furthermore, in this embodiment, where the computing device is located on the imaging device 110, the detachable device 100 on which the computing device is located can be attached to the imaging device 110, thereby diversifying and enhancing the image processing that can be performed on the imaging device 110.

[0021] Next, the input / output device 130 will be described. The input / output device 130 is a computer device such as a PC (personal computer), smartphone, or tablet terminal, and it displays the captured images transmitted from the imaging device 110 and accepts various instructions from the user.

[0022] Next, the network 120 will be described. The network 120 is composed of multiple routers, switches, cables, etc. that meet a communication standard such as Ethernet®. In this embodiment, the network 120 may be any network that enables communication between the imaging device 110 and the input / output device 130, and can be constructed with any size, configuration, and compliant communication standard. For example, the network 120 may be the Internet, a wired LAN (Local Area Network), a wireless LAN, a WAN (Wide Area Network), etc. Furthermore, the network 120 may be configured to enable communication using a communication protocol compliant with the ONVIF (Open Network Video Interface Forum) standard, for example. However, it is not limited to this, and the network 120 may be configured to enable communication using other communication protocols, such as a proprietary communication protocol.

[0023] <Configuration of each device> (Example configuration of imaging device 110) An example of the hardware configuration of the imaging device 110 will be explained using the block diagram in Figure 2. Note that Figure 2 shows the main components relevant to the following explanation and does not show all components of the imaging device 110.

[0024] The imaging unit 201 includes a lens unit for forming an image of light and an image sensor that outputs an analog signal corresponding to the light formed by the lens unit. The lens unit has a zoom function for adjusting the angle of view and an aperture function for adjusting the amount of light. The image sensor has a gain function for adjusting the sensitivity when converting light into an analog signal. These functions are adjusted based on setting values ​​output from the image processing unit 202.

[0025] The image processing unit 202 includes an A / D converter, an image processing engine, and peripheral devices. The A / D converter converts the analog signal output from the imaging unit 201 (image sensor) into a digital signal. The image processing engine generates an image by performing image processing such as development, filtering, sensor correction, and noise reduction on the RAW image represented by the digital signal. The image processing engine can also send set values ​​to the lens unit and image sensor to perform exposure adjustment so that an image with appropriate exposure can be obtained. Peripheral devices include, for example, RAM (Random Access Memory) and drivers for each I / F. The image is output to the arithmetic processing unit 203 via the peripheral devices.

[0026] The arithmetic processing unit 203 includes one or more processors such as a CPU or MPU, memory such as RAM or ROM, and drivers for each I / F. For example, the arithmetic processing unit 203 determines which processes to be executed by the imaging device 110 and which processes to be executed by the detachable device 100, and executes the processes determined to be executed by the imaging device 110. The arithmetic processing unit 203 also outputs the information necessary for the processes determined to be executed by the detachable device 100 to the detachable device 100, causing the detachable device 100 to execute those processes. The arithmetic processing unit 203 also outputs the captured images to the distribution unit 204 and the SD I / F unit 205. Furthermore, if the arithmetic processing unit 203 has performed processing on the captured images, it outputs the results of the processing to the distribution unit 204.

[0027] The distribution unit 204 includes a network distribution engine and peripheral devices such as RAM and an ETH PHY module. The ETH PHY module is a module that performs processing on the physical (PHY) layer of Ethernet. The distribution unit 204 converts the captured images and processing results output from the arithmetic processing unit 203 into "data in a format that can be distributed to the network 120," and transmits the converted data to the input / output device 130 via the network 120.

[0028] The SD I / F unit 205 is an interface for connecting to the detachable device 100, and includes, for example, a power supply and a mounting mechanism such as a detachable socket for attaching and detaching the detachable device 100. Here, the SD I / F unit 205 is assumed to be configured in accordance with the SD standard established by the SD Association. The imaging device 110 performs data communication with the detachable device 100 via the SD I / F unit 205.

[0029] Next, an example of the functional configuration of the imaging device 110 will be explained using the block diagram in Figure 3. The functional units shown in Figure 3 may be implemented as hardware incorporated into any of the functional units shown in Figure 2, or as software executed by any of the functional units shown in Figure 2.

[0030] The imaging control unit 301 controls the operation of the imaging unit 201. For example, based on the above-mentioned set values, the imaging control unit 301 controls the operation of the imaging unit 201 so that it captures the surrounding environment under appropriate exposure conditions.

[0031] The signal processing unit 302 generates an image as described above from the analog signal output from the imaging unit 201 controlled by the imaging control unit 301. The signal processing unit 302 may also encode the generated image. When the imaging unit 201 captures a still image, the signal processing unit 302 encodes the still image using an encoding method such as JPEG (Joint Photographic Experts Group). When the imaging unit 201 captures a moving image, the signal processing unit 302 encodes the moving image using an encoding method such as H.264 / MPEG-4 AVC (hereinafter referred to as "H.264") or HEVC (High Efficiency Video Coding). The signal processing unit 302 may also perform encoding using an encoding method selected by the user from among a plurality of pre-set encoding methods, for example, via an operation unit (not shown) of the imaging device 110.

[0032] The memory unit 303 stores a list of analysis processes that the analysis unit 305 can execute (hereinafter referred to as the first processing list) and a list of post-processing steps for the results of the analysis processes. The memory unit 303 also stores the results of the analysis processes performed by the analysis unit 305.

[0033] In this embodiment, the process to be executed is an analysis process, but any process may be executed, and the storage unit 303 stores a first process list and a post-processing list for processes related to the process to be executed.

[0034] The control unit 304 controls the operation of the imaging device 110, and for example, it controls the operation of each functional unit shown in Figure 3.

[0035] The analysis unit 305 selectively performs at least one of the following on the captured image generated by the signal processing unit 302: pre-analysis processing, analysis processing, and post-analysis processing.

[0036] Pre-processing is a process performed on an captured image before performing analysis on the captured image. In this embodiment, as an example, the pre-processing involves dividing the captured image and creating divided images.

[0037] The analysis process is a process that outputs information obtained by analyzing an input image. In the analysis process of this embodiment, as an example, the segmented image obtained by the pre-analysis processing is used as the input image, and at least one of the analysis processes of human body detection, face detection, or vehicle detection is performed on the input image, and the results of the analysis process are output. The analysis process may be configured to output the position of objects in the segmented image using a machine learning model that has been trained to detect objects contained in the image, for example, by the technology disclosed in Non-Patent Document 1.

[0038] Post-analysis processing is performed after the analysis process has been executed. In this embodiment, as an example, the post-analysis processing outputs the sum of the number of objects detected in each segmented image, based on the results of the analysis process for each segmented image, as the "result of the analysis process." The analysis process may also be a process that detects objects in an image by performing pattern matching and outputs their positions. Hereafter, these processes (pre-analysis processing, analysis process, and post-analysis processing) may be collectively referred to as the analysis process.

[0039] The temperature processing unit 308 extracts "temperature information indicating the temperature in the detachable device 100 (temperature inside the detachable device 100)" from the data received from the detachable device 100, and compares the temperature indicated by the temperature information with a threshold value.

[0040] The temperature processing unit 308 then controls the frequency of processing (calculation processing) in the detachable device 100 in order to lower the temperature in the imaging device 110 (temperature inside the imaging device 110) and the temperature in the detachable device 100 if the temperature indicated by the temperature information is above a threshold. While a higher processing frequency allows for faster processing, the increased processing load leads to increased power consumption and higher temperatures in the detachable device 100 and the imaging device 110. Therefore, in this embodiment, if the temperature of the detachable device 100 is above a threshold, the temperature rise is prevented by reducing the frequency of processing in the detachable device 100.

[0041] On the other hand, if the temperature indicated by the temperature information is below a threshold, the temperature processing unit 308 controls the execution frequency of processing (calculation processing) in the detachable device 100 to increase it according to the required execution frequency, or controls it to keep the execution frequency constant.

[0042] The frequency of processing in the detachable device 100 is dynamically determined by repeatedly comparing the temperature indicated by the temperature information received from the detachable device 100 with a threshold value, and controlling the frequency of processing according to the result of the comparison, in accordance with the operating environment at the time.

[0043] Here, in order to reduce the time required to determine the execution frequency of processing in the detachable device 100 and the processing load for such determination, the following configuration may be adopted, for example.

[0044] It is preferable to store the temperature and processing frequency of the currently running detachable device 100 in the storage unit 303, and then, when running this arithmetic process again, determine the processing frequency from the stored information, thereby reducing the time and processing required to determine the processing frequency by comparing the temperature information and the processing frequency.

[0045] In this way, for each calculation process, a table of processing frequencies corresponding to the temperature of the detachable device 100 is stored, and the corresponding processing frequency is determined using this table based on the temperature information obtained from the detachable device 100.

[0046] Here, the method by which the temperature processing unit 308 controls the execution frequency of processing in the detachable device 100 is not limited to a specific control method. For example, the execution frequency of processing in the detachable device 100 may be controlled by changing the operating frequency supplied from the imaging device 110 (temperature processing unit 308) to the detachable device 100. Specifically, the temperature processing unit 308 may increase or decrease the execution frequency of processing in the detachable device 100 (FPGA 402) by increasing or decreasing the operating frequency used for arithmetic processing in the FPGA 402 described later in the detachable device 100. Also, if the temperature processing unit 308 is using the SD clock (communication frequency) as the clock for arithmetic processing in the FPGA 402, it may increase or decrease the execution frequency of processing in the detachable device 100 (FPGA 402) by increasing or decreasing the SD clock.

[0047] Alternatively, for example, the temperature processing unit 308 may increase or decrease the frequency of arithmetic processing instructions supplied from the imaging device 110 (temperature processing unit 308) to the detachable device 100 (FPGA 402) by increasing or decreasing the frequency of these instructions (shortening or widening the intervals between the supply of arithmetic instructions), thereby increasing or decreasing the frequency of processing execution in the detachable device 100 (FPGA 402).

[0048] The threshold values ​​used for the above comparison are set by the temperature processing unit 308 so as to enable control that the temperature in the imaging device 110 and the temperature in the detachable device 100 are appropriately maintained. For example, the memory unit 303 stores a temperature that has been pre-set as the upper limit of the temperature in the imaging device 110 as "temperature upper limit A". The memory unit 404 of the detachable device 100, described later, stores a temperature that has been pre-set as the upper limit of the temperature in the detachable device 100 as "temperature upper limit B". When the imaging device 110 and the detachable device 100 are both started up and able to communicate data with each other, the imaging device 110 sends a request to the detachable device 100 to transmit "temperature upper limit B", and receives "temperature upper limit B" transmitted by the detachable device 100 in response to the transmission request. The temperature processing unit 308 then compares the magnitude of "temperature limit A" stored in the memory unit 303 with "temperature limit B" received from the detachable device 100, and sets the lower of "temperature limit A" and "temperature limit B" as the threshold value.

[0049] The device communication unit 306 performs data communication with the detachable device 100 via the SD I / F unit 205. The device communication unit 306 converts the transmission data provided as data to be sent to the detachable device 100 into a transmission data format that the detachable device 100 can process, and transmits the converted transmission data to the detachable device 100 via the SD I / F unit 205. The device communication unit 306 also receives data transmitted from the detachable device 100 and converts the received data into a data format that the imaging device 110 can process. In this embodiment, the device communication unit 306 performs a conversion process that converts decimal numbers between floating-point format and fixed-point format, but is not limited to this, and other processes may be performed by the device communication unit 306. In this embodiment, the device communication unit 306 communicates with the detachable device 100 by sending a predetermined command sequence within the range of the SD standard to the detachable device 100 and receiving a response from the detachable device 100. The network communication unit 307 performs data communication with the input / output device 130 via the network 120.

[0050] <Example configuration of detachable device 100> Next, an example of the hardware configuration of the detachable device 100 will be explained using the block diagram in Figure 4. Note that Figure 4 shows the main configurations relevant to the following explanation and does not show all configurations of the detachable device 100.

[0051] The detachable device 100 has a computing unit, which is composed of an FPGA or the like. The detachable device 100 is shaped to be insertable into and removeable from the detachable socket of the SD I / F unit 205 of the imaging device 110, that is, it is molded in a shape that conforms to the SD standard.

[0052] The I / F unit 401 is an interface section for connecting the imaging device 110 and other devices to the detachable device 100. The I / F unit 401 is configured to include, for example, electrical contact terminals that receive power from the imaging device 110 and generate and distribute power for use within the detachable device 100. The I / F unit 401, like the SD I / F unit 205 of the imaging device 110, shall conform to the items defined (compliant) within the SD standard. The detachable device 100 receives various data, such as captured images, transmitted from the imaging device 110 via the I / F unit 401. The detachable device 100 also transmits data output from the FPGA 402 to the imaging device 110 via the I / F unit 401.

[0053] FPGA402 is a type of semiconductor device whose internal logic circuit structure can be repeatedly reconfigured. The processing implemented by FPGA402 can add (provide) processing functions to a device to which the detachable device 100 is installed. Furthermore, because the reconfiguration function of FPGA402 allows the logic circuit structure to be changed later, by installing the detachable device 100 in a device in a field where technology is rapidly advancing, for example, it becomes possible to perform appropriate processing in a timely manner in that device. In this embodiment, an example in which FPGA is used is described, but a general-purpose ASIC or a dedicated LSI may be used, for example, as long as the processing described later can be realized. FPGA402 is started when configuration data containing information on the generated logic circuit structure is written to or read from a dedicated I / F. In this embodiment, this configuration data is assumed to be held in the storage unit 404. When power is turned on, FPGA402 reads the configuration data from the storage unit 404, generates the logic circuit, and starts up. However, this is not limited to the above. For example, by implementing a dedicated circuit within the detachable device 100, the imaging device 110 may write setting data to the FPGA 402 via the I / F unit 401. In this embodiment, the FPGA 402 performs calculation processing on the captured image received from the imaging device 110 via the I / F unit 401 and outputs the result of the calculation processing.

[0054] The input / output control unit 410 includes a circuit for controlling the transmission and reception of data with the imaging device 110, a circuit for analyzing commands received from the imaging device 110, and a circuit for performing control based on the results of analyzing the commands. The commands here are those defined in the SD standard, and the input / output control unit 410 can detect some of them. Details of the functions will be described later. When the input / output control unit 410 receives an image to be processed from the imaging device 110, it outputs the image to the calculation processing unit 412. Also, when the input / output control unit 410 receives an initialization sequence and a reconfiguration command from the imaging device 110, it outputs the initialization sequence and the reconfiguration command to the temperature acquisition unit 414. Furthermore, when the input / output control unit 410 receives setting data for switching processes from the imaging device 110, it outputs the setting data to the calculation switching unit 411.

[0055] The calculation switching unit 411 is configured to include a circuit for obtaining information on the functions of calculation processing, including analysis processing of the captured image, from the storage unit 404 based on setting data received from the imaging device 110, and for writing this information to the calculation processing unit 412. This information is, for example, setting parameters that indicate the order and type of calculations processed in the calculation processing unit 412, the coefficients of the calculations, etc.

[0056] The arithmetic processing unit 412 is configured to include multiple arithmetic circuits necessary for performing analysis processing on the captured image. Based on the information received from the arithmetic switching unit 411, the arithmetic processing unit 412 performs various arithmetic processes, transmits the results of the arithmetic processes to the input / output control unit 410, and stores them in the storage unit 404.

[0057] In this way, FPGA 402 extracts the configuration data for the processing function to be executed from the configuration data corresponding to multiple processing functions that are stored in advance, and rewrites the processing content to be executed by the arithmetic processing unit 412 based on the extracted configuration data. This allows the detachable device 100 to selectively execute at least one of its multiple processing functions. Furthermore, by adding configuration data for newly added processing as needed, the imaging device 110 can execute the latest processing. In the following, having multiple configuration data corresponding to each of the multiple processing functions will be expressed as "having multiple processing functions." That is, even if FPGA 402 of the detachable device 100 is configured to execute one processing function, if the processing content of the arithmetic processing unit 412 can be changed by the configuration data for other processing functions, it will be said to have multiple processing functions.

[0058] The memory interface 413 includes a circuit that converts the format of the received data into a protocol suitable for the memory unit 404 and sends and receives the data. The memory interface 413 supports several commands corresponding to the specifications of each memory and performs data transmission and reception (reading and writing data to the memory unit 404) between each component and the memory unit 404.

[0059] The storage unit 404 is a memory such as flash memory or DRAM, and may be non-volatile memory, volatile memory, or a combination of both. The storage unit 404 stores various types of information, such as data transmitted from the imaging device 110, information on the functions of the calculation processing, and the results of the calculation processing.

[0060] The temperature acquisition unit 414 includes a temperature sensor located outside the FPGA 402 that measures the temperature of the detachable device 100, and a generation unit that generates temperature information indicating the temperature measured by the temperature sensor. The temperature acquisition unit 414 outputs the temperature information generated by the generation unit to the input / output control unit 410.

[0061] The input / output control unit 410 generates transmission data that includes the result of the calculation process output from the calculation processing unit 412 and the temperature information output from the temperature acquisition unit 414, and transmits the generated transmission data to the imaging device 110.

[0062] Here, we will describe an example of a method for generating transmission data that includes the results of calculation processing and temperature information. First, the temperature information is approximately 1 byte of data. On the other hand, the data length of the calculation processing results is specified as 512 bytes per block, for example, in the case of the SD protocol, because data exchange must be done in blocks. Therefore, even if the calculation processing results are less than 512 bytes of data, it is necessary to send 512 bytes, which is the amount of data for one block. For example, if the calculation processing results are 500 bytes of data, the remaining 12 bytes of data are unnecessary (free), so this 12 bytes of free data is used to transmit approximately 1 byte of temperature information. In this way, communication for temperature information becomes unnecessary, and data can be efficiently transferred between devices. Furthermore, if the type of calculation processing is the same, the amount of data from the calculation processing results will be the same each time. Therefore, the imaging device 110 can determine which part of the transmission data for one block transmitted from the detachable device 100 is the result of calculation processing and which part is the temperature information, and can extract the temperature information from the transmission data.

[0063] Next, an example of the functional configuration of the detachable device 100 will be explained using the block diagram in Figure 5. The functional unit shown in Figure 5 may be implemented as hardware incorporated into any of the functional units shown in Figure 4, or it may be implemented as software executed by any of the functional units shown in Figure 4.

[0064] The analysis unit 501 performs various calculation processes, including analysis processing, on the captured image received from the imaging device 110. For example, when the analysis unit 501 receives a request to set up analysis processing, it performs settings to make the analysis processing executable. Also, when the analysis unit 501 receives a command for an captured image and calculation processing, it performs calculation processing, including analysis processing, on the captured image that has been set to an executable state. In this embodiment, the executable analysis processing is assumed to be human body detection processing and face detection processing, but is not limited to these. For example, it may be a process that determines whether or not a person previously stored is included in the captured image (face recognition processing described later). Alternatively, for example, it may be a process that calculates the degree of agreement between the image features of a person previously stored and the image features of a person detected from the input captured image, and determines that the person is a person previously stored if the degree of agreement is above a threshold. Alternatively, for example, for the purpose of protecting privacy, it may be a process that superimposes a predetermined mask image or applies mosaic processing to a person detected from the input captured image. Alternatively, for example, it may be a process that uses a learning model that has learned specific actions of people by machine learning to detect whether or not a person in the captured image is performing a specific action. Furthermore, the process may involve determining what kind of region a given area in the captured image is. Alternatively, it may involve using a machine learning model trained on buildings, roads, people, the sky, etc., to determine what kind of region a given area in the captured image is.

[0065] As described above, the executable analysis processes can be applied to both machine learning-based and non-machine learning-based analysis processes. Furthermore, each of the above analysis processes is not limited to being performed by the detachable device 100 alone, but may also be performed in cooperation with the imaging device 110. The communication unit 502 controls data communication with the imaging device 110 via the I / F unit 401.

[0066] (Example configuration of input / output device 130) Next, an example of the hardware configuration of the input / output device 130 will be explained using the block diagram in Figure 6. Note that Figure 6 shows the main configurations relevant to the following explanation and does not show all the configurations of the input / output device 130.

[0067] The processor 601 is a processor such as a CPU, MPU, or GPU, and it performs various processes using computer programs and data stored in RAM 602 and ROM 603. In this way, the processor 601 controls the operation of the entire input / output device 130, and also executes or controls the various processes that the input / output device 130 performs, as described below.

[0068] RAM 602 has areas for storing computer programs and data loaded from ROM 603 and HDD (hard disk drive) 604, and areas for storing data received from imaging device 110 via communication I / F 605. RAM 602 also has a work area used by processor 601 when performing various processes. In this way, RAM 602 can provide various areas as appropriate.

[0069] ROM 603 stores configuration data for the input / output device 130, computer programs and data related to the startup of the input / output device 130, computer programs and data related to the basic operation of the input / output device 130, and so on.

[0070] The HDD 604 stores computer programs and data that cause the processor 601 to perform various processes, which are described as being performed by the OS (operating system) and the input / output device 130. The computer programs and data stored in the HDD 604 are loaded into the RAM 602 as appropriate according to the control of the processor 601 and become the target of processing by the processor 601.

[0071] The communication interface 605 is a communication interface for connecting the input / output device 130 to the network 120, and the input / output device 130 performs data communication with the imaging device 110 on the network 120 via the communication interface 605.

[0072] The operation unit 606 is a user interface such as a keyboard, mouse, or touch panel, and allows the user to input various instructions to the processor 601 through operation.

[0073] The display unit 607 has a liquid crystal screen or a touch panel screen and displays the processing results from the processor 601 as images, text, etc. Alternatively, the display unit 607 may be a projection device such as a projector that projects images or text.

[0074] The processor 601, RAM 602, ROM 603, HDD 604, communication I / F 605, operation unit 606, and display unit 607 are all connected to the system bus 608. In the configuration shown in Figure 6, information is presented to the user by display on the display unit 607, but information may also be presented by sound or vibration in addition to or instead of this, and the method of presenting information to the user is not limited to a specific method.

[0075] Furthermore, instructions and information input to the input / output device 130 may be via voice, or via gesture input (recognizing the user's gestures and inputting instructions or information corresponding to the recognized gestures). In other words, the method of inputting information to the input / output device 130 is not limited to a specific input method.

[0076] Next, an example of the functional configuration of the input / output device 130 will be explained using the block diagram in Figure 7. The functional units shown in Figure 7 may be implemented as hardware or as software. In the latter case, this software is stored in the HDD 604, loaded into the RAM 602 as needed, and executed by the processor 601.

[0077] The communication unit 701 controls the communication interface 605 to perform data communication with the imaging device 110 via the network 120. This is just one example; for instance, the communication unit 701 may be configured to establish a direct connection with the imaging device 110 and communicate with the imaging device 110 without going through the network 120 or other devices. The control unit 702 controls the operation of the input / output device 130.

[0078] <Processing flow in the system> Next, an overview of the system operation in this embodiment will be described according to the flowchart in Figure 8. The process according to the flowchart in Figure 8 is initiated when the detachable device 100 is detected to be attached to the imaging device 110 through communication between the imaging device 110 and the detachable device 100 (the method of detection is not limited to this).

[0079] First, in step S801, the user attaches the detachable device 100 to the imaging device 110. In step S802, the arithmetic processing unit 203 executes an initialization sequence for the detachable device 100. In this initialization sequence, predetermined commands are sent and received between the imaging device 110 and the detachable device 100, so that the imaging device 110 becomes ready to use the detachable device 100.

[0080] In step S803, the arithmetic processing unit 203 determines from the above list stored in the memory unit 303 which processes the detachable device 100 can execute, and determines which processes can be executed by the imaging device 110 alone or by the combination of the imaging device 110 and the detachable device 100. The detachable device 100 can be configured to execute any process, but processes unrelated to the processes to be executed on the imaging device 110 side do not need to be considered. For example, the imaging device 110 may keep a "list of executable processes" that it has previously obtained from, for example, the input / output device 130. In this case, when the imaging device 110 obtains information from the detachable device 100 indicating which processes the detachable device 100 can execute, it can determine only the executable processes by checking whether or not those processes are included in the list.

[0081] Next, in step S804, the arithmetic processing unit 203 determines the processing to be executed and, if necessary, configures the detachable device 100. That is, if at least a part of the processing to be executed is performed by the detachable device 100, the configuration of the detachable device 100 for that processing is performed. In this configuration, for example, the FPGA 402 may be reconfigured using configuration data corresponding to the processing to be executed. Next, in step S805, the arithmetic processing unit 203 and / or the arithmetic processing unit 412 execute analysis processing (pre-analysis processing, analysis processing, post-analysis processing) and arithmetic processing.

[0082] The process shown in Figure 8 is performed, for example, when the detachable device 100 is attached to the imaging device 110. However, at least a part of the process shown in Figure 8 may be repeatedly performed, for example, by performing the process in step S803 again when the detachable device 100 is removed from the imaging device 110.

[0083] Next, the part of the process shown in Figure 8 that controls the execution frequency of the process in the detachable device 100 according to the temperature of the detachable device 100 will be explained using the flowchart in Figure 9. The process according to the flowchart in Figure 9 starts after power is turned on to the imaging device 110, data communication between the imaging device 110 and the detachable device 100 becomes possible, and the detachable device 100 becomes capable of performing calculations.

[0084] In step S901, the control unit 304 controls the device communication unit 306 to request the detachable device 100 to transmit temperature limit information indicating the temperature rise in the detachable device 100. Upon receiving this request, the input / output control unit 410 of the detachable device 100 retrieves the "temperature limit information indicating the temperature rise in the detachable device 100" stored in the storage unit 404 and transmits the acquired temperature limit information to the imaging device 110 via the I / F unit 401. The control unit 304 retrieves the temperature limit information transmitted from the detachable device 100 via the device communication unit 306.

[0085] In step S902, the control unit 304 reads the "temperature limit information indicating the temperature limit in the imaging device 110" stored in the memory unit 303. The temperature processing unit 308 then compares the magnitude of "temperature limit B in the detachable device 100" indicated by the temperature limit information acquired by the control unit 304 from the detachable device 100, and "temperature limit A in the imaging device 110" indicated by the temperature limit information read by the control unit 304 from the memory unit 303. The temperature processing unit 308 then sets the lower of "temperature limit A" and "temperature limit B" as the threshold.

[0086] In step S903, the control unit 304 transmits the captured image and instructions for calculation processing on the captured image to the detachable device 100 via the device communication unit 306. The calculation processing unit 412 in the detachable device 100 performs calculation processing on the captured image based on the captured image and calculation processing instructions obtained via the I / F unit 401 and the input / output control unit 410.

[0087] In step S904, the temperature acquisition unit 414 generates temperature information indicating the measured "temperature at the detachable device 100". The input / output control unit 410 generates transmission data including the generated temperature information and the result of the calculation processing in step S903. In step S905, the input / output control unit 410 transmits the transmission data generated in step S904 to the imaging device 110 via the I / F unit 401.

[0088] In step S906, the control unit 304 receives the transmission data sent from the detachable device 100 via the device communication unit 306, and the temperature processing unit 308 extracts temperature information from the received transmission data. The analysis unit 305 extracts the results of calculation processing from the received transmission data and performs analysis processing on the captured image using the extracted calculation results.

[0089] In step S907, the temperature processing unit 308 determines whether the temperature indicated by the temperature information extracted from the transmitted data in step S906 is equal to or greater than the threshold set in step S902. If, as a result of this determination, the temperature indicated by the temperature information extracted from the transmitted data in step S906 is equal to or greater than the threshold set in step S902, the process proceeds to step S908. On the other hand, if the temperature indicated by the temperature information extracted from the transmitted data in step S906 is less than the threshold set in step S902, the process proceeds to step S910.

[0090] In step S908, the temperature processing unit 308 controls the frequency of processing (arithmetic processing) in the detachable device 100 in order to lower the temperature in the imaging device 110 and the detachable device 100. For example, the temperature processing unit 308 controls the frequency of sending the captured image and arithmetic processing commands in step S903. In addition to or instead of this control, in step S908, the operating frequency used for arithmetic processing by the FPGA 402 in the detachable device 100 may also be controlled to be lowered. Then the process proceeds to step S903.

[0091] In step S910, the temperature processing unit 308 controls the frequency of processing (arithmetic processing) in the detachable device 100 in order to raise the temperature in the imaging device 110 and the detachable device 100. For example, the temperature processing unit 308 controls the frequency of sending the captured image and arithmetic processing commands in step S903. Alternatively, in step S910, the operating frequency used for arithmetic processing by the FPGA 402 in the detachable device 100 may be increased. Then the process proceeds to step S903.

[0092] Furthermore, if the temperature information extracted from the transmitted data in step S906 is less than the threshold set in step S902, the process in step S910 may be omitted, and the process may proceed to step S903.

[0093] This process allows for continuous monitoring of the temperature state of the detachable device 100, enabling the detachable device 100 to operate within a range where the temperature of the imaging device 110 and the detachable device 100 does not exceed a certain level.

[0094] <Example 1> The entities performing each of the above processes are merely examples and are not limited to them. For example, if the imaging device 110 or the detachable device 100 is equipped with dedicated hardware, some or all of the above processes may be performed by that hardware.

[0095] <Modification 2> As a control method for lowering the temperature in the imaging device 110, the following configuration may be adopted. For example, the imaging device 110 may be provided with a fan to lower the temperature of the hardware inside the imaging device 110, and the temperature processing unit 308 may stop the fan in step S910 and start the fan (rotate the fan) in step S908. Alternatively, the temperature processing unit 308 may rotate the fan at a rotational speed P (P≧0) in step S910 and rotate the fan at a rotational speed Q (Q>P) in step S908.

[0096] Furthermore, in step S908, the imaging device 110 may provide a warning to allow the user to adjust the operation of other functions of the imaging device 110. For example, the LEDs of the imaging device 110 may be blinked / lit, or a warning image or text may be displayed on the display screen of the imaging device 110.

[0097] Thus, according to this embodiment, the detachable device 100 and the imaging device 110 can be operated within a range where the temperature of the imaging device 110 and the detachable device 100 does not exceed a certain level.

[0098] [Second Embodiment] In this embodiment, the differences from the first embodiment will be explained, and unless otherwise specified below, it will be assumed to be the same as the first embodiment. In the first embodiment, the imaging device 110 controlled the operation of the detachable device 100 based on its temperature, but in this embodiment, such control is performed by the detachable device 100.

[0099] An example of the hardware configuration of the detachable device 100 according to this embodiment will be explained using the block diagram in Figure 10. In the first embodiment, the imaging device 110 has a temperature processing unit 308, but in this embodiment, the imaging device 110 does not have a temperature processing unit 308, and instead the detachable device 100 has a temperature processing unit 415.

[0100] When the detachable device 100 and the imaging device 110 are able to communicate, the temperature processing unit 415 obtains temperature limit information indicating "temperature limit A in the imaging device 110" from the imaging device 110 (storage unit 303). The temperature processing unit 415 then compares the "temperature limit B in the detachable device 100" indicated by the temperature limit information stored in the storage unit 404 with the "temperature limit A in the imaging device 110" indicated by the temperature limit information obtained from the imaging device 110. The temperature processing unit 415 then sets the lower of "temperature limit A" and "temperature limit B" as the threshold.

[0101] Furthermore, the temperature processing unit 415 determines whether the temperature indicated by the temperature information acquired by the temperature acquisition unit 414 is above a threshold. If, as a result of this determination, the temperature indicated by the temperature information acquired by the temperature acquisition unit 414 is above the threshold, the temperature processing unit 415 controls the system to lower the temperature of the imaging device 110 and the detachable device 100 by reducing the execution frequency of processing in the detachable device 100. For example, the temperature processing unit 415 lowers the frequency of the IP that performs calculation processing on the FPGA 402 to lower the operating frequency within the detachable device 100. On the other hand, if the temperature indicated by the temperature information acquired by the temperature acquisition unit 414 is below the threshold, the temperature processing unit 415 controls the system to raise the temperature of the imaging device 110 and the detachable device 100 by increasing the execution frequency of processing in the detachable device 100. Note that if the temperature indicated by the temperature information acquired by the temperature acquisition unit 414 is below the threshold, the temperature processing unit 415 may also control the system to maintain a constant execution frequency of processing in the detachable device 100.

[0102] Furthermore, the memory unit 404 may have a work memory for arithmetic processing such as DRAM, and the FPGA 402 may have a configuration that allows control of the communication frequency with the DRAM and the operating frequency of the DRAM itself. In this case, if the temperature indicated by the temperature information acquired by the temperature acquisition unit 414 is above a threshold, the load on arithmetic processing can be reduced by lowering the frequency of access to the DRAM or lowering the operating frequency of the DRAM. This makes it possible to operate the detachable device 100 within a range where the temperature of the detachable device 100 does not exceed a certain level.

[0103] Next, the part of the process shown in Figure 8 that controls the execution frequency of the process in the detachable device 100 according to the temperature of the detachable device 100 will be explained using the flowchart in Figure 10. The process according to the flowchart in Figure 10 starts after power is turned on to the imaging device 110, data communication between the imaging device 110 and the detachable device 100 becomes possible, and the detachable device 100 becomes capable of performing calculations.

[0104] In step S1101, the input / output control unit 410 requests the imaging device 110 to transmit temperature limit information indicating the temperature rise in the imaging device 110. Upon receiving this request, the control unit 304 in the imaging device 110 retrieves the "temperature limit information indicating the temperature rise in the imaging device 110" stored in the storage unit 303 and transmits the acquired temperature limit information to the detachable device 100 via the device communication unit 306. The input / output control unit 410 receives the temperature limit information transmitted from the imaging device 110 via the I / F unit 401.

[0105] In step S1102, the temperature processing unit 415 reads the "temperature limit information indicating the temperature limit for the detachable device 100" stored in the memory unit 404. The temperature processing unit 415 then compares the "temperature limit B for the detachable device 100" indicated by the temperature limit information read from the memory unit 404 with the "temperature limit A for the imaging device 110" indicated by the temperature limit information received from the imaging device 110. The temperature processing unit 415 then sets the lower of "temperature limit A" and "temperature limit B" as the threshold.

[0106] Step S1103 is the same as step S903 above, so no explanation is given. In step S1104, the temperature acquisition unit 414 generates temperature information indicating the measured "temperature at the detachable device 100".

[0107] In step S1105, the temperature processing unit 415 determines whether the temperature indicated by the temperature information generated in step S1104 is equal to or greater than the threshold set in step S1102. If, as a result of this determination, the temperature indicated by the temperature information generated in step S1104 is equal to or greater than the threshold set in step S1102, the process proceeds to step S1106. On the other hand, if the temperature indicated by the temperature information generated in step S1104 is less than the threshold set in step S1102, the process proceeds to step S1110.

[0108] In step S1106, the temperature processing unit 415 controls the frequency of processing (calculation processing) in the detachable device 100 in order to lower the temperature in the imaging device 110 and the detachable device 100. The control details are the same as in the first embodiment (same as in step S908 above), so the explanation is omitted.

[0109] In step S1110, the temperature processing unit 415 controls the frequency of processing (calculation processing) in the detachable device 100 in order to raise the temperature in the imaging device 110 and the detachable device 100. The control details are the same as in the first embodiment (same as in step S910 above), so the explanation is omitted.

[0110] In addition, as in the first embodiment, if the temperature indicated by the temperature information generated in step S1104 is less than the threshold set in step S1102, the process in step S1110 may be omitted, and the process may proceed to step S1103.

[0111] Thus, according to this embodiment, the temperature state of the detachable device 100 can be continuously monitored, and the detachable device 100 can be operated within a range where the temperature of the imaging device 110 and the detachable device 100 does not exceed a certain level.

[0112] In the above embodiments and modifications, we have used as examples systems that analyze the number and position of objects in captured images. However, the above embodiments and modifications can also be applied to systems for other purposes (such as systems that perform some kind of analysis on images captured by the imaging device 110).

[0113] The numerical values, processing timings, processing order, processing entity, data (information) structure / destination / source / storage location, etc., used in each of the above embodiments and modifications are given as examples for the purpose of providing a concrete explanation, and are not intended to limit the scope to such examples.

[0114] Furthermore, some or all of the embodiments and modified examples described above may be used in appropriate combinations. Alternatively, some or all of the embodiments and modified examples described above may be used selectively.

[0115] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0116] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of Symbols]

[0117] 301: Imaging control unit 302: Signal processing unit 303: Memory unit 304: Control unit 305: Analysis unit 306: Device communication unit 307: Network communication unit 308: Temperature processing unit

Claims

1. An imaging device, an acquisition means for acquiring temperature information indicating a temperature in a device detachably attached to the imaging device from the device; a control means for controlling the frequency of execution of a process executed by the device based on the temperature information; An imaging device comprising:

2. 2. The imaging device according to claim 1, wherein the control means performs control to reduce the execution frequency if the temperature indicated by the temperature information is equal to or higher than a threshold value.

3. 3. The imaging device according to claim 1, wherein the control unit performs control to increase the execution frequency if the temperature indicated by the temperature information is less than a threshold value.

4. 3. The imaging device according to claim 1, wherein the control means performs control to keep the execution frequency constant if the temperature indicated by the temperature information is below a threshold value.

5. 5. The imaging apparatus according to claim 1, wherein the control means controls the execution frequency by controlling an operating frequency of the device.

6. 5. The imaging apparatus according to claim 1, wherein the control means controls the execution frequency by controlling the frequency at which the command for the process is sent to the device.

7. 5. The imaging device according to claim 1, wherein the control means controls the rotation of a fan for lowering the temperature of hardware of the imaging device based on the temperature information.

8. the acquiring means acquires, from the device, first information indicating an upper temperature limit of the device; The imaging device further comprises: The imaging device according to any one of claims 2 to 4, further comprising a setting means for acquiring second information indicating an upper temperature limit in the imaging device, and setting the lower upper temperature limit between the upper temperature limit indicated by the first information and the upper temperature limit indicated by the second information as the threshold value.

9. 9. The imaging apparatus according to claim 1, wherein the acquisition unit acquires, from the device, temperature information indicating a temperature in the device measured by a sensor provided in the device.

10. the imaging device further includes a transmitting means for transmitting an instruction for arithmetic processing to the device; 10. The imaging device according to claim 1, wherein the acquisition unit acquires the temperature information and a result of a calculation performed by the device in response to the command from the device.

11. The device includes a computing unit that executes predetermined processing; 11. The imaging apparatus according to claim 1, wherein the control unit controls the execution of processing executed by the arithmetic unit included in the device.

12. A device detachable from an imaging device, the device comprising a computing device, an acquisition means for acquiring temperature information indicating a temperature in the device; a control means for controlling the frequency of execution of the processing executed by the arithmetic unit based on the temperature information; A device comprising:

13. A control method for an imaging device, comprising: an acquisition step in which an acquisition means of the imaging device acquires, from a device detachably attached to the imaging device, temperature information indicating a temperature at the device; a control step in which a control means of the imaging device controls an execution frequency of a process executed by the device based on the temperature information; 11. A method for controlling an imaging device, comprising:

14. A method for controlling a device detachable from an imaging device, the method comprising: an acquisition step in which acquisition means of the device acquires temperature information indicating a temperature in the device; a control step in which a control means of the device controls an execution frequency of a process executed by the arithmetic unit based on the temperature information; A device control method comprising:

Images