Electronic equipment, monitoring systems, and control methods

Abstract

Providing electronic devices, monitoring systems, and control methods that automatically set new preset information based on existing preset information. [Solution] The electronic device includes a processing unit that performs analysis processing based on an image captured by a first camera, a control unit that controls the imaging range of the first camera, and a communication unit that communicates with an external device. When a predetermined preset position is specified as the position to be imaged by the first camera, the communication unit acquires second preset information, which is a parameter for a second camera positioned differently from the first camera to image the preset position. Based on the second preset information, the processing unit determines first preset information, which is a parameter for the first camera to image the preset position, and the control unit controls the first camera based on the first preset information.

Description

Technical Field

[0001] The present invention relates to electronic devices, monitoring systems, control methods, and the like.

Background Art

[0002] Conventionally, a system for performing monitoring using a camera or a sensor has been known. For example, Patent Document 1 discloses a monitoring system that determines the moving direction of an intruder, selects an optimal camera, and pans and tilts to track the intruder. Patent Document 1 also discloses registering a plurality of presets in advance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The method of Patent Document 1 uses a monitoring camera that has been installed and set, and is for tracking an intruder.

[0005] According to some aspects of the present disclosure, it is possible to provide an electronic device, a monitoring system, a control method, and the like that automatically set new preset information based on existing preset information.

Means for Solving the Problems

[0006] One aspect of the present disclosure includes a processing unit that performs analysis processing based on an image captured by a first camera, a control unit that controls the imaging range of the first camera, and a communication unit that communicates with an external device, wherein when a predetermined preset position is specified as the position to be imaged by the first camera, the communication unit acquires second preset information which is parameters for a second camera, which is positioned differently from the first camera, to image the preset position, the processing unit determines first preset information which is parameters for the first camera to image the preset position based on the second preset information, and the control unit relates to electronic equipment that controls the first camera based on the first preset information.

[0007] Other aspects of the present disclosure include a first electronic device having a first camera and a second electronic device having a second camera positioned differently from the first camera, wherein the first electronic device relates to a monitoring system that, when a preset position is specified as a position to be imaged by the first camera, obtains second preset information from the second electronic device, which is parameters for the second camera to image the preset position, and performs a process of determining first preset information, which is parameters for the first camera to image the preset position, based on the second preset information.

[0008] Further aspects of the present disclosure relate to a control method for electronic equipment having a first camera, wherein when a preset position is designated as a position to be imaged by the first camera, the electronic equipment acquires second preset information which is parameters for a second camera positioned at a different location from the first camera to image the preset position, and performs a process to determine first preset information which is parameters for the first camera to image the preset position based on the second preset information. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example configuration of the monitoring system according to this embodiment. [Figure 2]This figure shows an example of the configuration of the electronic device according to this embodiment. [Figure 3] This figure shows an example of a screen displayed on the display unit of a management terminal device. [Figure 4A] This is a flowchart explaining the setup process, including the manual creation of preset information. [Figure 4B] This is a flowchart explaining the process of manually creating preset information. [Figure 5] This figure shows a concrete example of preset information. [Figure 6] This is a flowchart explaining the setup process, including the automatic creation of preset information. [Figure 7] This diagram illustrates the relationship between multiple cameras and their preset positions. [Figure 8] This is a flowchart explaining the process of automatically creating preset information. [Figure 9] This diagram illustrates the relationship between multiple cameras and their preset positions. [Figure 10B] This diagram illustrates the relationship between the second camera and multiple preset positions. [Figure 10A] This diagram illustrates the relationship between the first camera and multiple preset positions. [Figure 11] This is a flowchart explaining the object extraction process. [Figure 12A] This figure shows an example of a second image captured by the second camera. [Figure 12B] This figure shows an example of the first image captured by the first camera. [Figure 12C] This figure shows an example of the first image captured by the first camera. [Figure 12D] This figure shows an example of the first image captured by the first camera. [Figure 13A] This figure illustrates an example of an object evaluation value based on the second captured image. [Figure 13B] This figure illustrates an example of an object evaluation value based on the first captured image. [Figure 13C] This figure shows the results of the pattern matching process. [Figure 13D] It is a diagram for explaining an example of an evaluation value of an object based on a first captured image. [Figure 14] It is a flowchart for explaining a correction process of preset information. [Figure 15] It is a flowchart for explaining patrol control. [Figure 16A] It is a diagram for explaining a patrol route. [Figure 16B] It is a diagram for explaining a patrol schedule. [Figure 17] It is a diagram for explaining an example in which the shape of an object changes over time. [Figure 18] It is a diagram for explaining an example of a temporal change in an evaluation value of an object. [Figure 19] It is an explanatory diagram of a process for discovering a new object in automatic patrol control.

Mode for Carrying Out the Invention

[0010] Hereinafter, this embodiment will be described with reference to the drawings. For the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant explanations are omitted. Note that the embodiments described below do not unduly limit the content described in the claims. Also, not all of the configurations described in this embodiment are essential constituent elements of the present disclosure.

[0011] 1. Example of System Configuration Figure 1 is a diagram illustrating an example configuration of a monitoring system 10 including an electronic device 100 (monitoring device) according to this embodiment. The monitoring system 10 according to this embodiment includes an electronic device 100, a management terminal device 200, and storage 300. However, the configuration of the monitoring system 10 is not limited to the example shown in Figure 1, and various modifications can be made, such as omitting some components or adding other components. For example, in Figure 1, two electronic devices 100A and 100B are shown as examples of electronic devices 100, but the number of electronic devices 100 is not limited to two, and may be three or more. Furthermore, the method of this embodiment can also be applied when the electronic device 100 is moved, in which case the number of electronic devices 100 may be one. Below, examples where there are multiple electronic devices 100 will be mainly described. Also, when there is no need to distinguish between multiple electronic devices, they will simply be referred to as electronic device 100.

[0012] The electronic device 100, the management terminal device 200, and the storage device 300 are connected via a network NE. Here, the network NE is a public communication network such as the Internet, but it may also be a network such as a LAN (Local Area Network).

[0013] Electronic device 100 is a device that acquires images captured by camera 150. Multiple electronic devices 100 are placed in the location to be monitored. The location to be monitored can be any location, such as a commercial facility, parking lot, or park. The object of monitoring may be a person, a vehicle, or equipment or structures placed in the facility.

[0014] Multiple electronic devices 100 are, for example, located in different positions within the same facility, and each performs the process of monitoring the target within the facility from different positions and angles. For example, multiple locations to be monitored are pre-configured for each electronic device 100, and the electronic device 100 monitors these locations according to a predetermined schedule. Hereinafter, the locations pre-configured as monitoring targets will be referred to as preset locations.

[0015] Figure 2 shows an example configuration of the electronic device 100. The electronic device 100 includes a control unit 110, a processing unit 120, a communication unit 130, and a storage unit 140. The electronic device 100 is also connected to an external camera 150 and a sensor 160, as shown in Figure 2. However, it is not necessary for the camera 150 and sensor 160 to be external components of the electronic device 100; the electronic device 100 may include one or both of the camera 150 and sensor 160. Furthermore, the configuration of the electronic device 100 is not limited to the example shown in Figure 2, and various modifications can be made, such as omitting some components or adding other components. For example, in this embodiment, the sensor 160 may be omitted.

[0016] In the following, when distinguishing electronic device 100A from other electronic devices 100, the parts of electronic device 100A will be referred to as the control unit 110A, processing unit 120A, communication unit 130A, and storage unit 140A. Similarly, the camera 150 and sensor 160 corresponding to electronic device 100A will be referred to as camera 150A and sensor 160A. When distinguishing electronic device 100B from other electronic devices 100, the designation "B" will be added to each component for distinction.

[0017] The control unit 110 is connected to and controls the processing unit 120, communication unit 130, storage unit 140, camera 150, and sensor 160. The control unit 110 in this embodiment is composed of the following hardware. The hardware may include at least one of a circuit that processes digital signals and a circuit that processes analog signals. For example, the hardware may consist of one or more circuit devices or one or more circuit elements mounted on a circuit board. One or more circuit devices may be, for example, an IC (Integrated Circuit) or an FPGA (field-programmable gate array). One or more circuit elements may be, for example, a resistor or a capacitor.

[0018] The control unit 110 may also be implemented by the following processor. The electronic device 100 of this embodiment includes a memory for storing information and a processor that operates based on the information stored in the memory. The information is, for example, a program and various types of data. The memory may be a storage unit 140 or another type of memory. The processor includes hardware. Various types of processors can be used, such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a DSP (Digital Signal Processor). The memory may be a semiconductor memory such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), or flash memory, or a register, or a magnetic storage device such as a hard disk drive (HDD), or an optical storage device such as an optical disk drive. For example, the memory stores instructions that can be read by a computer, and the functions of the control unit 110 are realized as processing when the processor executes these instructions. The instructions here may be instructions from an instruction set that constitutes a program, or instructions that instruct the hardware circuit of the processor to operate.

[0019] The processing unit 120 acquires the captured image taken by the camera 150 via the control unit 110 and performs processing based on the captured image. For example, the processing unit 120 may perform processing to estimate the distance to the subject based on the captured image, processing to estimate the relative positional relationship between the two electronic devices 100, and processing to create preset information based on the relative positional relationship. The processing unit 120 may also perform processing to detect objects from the captured image and processing to correct the preset information.

[0020] The processing unit 120 in this embodiment may be implemented by a processor including hardware, similar to the control unit 110. Various types of processors can be used, such as a CPU, GPU, or DSP. Furthermore, the control unit 110 and the processing unit 120 may be implemented by the same processor, or they may be implemented by different processors.

[0021] The communication unit 130 communicates with other electronic devices 100, management terminal devices 200, and storage devices 300 via the network NE.

[0022] The communication unit 130 is an interface for communication and, when the electronic device 100 performs wireless communication, includes, for example, an antenna, an RF (radio frequency) circuit, and a baseband circuit. The communication unit 130 operates according to the control of the control unit 110. The communication method performed by the communication unit 130 may be a method compliant with IEEE 802.11, a method of short-range wireless communication such as Bluetooth®, or another method.

[0023] The storage unit 140 is the work area of ​​the processing unit 120 and stores various information. The storage unit 140 may store, for example, preset information, which is setting information for imaging the preset positions described above using the camera 150. Details of the preset information will be described later with reference to Figure 5, etc. The storage unit 140 can be implemented with various types of memory, and the memory may be semiconductor memory such as SRAM, DRAM, ROM (Read Only Memory), or flash memory, or it may be a register, a magnetic storage device, or an optical storage device.

[0024] Camera 150 is a device that acquires captured images using an image sensor. The image sensor may be a CMOS (Complementary Metal Oxide Semiconductor) image sensor, a CCD (Charge-Coupled Device) image sensor, or another type of sensor. The camera 150 according to this embodiment is a PTZ camera capable of, for example, horizontal panning, vertical tilting, and zoom magnification changes (zoom operation).

[0025] Sensor 160 is, for example, a sensor that detects the presence or movement of an object within a predetermined range. For example, sensor 160 is an infrared sensor that emits and receives infrared light. Alternatively, sensor 160 may be a sensor that detects objects using ultrasound or visible light. Furthermore, sensor 160 according to this embodiment may be a sensor that detects temperature, humidity, illuminance, odor, etc. Also, if electrically driven equipment is installed in the facility to be monitored, sensor 160 may be a sensor that detects current values ​​or voltage values.

[0026] The electronic device 100 according to this embodiment monitors multiple preset positions using a camera 150. These preset positions are, for example, several dozen locations, specifically 32 locations, but the number of preset positions is not limited to these. If the camera 150 is the aforementioned PTZ camera, the preset information represents the control amounts for pan, tilt, and zoom operations corresponding to each of the 32 preset positions. The control unit 110 periodically monitors the preset positions based on the preset information, thereby enabling periodic monitoring of positions considered important. In this embodiment, such periodic monitoring is also referred to as patrolling. Patrolling will be described later using Figures 16A-16B, etc.

[0027] The management terminal device 200 is a device that collects information from multiple electronic devices 100. The management terminal device 200 is, for example, a PC (Personal Computer), but may be implemented by other devices. The management terminal device 200 controls the multiple electronic devices 100 based on the information from the multiple electronic devices 100. The management terminal device 200 is also a device used by a user performing monitoring tasks, and may perform processing to present the user with images acquired by the electronic devices 100.

[0028] Figure 3 shows an example of a monitoring screen displayed on the display unit of the management terminal device 200. Note that the management terminal device 200 is a server system, and this server system may perform the process of displaying the monitoring screen shown in Figure 3 on the display unit of the client device.

[0029] As shown in Figure 3, the monitoring screen may include areas RE1-RE6. Area RE1 is the area for displaying the captured image. This allows the captured image acquired by the electronic device 100 to be presented to the user of the management terminal device 200. Figure 3 shows an example where captured images are displayed when monitoring the area around playground equipment in a park.

[0030] Area RE2 includes a pull-down menu for setting the video compression standard and image size when storing captured images in storage 300.

[0031] Area RE3 includes buttons for various operations such as extracting still images, turning recording on / off, and turning the microphone and speaker on / off.

[0032] Region RE4 includes an operating interface for controlling the attitude of camera 150. As described above, camera 150 may be a PTZ camera capable of switching between pan, tilt, and zoom. In this case, region RE4 displays objects for changing the attitude of camera 150 in each direction (eight directions in the example of Figure 3), and a pull-down menu for setting the attitude change speed of camera 150 when an object is selected.

[0033] The watershed RE5 includes an object for changing the zoom magnification and a pull-down menu for setting the zoom magnification change speed of the camera 150 when the object is selected. The region RE5 may also include objects for setting imaging parameters of the camera 150, such as focus speed and autofocus.

[0034] Area RE6 contains link information for performing detailed settings in the monitoring system 10. In the example in Figure 3, text such as "PTZ settings" is displayed in area RE6, and when this text is selected, the management terminal device 200 transitions to a settings screen for performing the corresponding settings. For example, if "Preset settings" in Figure 3 is selected, the management terminal device 200 may transition to a screen for performing the preset information settings (addition, modification) described above.

[0035] The storage 300 stores various types of data. The storage 300 may be, for example, a storage server, and can be implemented using various hardware such as HDDs and SSDs (Solid State Drives). The storage 300 stores, for example, images acquired by multiple electronic devices 100 in the form of still images or moving images. The storage may also store information related to the settings of the electronic devices 100, such as the preset information of the camera 150.

[0036] As described above, the electronic device 100 according to this embodiment includes a processing unit 120 that performs analysis processing based on an image captured by a first camera (camera 150), a control unit 110 that controls the imaging range of the first camera, and a communication unit 130 that communicates with an external device. When a predetermined preset position is specified as the position to be imaged by the first camera, the communication unit 130 of the electronic device 100 acquires second preset information, which is a parameter for a second camera positioned differently from the first camera to image the preset position.

[0037] For example, suppose that the preset information for camera 150B of electronic device 100B has already been set, and then camera 150A is newly installed. In this case, electronic device 100A, which corresponds to camera 150A, acquires the preset information for camera 150B as second preset information via the communication unit 130A. Note that electronic device 100A may acquire the second preset information directly from electronic device 100B, or it may acquire the second preset information via the management terminal device 200 or storage 300.

[0038] The processing unit 120 then determines first preset information, which is the parameters for the first camera to image the preset position, based on the second preset information. After the first preset information is determined, the control unit 110 controls the first camera based on the first preset information. In the example above, the electronic device 100A creates preset information for camera 150A based on the preset information for camera 150B, and controls camera 150A using the preset information for camera 150A.

[0039] The monitoring system 10 is required to detect abnormalities in the target facility as comprehensively as possible. Therefore, as mentioned above, it is expected that there will be a considerable number of preset locations, such as several dozen. In this case, when adding a new camera 150A or moving an existing camera 150A to a different location, it is necessary to set the preset information for camera 150A. If the preset information is to be set manually, it is necessary to input sets of control amounts for pan, tilt, and zoom operations for each preset location, which places a heavy burden on the user. For example, when manually creating preset information, the user, who is the administrator of the monitoring system 10, would have to repeatedly manipulate objects in area RE4 while viewing the captured image in area RE1 of the monitoring screen shown in Figure 3, so that the desired object is captured at the appropriate size and position on the image, for a number of times corresponding to the preset location. In this respect, the method of this embodiment can automatically generate new preset information using existing preset information (preset information for camera 150B) if such information exists, thereby reducing the user burden in setting up the monitoring system 10.

[0040] Furthermore, the method of this embodiment can be applied to a monitoring system 10 including an electronic device 100. The monitoring system 10 includes a first electronic device having a first camera and a second electronic device having a second camera positioned differently from the first camera. In the example described above, the first electronic device is the electronic device 100A corresponding to camera 150A. The second electronic device is the electronic device 100B corresponding to camera 150B.

[0041] When a preset position is specified as the position to be imaged by the first camera, the first electronic device obtains second preset information, which is the parameters for the second camera to image the preset position, from the second electronic device, and performs a process to determine the first preset information, which is the parameters for the first camera to image the preset position, based on the second preset information.

[0042] Furthermore, some or all of the processing performed by the electronic device 100 in this embodiment may be implemented by a program. In a narrow sense, the processing performed by the electronic device 100 refers to the processing performed by at least one of the control unit 110 and the processing unit 120, but it may also include processing performed by other components. For example, the program may be a program that implements the processing performed by the processor of the management terminal device 200.

[0043] The program according to this embodiment can be stored in a non-temporary information storage medium (information storage device), which is a medium readable by a computer. The information storage medium can be implemented as, for example, an optical disc, a memory card, an HDD, or a semiconductor memory. The semiconductor memory is, for example, ROM. The control unit 110 and the processing unit 120, etc., perform various processing according to this embodiment based on the program stored in the information storage medium. That is, the information storage medium stores a program that causes the computer to function as the control unit 110, etc. The computer is a device that includes an input device, a processing unit, a storage unit, and an output unit. Specifically, the program according to this embodiment is a program that causes the computer to execute each step described later using Figures 4A, 4B, 6, 8, 11, 14, 15, etc. Furthermore, the method of this embodiment may be applied to a program that realizes the processing performed by the monitoring system 10.

[0044] Furthermore, the method of this embodiment can be applied to a control method for electronic equipment having a first camera. The control method includes, when a preset position is specified as the position to be imaged by the first camera, acquiring second preset information which is parameters for a second camera positioned at a different location from the first camera to image the preset position, and performing a process to determine first preset information which is parameters for the first camera to image the preset position based on the second preset information.

[0045] 2. Processing Flow Next, the processing flow according to this embodiment will be described in detail. Below, the manual preset information setting process, the automatic preset information creation process using existing preset information, and the preset information correction process will be described. In relation to the automatic creation process, several specific examples of the process for determining the relative position between two electronic devices 100 will be described.

[0046] The following describes an example where preset information for camera 150B is manually created, and preset information for camera 150A is automatically created based on that preset information. However, for automatic creation of preset information, it is sufficient that existing preset information exists, regardless of whether that existing preset information was created manually or automatically. For example, if preset information for camera 150A is automatically created based on manually created preset information for camera 150B, then when automatically creating preset information for a different third camera, either the preset information for camera 150B or the preset information for camera 150A may be used.

[0047] 2.1 Electronic device configuration process, including manual creation of preset information Figure 4A is a flowchart illustrating the setup process for electronic device 100, including the manual creation of preset information. For example, the camera 150 that is the target of the manual creation of preset information is camera 150B, which corresponds to electronic device 100B. In the drawing, camera 150B is also simply referred to as camera B. Camera 150B (camera B) corresponds to the second camera mentioned above.

[0048] In step S101, the control unit 110B sets the initial angle of the camera 150B. The initial angle here represents the reference orientation of the camera 150B and is the reference angle for the control amount (rotation amount) in the pan and tilt operations. For example, from the viewpoint of unifying the reference orientation for multiple cameras 150, the control unit 110B may control the camera 150B to image a predetermined subject. The subject here may be a subject whose direction relative to the installation position of the camera 150B is known and which can be assumed to exist at infinity. For example, the subject may be Polaris. For example, the control unit 110B accepts a user operation to image Polaris near the center of the captured image and performs the pan and tilt operations of the camera 150B based on that user operation. The user operation is performed, for example, using the operation interface of the management terminal device 200 (see area RE4 in Figure 3). The control unit 110B corrects the drive amount of the pan operation and the drive amount of the tilt operation in that state to 0. Hereinafter, the drive amount of the pan operation will also be referred to as the horizontal angle θ. The amount of tilt movement is also expressed as the vertical angle φ. In other words, the state where θ=φ=0 is set as the reference posture, and the posture of camera 150B is expressed as the difference between the horizontal angle and the vertical angle relative to that reference posture.

[0049] By using a camera orientation that images a subject that can be assumed to be at infinity as the reference orientation, it becomes easier to unify the reference orientation of multiple cameras 150 when multiple cameras 150 are used. For example, by setting the reference orientation for each of the multiple cameras 150 based on Polaris, the reference orientation of the multiple cameras 150 becomes common. However, the reference orientation may differ depending on the camera 150, in which case the reference orientation can be set arbitrarily.

[0050] In step S102, the processing unit 120B initializes the variable N to 1. Here, N is a variable representing the number of locations to be monitored. In other words, the variable N represents the number of locations for which preset information will be created.

[0051] In step S103, the processing unit 120B performs a manual creation process to create preset information for the Nth preset position corresponding to the variable N, based on user operation.

[0052] Figure 4B is a flowchart illustrating the manual creation process of preset information in step S103. In step S201, the control unit 110B selects the target to be monitored. Specifically, the control unit 110B receives a user operation to capture the target to be monitored in the appropriate position and size in the captured image, and performs pan, tilt, and zoom operations of the camera 150B based on the user operation. The user operation is performed, for example, using the operation interface of the management terminal device 200. For example, the user checks the image captured by the camera 150B in area RE1 of the monitoring screen and performs user operations to adjust the position and size of the target to be monitored in the image by performing pan, tilt, and zoom operations using the operation interface in area RE4. Then, when the target to be monitored is captured in the appropriate state, the user performs a user operation to instruct the preset setting. The processing unit 120B acquires the user operation via the communication unit 130B.

[0053] In step S202, the processing unit 120B sets the preset information based on the user operation described above. Specifically, it adds the horizontal angle θ, vertical angle φ, and zoom magnification M in the state in which the user instructed the preset setting to the preset information of the preset position N. This makes it possible to image the target being monitored in an appropriate state using the camera 150B.

[0054] The processing unit 120B may also add information other than the horizontal angle θ, vertical angle φ, and zoom magnification M to the preset information. In step S203, the processing unit 120B measures the distance R from the camera 150B to the subject to be monitored. For example, the processing unit 120B may measure the distance to the subject based on the captured image. The processing unit 120B may also perform stereo matching using a stereo camera. Furthermore, a method of measuring distance using a monocular camera by utilizing AI (Artificial Intelligence) is known, and the processing unit 120B may perform processing according to this method. Alternatively, a method of measuring distance using autofocus technology is known, and the processing unit 120B may perform processing according to this method. Furthermore, if the camera 150B has a light-emitting element that emits light in addition to a photodetector, the processing unit 120B may measure the distance using ToF (Time of Flight). In ToF, the distance is calculated by measuring the time it takes for the emitted light to reflect off the subject and return. Furthermore, the electronic device 100 may include a ToF sensor different from the camera 150B.

[0055] In step S204, the processing unit 120B calculates the preset coordinates. The preset coordinates here represent the three-dimensional coordinates of the preset position in a Cartesian coordinate system with the camera 150B as the origin. The horizontal angle θ, vertical angle φ, and distance R to the subject mentioned above represent coordinate values ​​in a polar coordinate system with the camera 150B as the origin. In other words, in step S204, the processing unit 120B performs a process to convert the coordinate values ​​in the polar coordinate system to coordinate values ​​in the Cartesian coordinate system. Since the coordinate transformation from polar coordinate system to Cartesian coordinate system is publicly known, a detailed explanation is omitted.

[0056] In step S205, the processing unit 120B stores the horizontal angle θ, vertical angle φ, zoom magnification M, distance R, and Cartesian coordinate system coordinate values ​​(X, Y, Z) obtained above in the storage unit 140B as preset information corresponding to the Nth preset position.

[0057] Returning to Figure 4A, let's continue the explanation. Once the preset information corresponding to the Nth preset position is created by the process shown in Figure 4B, in step S104, the processing unit 120B performs an object extraction process from the captured image. For example, the processing unit 120B performs an object extraction process from the captured image of the preset position to determine the type of object and its position in the image. In this way, it becomes possible to determine what kind of object is present at the Nth preset position and in what state. The results of the object extraction process are used, for example, in the preset information correction process described later. Therefore, the details of the object extraction process will be described later together with the preset information correction process. The results of the object extraction process may also be used for patrolling, which will be described later.

[0058] In step S105, the processing unit 120B determines whether the creation of preset information has been completed for all monitored locations. If there are still monitored locations for which preset information has not been created (step S105: No), in step S106, the processing unit 120B increments the value of N and then returns to the process in step S103. In other words, the manual creation of preset information (step S103) and the object extraction process (step S104) described above are executed for the new preset locations.

[0059] Once the creation of preset information for all monitored targets is complete (Step S105: Yes), the process shown in Figure 4A is completed.

[0060] Figure 5 shows an example of preset information created by performing the processing shown in Figures 4A and 4B. For example, the preset information may include the coordinates of camera 150B. The coordinates of camera 150B are coordinate values ​​that represent the position of camera 150B in a Cartesian coordinate system based on camera 150B. In the example shown in Figure 5, since the position of camera 150B is the origin, the coordinates of camera 150B are (X,Y,Z)=(0,0,0).

[0061] In Figure 5, Preset 1, Preset 2, and Preset 3 correspond to the first, second, and third preset positions, respectively. As described above, each preset position is associated with a horizontal angle θ corresponding to the pan drive amount, a vertical angle φ corresponding to the tilt drive amount, Cartesian coordinate values ​​(X, Y, Z), distance R to the preset position, and zoom magnification M. If there are 32 preset positions, the preset information contains 32 sets of these values.

[0062] 2.2 Configuration process for electronic devices, including automatic creation of preset information Figure 6 is a flowchart illustrating the setup process for electronic device 100, including the automatic creation of preset information. For example, the camera 150 targeted by the automatic creation of preset information is camera 150A, which corresponds to electronic device 100A. In the drawing, camera 150A is also simply referred to as camera A. Camera 150A (camera A) corresponds to the first camera described above.

[0063] In step S301, camera 150A is newly installed. The camera 150A installed here may be a newly added camera or an existing camera that has been moved to another location.

[0064] In step S302, the control unit 110A and / or processing unit 120A of the electronic device 100A corresponding to the newly installed camera 150A acquire the relative positions of the newly installed camera 150A and the existing camera 150B. The specific method for determining the relative positions will be described later with reference to Figures 7-10.

[0065] In step S303, the processing unit 120A initializes the variable N to 1. Here, N is a variable that represents the number of locations to be monitored.

[0066] In step S304, the processing unit 120A automatically creates (automatically duplicates) preset information. Specifically, the processing unit 120A creates first preset information from second preset information based on the relative positions of the first camera and second camera determined by the processing in step S302. For example, the relative position may be the direction of movement and distance between the first camera (camera 150A) and the second camera (camera 150B). The relative position here may also include a rotation amount representing the difference in the reference orientation.

[0067] As described above using Figure 5, the existing preset information may include not only the horizontal angle θ, vertical angle φ, and zoom magnification M for controlling the PTZ camera, but also the coordinate values ​​(X, Y, Z) and distance R of the Cartesian coordinate system. The Cartesian coordinate system here is the Cartesian coordinate system based on the existing second camera, camera 150B. Hereafter, the Cartesian coordinate system based on camera 150B will also be referred to as Cartesian coordinate system B. If the Cartesian coordinate system based on the newly placed first camera, camera 150A, is called Cartesian coordinate system A, then if the direction and amount of translational movement and the amount of rotation between Cartesian coordinate system A and Cartesian coordinate system B are known, it is possible to convert the coordinate values ​​of Cartesian coordinate system B to the coordinate values ​​of Cartesian coordinate system A by coordinate transformation. The amount of translational movement of Cartesian coordinate system A with respect to Cartesian coordinate system B can be expressed by a vector indicating the direction and distance of movement from camera 150B to camera 150A. The amount of rotation of Cartesian coordinate system A with respect to Cartesian coordinate system B is the amount of rotation that represents the difference between the reference orientation of camera 150B and the reference orientation of camera 150A.

[0068] In other words, if the relative positions of camera 150A and camera 150B are known, the processing unit 120A can determine the coordinate values ​​of the preset position in Cartesian coordinate system A based on the coordinate values ​​of the preset position included in the preset information of camera 150B. Then, the processing unit 120A can convert the coordinate values ​​in Cartesian coordinate system A to coordinate values ​​in polar coordinate system. The coordinate values ​​in polar coordinate system are the horizontal angle θ, the vertical angle φ, and the distance R. Furthermore, once the distance R is determined, it is possible to determine the approximate zoom magnification M. For example, if the processing unit 120A had set the zoom magnification to the 15th level when the distance between camera 150B and the subject was 15m, then when the distance between camera 150A and the subject is 5m, it will set the zoom magnification of camera 150A to the 5th level.

[0069] As described above, based on the relative positional relationships and existing preset information, the coordinate values ​​(X, Y, Z), horizontal angle θ, vertical angle φ, distance R, and zoom magnification M in the Cartesian coordinate system A can be determined, making it possible to automatically create preset information.

[0070] In step S305, the processing unit 120A performs correction processing on the preset information obtained in step S304. This correction processing can be rephrased as accuracy adjustment processing. Details of the correction processing will be described later using Figure 11-13D. By performing correction processing, the accuracy of the preset information can be improved, so there is no need to make the accuracy excessively high at the automatic creation stage. Therefore, as in the example of the zoom magnification M described above, it becomes possible to use approximate values ​​in the automatic generation process. Also, as will be described later, when determining the relative positions of two cameras 150, it is sometimes assumed that the reference orientation of the two cameras 150 is the same. In that case, even if an error occurs due to a discrepancy in the reference orientation, this error can be reduced by the correction processing. In other words, since high accuracy is not required in adjusting the reference orientation, it becomes possible to improve user convenience.

[0071] In step S306, the processing unit 120A determines whether the automatic creation of preset information has been completed for all preset positions. For example, if information corresponding to 32 preset positions is already registered in the existing preset information, the processing unit 120A determines whether the corresponding 32 sets of parameters have been automatically created. If there are still preset positions for which preset information has not been automatically created (step S306: No), in step S307, the processing unit 120A increments the value of N and then returns to the process in step S304. In other words, the processing unit 120A performs the automatic creation of preset information process (step S304) and the correction process (step S305) described above for the new preset positions.

[0072] Once processing is complete for all preset positions (step S306: Yes), the process shown in Figure 6 is terminated.

[0073] 2.3 Acquisition of relative position and automatic creation of preset information Next, we will explain in detail the process of acquiring the relative positions of the two cameras 150 shown in step S302 of Figure 6, and the process of automatically creating preset information shown in step S304.

[0074] <Image capture of camera A by camera B> The processing unit 120 of the electronic device 100 may determine the relative position by measuring the distance and direction between the second camera and the first camera based on the image captured by the second camera, when the first camera is included in the image captured by the second camera. In this way, preset information can be automatically created by simple control, such as capturing an image of the newly placed camera 150 with the existing camera 150. Here, "based on the image captured by the second camera" may mean that the image captured by the second camera is used in the control to realize the state in which the first camera is included in the image captured by the second camera, or that the image captured by the second camera is used in the process of measuring the distance between the second camera and the first camera, or both.

[0075] First, the electronic device 100B, which corresponds to the existing camera 150B, uses camera 150B to image camera 150A after the installation of the new camera 150A as shown in step S301 of Figure 6. Specifically, the processing unit 120B of the electronic device 100B accepts user operations to center camera 150A in the captured image. User operations are performed, for example, using the operation interface of the management terminal device 200. As a result, the processing unit 120 of the electronic device 100 obtains the horizontal angle θ, vertical angle φ, and zoom magnification M when camera 150B images camera 150A. The processing unit 120B also determines the distance R from camera 150B to the subject camera 150A using the various methods described above. Furthermore, the processing unit 120B determines the coordinate values ​​(X, Y, Z) of camera 150A in the Cartesian coordinate system B based on the horizontal angle θ, vertical angle φ, and distance R. This process is the same as the process described above using Figure 4B, except that the target is changed from the monitored target (preset position) to camera 150A. In other words, the processing unit 120B of the electronic device 100B performs a process to acquire preset information with the position of camera 150A as the preset position.

[0076] Figure 7 shows the relationship between camera 150A, camera 150B, and a given preset position. Note that Figure 7 uses two-dimensional coordinates for simplicity; however, those skilled in the art will readily understand that the following explanation can be extended to three-dimensional coordinates.

[0077] In Figure 7, the solid lines representing the X and Y axes represent the two axes that constitute the Cartesian coordinate system B, with camera 150B as the reference point, and camera 150B is positioned at the origin (0,0) of the Cartesian coordinate system B. In Figure 7, the upward direction in the drawing is the positive X-axis direction, and the rightward direction is the positive Y-axis direction. As shown in Figures 4A and 4B, the distance and direction from camera 150B to the preset position are known. In Figure 7, distance is denoted by S and angle by α. As mentioned above, by having camera 150B image camera 150A, the distance and direction from camera 150B to camera 150A are also known. In Figure 7, distance is denoted by T and angle by β.

[0078] As described above, it is possible to convert polar coordinate values ​​to Cartesian coordinate values, and the coordinate values ​​of camera 150A in Cartesian coordinate system B, with camera 150B as the reference, are (X,Y)=(Tsinβ,Tcosβ). This allows us to obtain a vector representing the distance and direction between the second camera and the first camera.

[0079] In this case, the reference orientation of camera 150A may be the same as the reference orientation of camera 150B. The processing unit 120A may determine the state in which the first camera (camera 150A) has imaged a predetermined target object under predetermined conditions as the reference orientation of the first camera, and determine the first preset information based on the reference orientation of the first camera and its relative position. In this way, since the reference orientation of camera 150A is known, the amount of rotation of camera 150A relative to camera 150B can be determined.

[0080] For example, if the reference orientation of camera 150B is set using Polaris, then when installing camera 150A, the reference orientation of camera 150A is also set using Polaris. The specific setting method is the same as in the example described above. This makes it possible to consider the reference orientations of camera 150B and camera 150A to be the same. Therefore, the Cartesian coordinate system A, with camera 150A as the reference, has the X and Y axes in the direction of the dashed lines in Figure 7. The X axis of Cartesian coordinate system A is parallel to the X axis of Cartesian coordinate system B. The Y axis of Cartesian coordinate system A is parallel to the Y axis of Cartesian coordinate system B. The same applies to the Z axis when extended to three dimensions.

[0081] As a result, the relative relationship between the two coordinate systems is such that the translational movement is (Tsinβ, Tcosβ) and the rotation is 0. This makes it possible to convert the coordinate values ​​of a preset position in Cartesian coordinate system B to coordinate values ​​in Cartesian coordinate system A. For example, in Figure 7, the coordinate values ​​of the preset coordinates in Cartesian coordinate system B are (X,Y)=(Ssinα, Scosα), but these can be converted to the coordinate values ​​in Cartesian coordinate system A, which are (X,Y)=(Usinγ, Ucosγ). U and γ can be expressed using S, T, α, and β, respectively.

[0082] Note that the subject used to determine the reference attitude is not limited to Polaris. For example, the subject used to determine the reference attitude may be a large object that can be observed even at a distance greater than a predetermined distance, such as a mountain. In this case, the distance from camera 150 to the mountain is too close to assume infinity, so errors may occur in the reference attitude between multiple cameras 150. However, in this embodiment, the accuracy of the preset information automatically created by the correction process shown in step S304 of Figure 6 can be improved, making it possible to suppress such errors.

[0083] Furthermore, when camera 150 is imaging an interior space, or when there is no suitable target, the initial orientation may be set based on the direction indicating "north" as determined by a compass, or the direction of gravity. The direction of gravity is, for example, the direction a string would point if a weight were suspended from the end of a string and it were stationary. For example, the user can perform panning and tilting operations so that the direction indicating north or the direction of gravity aligns with the optical axis of camera 150. In this way, it becomes possible to align the reference orientation of multiple cameras even indoors.

[0084] Figure 8 is a flowchart illustrating the automatic creation process of preset information shown in step S304 of Figure 6. The process in Figure 8 is performed in the electronic device 100A corresponding to the newly placed camera 150A, similar to the process in Figure 6.

[0085] In step S401, the processing unit 120A obtains the Nth preset information from the existing preset information for camera 150B. In the example in Figure 7, the processing unit 120 obtains the coordinate values ​​of the preset coordinates, (X,Y)=(Ssinα,Scosα).

[0086] In step S402, the processing unit 120A calculates the coordinate values ​​of the preset position as seen from camera 150A. The coordinate values ​​of the preset position as seen from camera 150A represent the coordinate values ​​in Cartesian coordinate system A. As is clear from Figure 7, the coordinate values ​​(X,Y)=(Usinγ,Ucosγ) of the preset position in Cartesian coordinate system A can be expressed using S, T, α, and β, so a detailed explanation is omitted.

[0087] In step S403, the processing unit 120A calculates the distance from the camera 150A to the preset position. Specifically, the processing unit 120A calculates U. U is calculated from the known values ​​S, T, α, and β.

[0088] In step S404, the processing unit 120A calculates the zoom magnification M based on the distance U obtained in step S403. The zoom magnification M here may be an approximate value as described above. For example, the storage unit 140A may store table data that associates distance with zoom magnification. The processing unit 120A determines the zoom magnification M based on the distance U obtained in step S403 and the said table data.

[0089] In step S405, the processing unit 120A determines the horizontal angle θ and the vertical angle φ. Since Figure 7 illustrates a two-dimensional coordinate system, only the angle γ is needed, and γ is determined from the known values ​​S, T, α, and β. By extending this to three dimensions, it is possible to determine the horizontal angle θ and the vertical angle φ.

[0090] Through the above processing, the processing unit 120A can obtain information including the horizontal angle θ, vertical angle φ, and zoom magnification M as information for imaging the Nth preset position using the camera 150A. In step S406, the processing unit 120 stores the obtained information in the storage unit 140A as preset information for the camera 150A. The processing unit 120A may also add the coordinate values ​​(X, Y, Z) in the Cartesian coordinate system A obtained in step S402, and the distance R obtained in step S403, to the preset information.

[0091] <Image capture of a reference object by cameras A and B 1> Furthermore, the processing unit 120 may determine the relative position by measuring at least one of the distance and direction between the first camera and the reference object based on the image captured by the first camera (camera 150A) and the image captured by the second camera (camera 150B) when one or more reference objects are included in the image captured by the first camera and the image captured by the second camera. Here, "based on the image captured by the first camera" may indicate that the image captured by the first camera is used for control to realize the state in which the image captured by the first camera includes a reference object, or that the image captured by the first camera is used for processing to measure the distance between the first camera and the reference object, or both. The same applies to the statement "based on the image captured by the second camera".

[0092] The reference object here may be an object located at a preset position. In this case, the process of measuring at least one of the distance and direction between the second camera (camera 150B) and the reference object based on the image captured by the second camera (camera 150B) is performed in the initial setup process of camera 150B shown in Figures 4A and 4B. Preset information related to camera 150B may be used in determining the relative position.

[0093] Alternatively, the reference object may be an object used for determining relative positions, independent of preset positions. In this case, the reference object may be an object such as a flag that the user places to determine the relative position.

[0094] For example, as described above using Figure 7, if the reference orientation of camera 150A is known, the number of reference objects may be one. Below, we will describe an example in which the reference orientation of camera 150B and the reference orientation of camera 150A are the same.

[0095] Figure 9 shows the relationship between camera 150B, camera 150A, and a reference object. First, the control unit 110B of the electronic device 100B receives a user command to capture the reference object near the center of the captured image. The processing unit 120B obtains the horizontal angle θ and vertical angle φ at that time. The processing unit 120B also measures the distance from camera 150B to the reference object using one of the various methods described above. As a result, the processing unit 120B can determine the distance and direction from camera 150B to the reference object. In the example in Figure 9, the distance is S and the angle representing the direction is α.

[0096] Similarly, the control unit 110A of the electronic device 100A can determine the distance and direction from the camera 150A to the reference object by receiving a user operation to capture the reference object near the center of the captured image. In the example in Figure 9, U is the distance and γ is the angle representing the direction.

[0097] The control unit 110A of the electronic device 100A acquires the distance S and angle α from the electronic device 100B and outputs them to the processing unit 120A. The processing unit 120A determines the coordinate values ​​of the reference object in the Cartesian coordinate system B based on the distance S and angle α. In the example in Figure 9, the coordinate values ​​of the reference object are (X,Y)=(Ssinα,Scosα).

[0098] Furthermore, the processing unit 120A acquires information that the distance between camera 150A and the reference object is U. Therefore, camera 150A is located somewhere on a circle with radius U centered at (Ssinα,Scosα), which is the position of the reference object. In Figure 9, this circle is shown as a dashed line.

[0099] As mentioned above, the reference orientation of camera 150A is the same as that of camera 150B. Therefore, the X and Y axes of Cartesian coordinate system A, with camera 150A as the reference, are parallel to the X and Y axes of Cartesian coordinate system B, respectively. Also, as mentioned above, the processing unit 120A obtains information that the angle of the reference object relative to camera 150A is γ. In the example in Figure 9, γ is the angle between the direction from camera 150A toward the reference object and the positive Y-axis. This allows the position of camera 150A to be determined as a single point on a circle of radius U.

[0100] This allows the processing unit 120A to determine the coordinate values ​​of the camera 150A in the Cartesian coordinate system B. For example, the processing unit 120A finds a line that is parallel to a line with an angle γ with respect to the positive Y-axis in the Cartesian coordinate system B and passes through the coordinates of the reference object. This line passes through both the camera 150A and the reference object. The processing unit 120A then determines the coordinate values ​​of the camera 150A in the Cartesian coordinate system B by finding the intersection of the determined line and a circle with radius U centered on the reference object. Since the line here passes through the reference object, i.e., the center of the circle, two intersection points appear, but it is possible to determine which one to select depending on the magnitude of γ.

[0101] The processing after the coordinate values ​​of camera 150A in Cartesian coordinate system B have been determined is the same as in the example described above, using Figures 7 and 8. Specifically, the distance and direction of translational movement from Cartesian coordinate system B to Cartesian coordinate system A correspond to the coordinate values ​​of camera 150A in Cartesian coordinate system B. Also, since the reference orientation is common, the amount of rotation of Cartesian coordinate system A with respect to Cartesian coordinate system B is 0. The processing unit 120 obtains the coordinate values ​​in Cartesian coordinate system A by performing a coordinate transformation based on relative position on the coordinate values ​​included in the preset information of camera 150B (steps S401, S402). Then, the processing unit 120A performs the process of converting the coordinate values ​​in Cartesian coordinate system A to polar coordinates, and the process of determining the zoom magnification M from the distance R, which is the coordinate value in polar coordinates (steps S403-S405).

[0102] <Image capture of a reference object by cameras A and B 2> Furthermore, the reference object used to determine the relative positions of camera 150B and camera 150A is not limited to one; there may be multiple reference objects. For example, there may be three reference objects. In this case, even if the reference orientation of camera 150A is unknown, it becomes possible to determine the relative position and automatically create preset information for camera 150A.

[0103] Figure 10A illustrates the relationship between camera 150B and three reference objects. Hereafter, the three reference objects will be referred to as reference object 1, reference object 2, and reference object 3.

[0104] The control unit 110B of the electronic device 100B receives user input such that reference object 1 is captured near the center of the image captured by camera 150B. The processing unit 120B acquires the horizontal angle θ and vertical angle φ based on the amount of pan and tilt movement input at that time. Similarly, the control unit 110B also acquires the corresponding horizontal angle θ and vertical angle φ for reference objects 2 and 3. The processing unit 120B also measures the distance from camera 150B to each of reference objects 1-3 using one of the various methods described above. As a result, the processing unit 120B can determine the distance and direction from camera 150B to reference objects 1-3. The control unit 110A of the electronic device 100A acquires information representing the distance and direction from camera 150B to reference objects 1-3 via the communication unit 130 and outputs this information to the processing unit 120A.

[0105] Figure 10B is a diagram illustrating the relationship between camera 150A and three reference objects. Similar to electronic device 100B, the control unit 110A of electronic device 100A determines the distance and direction from camera 150A to each of reference objects 1 to 3 by receiving user input such that reference objects 1 to 3 are captured near the center of the image captured by camera 150A.

[0106] The processing unit 120A determines the coordinate values ​​of reference object 1 in Cartesian coordinate system B based on the distance and direction between camera 150B and reference object 1. Similarly, the processing unit 120A determines the coordinate values ​​of reference object 2 and reference object 3 in Cartesian coordinate system B.

[0107] Furthermore, the processing unit 120A has already obtained the distance and direction from camera 150A to each of the reference objects 1 to 3. When the distances between camera 150A and reference objects 1 to 3 are R1 to R3, respectively, the processing unit 120A can determine the coordinates of camera 150A in the Cartesian coordinate system B by finding the intersection points of a circle with radius R1 centered on the coordinates of reference object 1, a circle with radius R2 centered on the coordinates of reference object 2, and a circle with radius R3 centered on the coordinates of reference object 3. Depending on the precision, the three circles may not intersect at a single point. Therefore, the processing unit 120A may set the position of camera 150A to a point that minimizes the evaluation value obtained based on the distance to each circle.

[0108] In this way, by using the distances from the three reference objects, the processing unit 120A can determine the coordinate values ​​of camera 150A in the Cartesian coordinate system B, even when the reference orientation of camera 150A is unknown.

[0109] Furthermore, if the coordinate values ​​of camera 150A are known, processing unit 120A can calculate a first angle representing the direction from camera 150A to each reference object, assuming that the reference orientation of camera 150A is the same as that of camera 150B. Processing unit 120A also obtains an angle (hereinafter referred to as the second angle) representing the direction from camera 150A to reference objects 1 to 3 based on the captured images of reference objects 1 to 3. In other words, processing unit 120A can determine the amount of rotation that represents how much the reference orientation of camera 150A deviates from the reference orientation of camera 150B, based on the difference between the first angle and the second angle. Processing unit 120A may also perform a process to correct the reference orientation of camera 150A so that its reference orientation is the same as that of camera 150B.

[0110] Through the above process, the direction and distance of translational movement between Cartesian coordinate system A and Cartesian coordinate system B, as well as the amount of rotation, can be determined, making it possible to convert the coordinate values ​​of Cartesian coordinate system B to the coordinate values ​​of Cartesian coordinate system A. Therefore, the subsequent processing is the same as in the example described above using Figures 7 and 8.

[0111] As described above, the reference object may include the first to third reference objects. The processing unit 120 may determine the first distance between the first camera and the first reference object, the second distance between the first camera and the second reference object, and the third distance between the first camera and the third reference object based on the captured images taken by the first camera (camera 150A) of the first reference object, the second reference object, and the third reference object, respectively. Based on the first, second, and third distances, the processing unit 120 determines the coordinates of the first camera in a second coordinate system (the Cartesian coordinate system B of camera 150B) with the second camera as the reference, thereby determining the first preset information. In this way, it becomes possible to automatically create preset information without strictly setting the reference orientation of camera 150A.

[0112] 2.4 Correction of Preset Information Next, the correction process for preset information will be explained. The processing unit 120 of the electronic device 100 that automatically created the preset information may perform the correction process in the following manner, for example. First, the processing unit 120 determines the main object by performing object detection processing on the second image, which is an image captured by the second camera at the preset position. The processing unit 120 determines an evaluation value for evaluating the imaging state of the main object determined from the second image in the first image by performing object detection processing on the first image, which is an image captured by the first camera at the preset position. Then, the processing unit 120 corrects the first preset information for the first camera to image the preset position so that the evaluation value satisfies the given conditions.

[0113] In this way, the processing unit 120 (for example, processing unit 120A) can detect the object being monitored by the existing second camera (for example, camera 150B) as the main object, and then correct the preset information based on the imaging state of the main object in the newly placed first camera (for example, camera 150A). As a result, it becomes possible to set preset information in the second camera as well so that the object being monitored is imaged in an appropriate state. Because the accuracy of the preset information is improved by the correction process, a certain degree of error can be tolerated during the automatic creation of the preset information, improving convenience.

[0114] In the correction process according to this embodiment, as described above, an object extraction process, including the detection of major objects, is performed in the captured image of camera 150B (hereinafter also referred to as captured image B). This process is performed during the initial setup of camera 150B, for example, as shown in step S104 of Figure 4A. However, the object extraction process based on captured image B may be performed at other times. For example, the object extraction process based on captured image B may be performed when camera 150A is added. Alternatively, as will be described later using Figures 16A-16B, etc., the object extraction process may be performed periodically in the patrol process using preset information, and the result of this process (in a narrow sense, the latest processing result) may be used in the correction process of the preset information.

[0115] Figure 11 is a flowchart illustrating the object extraction process shown in step S104 of Figure 4A. In step S501, the processing unit 120B of the electronic device 100B acquires the image captured by the camera 150B. Although Figure 11 shows an example of acquiring a moving image, the processing unit 120B may also acquire a still image. The image acquired here corresponds to the captured image B described above.

[0116] In step S502, the processing unit 120B performs the process of detecting objects from the acquired image B. Various methods are known for object detection, and since these methods can be widely applied in this embodiment, a detailed explanation is omitted. For example, the storage unit 140B stores pattern learning data, and the processing unit 120B may perform the object detection process based on this pattern learning data.

[0117] In step S503, the processing unit 120B performs a process to calculate an evaluation value for each detected object. The evaluation value here may be information in which the value increases the closer each object is to the center of the captured image B, and the larger the image is.

[0118] Figure 12A is an example of captured image B, and Figure 13A is an example of evaluation values ​​calculated based on said captured image B. As shown in Figure 12A, captured image B is an image taken inside a park, and a mound, which is a playground structure where children are likely to gather, is captured near the center as the target of monitoring. Although not shown in Figure 12A, it is assumed that other playground equipment and structures placed in the park, such as monkey bars, slides, and fences, are also captured in captured image B.

[0119] As shown by the dashed ellipse in Figure 12A, the processing unit 120B may, for example, divide the captured image B into multiple regions corresponding to their distance from the center. In the example in Figure 12A, the captured image B is divided into multiple regions by a plurality of concentric ellipses. As shown in Figure 13A, these multiple regions are here referred to as the central circle, outer circle 1, outer circle 2, outer circle 3, and outer circle 4. The central circle represents the region inside the ellipse closest to the center. Outer circle 1 represents the region outside the ellipse closest to the center and inside the second closest ellipse. Similarly, outer circles 2 and beyond represent regions in this order of increasing distance from the center.

[0120] In step S503, the processing unit 120B detects feature points for each object detected in step S502 and counts which region each feature point is located in. In the example in Figure 13A, 100 feature points related to the mound were detected in the central circle, 90 in outer circle 1, 50 in outer circle 2, 10 in outer circle 3, and 10 in outer circle 4. The method of determining feature points here is arbitrary; they may be points detected by edge detection or points detected by other image processing. If semantic segmentation is used for object detection, each pixel representing an object may be used as a feature point. Alternatively, the processing unit 120B may identify the region of the object and calculate the overlap ratio between that region and the central circle, outer circle 1, outer circle 2, outer circle 3, and outer circle 4.

[0121] Furthermore, the processing unit 120B calculates the total score as an evaluation value by multiplying the number of feature points in each region by a weight and then summing them up. As mentioned above, since the object closest to the center is considered the main object here, the central circle has a large weight and the outer circle 4 has a small weight. Figure 13A shows an example where the weight of the central circle is 5, the weight of outer circle 1 is 4, the weight of outer circle 2 is 3, the weight of outer circle 3 is 2, and the weight of outer circle 4 is 1. However, the weight examples are not limited to this. Also, the evaluation value used to determine the main object is not limited to the total score explained above.

[0122] In step S504, the processing unit 120B determines the main object based on the total points. In the example in Figure 13A, four objects B1 to B4 are detected, with total points of 1040, 100, 15, and 5, respectively. Therefore, the processing unit 120B selects object B1, which has the highest total points, as the main object. In this example, object B1 is a mound.

[0123] In step S505, the processing unit 120B registers the selected main object. For example, the processing unit 120B stores the information of the main object in the storage unit 140B. Alternatively, the processing unit 120B may transmit the information of the main object to the management terminal device 200 via the communication unit 130B. The processing unit 120B may also register an image containing the main object. The image here may be the entire captured image B in which the main object is captured, or it may be a cropped image of the captured image B that includes the main object.

[0124] With the above steps, the main object when the existing camera 150B captures an image at a given preset position is registered. Based on this main object, the electronic device 100A corrects the preset information so that the newly placed camera 150A can capture an image at the same preset position.

[0125] Specifically, the processing unit 120 may calculate a higher evaluation value when the preset position of the main object detected in the object detection process for the first captured image is close to the center of the first captured image, compared to when the preset position is far from the center. The processing unit 120 is, for example, processing unit 120A. The first captured image is, for example, captured image A. The preset position of the main object is the position of the main object in captured image A, which is captured using the current preset information. In other words, the preset information is corrected so that the main object is captured near the center and of a certain size in the electronic device 100A. This will be explained in detail below.

[0126] Figure 14 is a flowchart illustrating the correction process of preset information in the electronic device 100A, as shown in step S305 of Figure 6. In step S601, the camera 150A acquires an image A that captures the target preset position. Specifically, the control unit 110A uses the horizontal angle θ, vertical angle φ, and zoom magnification M determined in the above-described process using Figure 8 to perform pan, tilt, and zoom operations on the camera 150A. The control unit 110A then outputs the image captured by the camera 150A as image A to the processing unit 120A.

[0127] In step S602, the processing unit 120A detects objects from the captured image A. In step S603, the processing unit 120 calculates a total score, which is an evaluation value, for each object. The specific processing is the same as steps S502-S503 in Figure 11, so a detailed explanation is omitted.

[0128] Figure 12B shows an example of image A acquired in step S601. Figure 13B shows an example of evaluation values ​​calculated based on the image A. As shown in Figure 12B, an artificial hill is captured in image A, but its position is shifted to the right of the center, and its size is smaller than that of Figure 12A.

[0129] As shown in Figure 13B, objects A1 to A4 were detected in the captured image A, with total points of 23, 5, 405, and 25 respectively. In this case, object A3, which has the largest total points, is likely the main object (the mound in this example), but depending on the error in the automatically generated preset information, it is also possible that one of the other objects is the main object.

[0130] Therefore, the processing unit 120 may perform pattern matching between the main object detected in the object detection process for the second captured image and the object detected in the object detection process for the first captured image, and determine the object with a high degree of match as the main object in the first captured image. Here, the processing unit 120 is, for example, processing unit 120A. Also, for example, the first captured image is captured image A, and the second captured image is captured image B. In this way, the main object in captured image A can be appropriately determined based on the degree of match between objects.

[0131] Specifically, in step S604, the processing unit 120A acquires an image of the main object captured by the camera 150B. This image is the image registered in step S505 of Figure 11. The processing unit 120A may acquire the image from the electronic device 100B or from the management terminal device 200.

[0132] In step S605, the processing unit 120A performs pattern matching between each of the objects detected in captured image A and the main object in captured image B. Since pattern matching is publicly known, a detailed explanation is omitted.

[0133] Figure 13C shows the results of pattern matching. Here, an example is shown of determining the degree of agreement between objects A1 to A4 and objects B1 to B4. As shown in Figure 13C, object B1, which is the main object, and object A3 have a high degree of agreement. Therefore, in step S606, the processing unit 120A determines that the object with a higher degree of agreement with the main object compared to the other objects is the main object in the captured image A.

[0134] In step S607, the control unit 110A performs panning and tilting operations on the camera 150A to adjust the image A so that the main object is in the center of the field of view. For example, the processing unit 120A may perform a process to determine the drive amounts for panning and tilting based on the position of the main object in the state before correction (corresponding to Figure 12B in this case). The control unit 110A controls the camera 150A based on the drive amounts determined by the processing unit 120A.

[0135] Figure 12C shows the state of image A after the adjustment in step S607 has been performed. The adjustment shown in step S607 causes the main object, the artificial hill, to be captured near the center of image A.

[0136] Furthermore, in step S608, the control unit 110A performs a process to adjust the size of the main object in the captured image A so that it falls within a predetermined range by zooming the camera 150A.

[0137] Figure 12D shows the state of image A after the adjustment in step S608 has been performed. The adjustment shown in step S608 ensures that the main object, the artificial hill, is imaged at a size greater than a certain threshold.

[0138] In step S608, during the adjustment process, the processing unit 120A may periodically perform object detection and total point calculation. The control unit 110A then performs a zoom operation until the total points of the main object exceed a predetermined threshold. In this way, the zoom magnification can be adjusted so that the main object is imaged in an appropriate state. However, if the zoom magnification M is too large, the main object can be imaged larger, but information about other objects may be lost. Therefore, the processing unit 120A may consider both the total points of the main object and the total points of the object with the second highest total points to determine whether or not to perform a zoom operation and in what direction to change the zoom.

[0139] In step S609, the processing unit 120A registers the adjusted horizontal angle θ, vertical angle φ, and zoom magnification M as preset information after correction processing. The processing unit 120A stores the corrected preset information in the storage unit 140A. The processing unit 120A may also transmit the corrected preset information to the management terminal device 200. The processing unit 120A may also determine the distance R to the main subject (distance to the preset position) and the coordinate values ​​(X, Y, Z) in the Cartesian coordinate system A in the corrected state, and register this information as preset information.

[0140] If no matching pattern is found in step S605, the processing unit 120A may change the zoom magnification M of the camera 150A's preset information to adjust the field of view by zooming out or zooming in. By controlling the camera 150A according to the zoom magnification M, the control unit 110A can change the imaging range and the size of the subject, making it possible to search for an object with a high degree of matching.

[0141] Furthermore, the angle at which the object is captured varies significantly depending on the position of camera 150, which may raise concerns about the accuracy of matching detection. For example, if camera 150B images the surface of an object and camera 150A images the back surface of the object, the matching degree will decrease if the object has significantly different shapes on its front and back surfaces. In this case, processing unit 120A may perform 3D modeling of the object from the image and recalculate the matching degree by changing the orientation of the 3D object. Note that the method for performing 3D modeling of objects on a planar image is publicly known, so a detailed explanation will be omitted.

[0142] Furthermore, if the monitoring location is an open field or similar area where it is difficult to extract static objects, one possible solution is to temporarily place objects such as identification flags (printed with identification patterns) at preset locations while the camera is being set up, and use those objects as a reference.

[0143] 3. Patrol The monitoring device according to this embodiment may perform patrols to monitor a predetermined area by controlling the camera 150 in a predetermined pattern. In patrols, it is not relevant whether the preset information was created manually or automatically, so in the following description, multiple electronic devices 100 will not be distinguished from each other unless necessary.

[0144] 3.1 Specific Examples of Patrols For example, the control unit 110 of the electronic device 100 may perform normal patrol control, which mainly monitors predetermined preset positions, and automatic patrol control, which differs from normal patrol. For example, the automatic patrol control may be a control that comprehensively monitors the range that the camera 150 can monitor. Alternatively, the automatic patrol control may be a control that corresponds to a pattern automatically generated by randomly rearranging the patterns of normal patrol control, if the patterns of normal patrol control are known.

[0145] Figure 15 is a flowchart illustrating patrol. In step S701, the control unit 110 reads preset information from the storage unit 140. In step S702, the control unit 110 reads patrol settings from the storage unit 140. The control unit 110 may also obtain one or both of the preset information and / or patrol settings from the management terminal device 200.

[0146] Figures 16A and 16B illustrate the patrol settings. Figure 16A shows an example of data that determines the patrol pattern. A patrol is, for example, a pattern of multiple routes, each route defined by movement to a preset position, stopping, panning, tilting, zooming, etc. In Figure 16A, patterns corresponding to patrol IDs 1 to 3 are normal patrols, and patterns corresponding to 97 to 99 are automatic patrols.

[0147] For example, a patrol with Patrol ID 1 follows a pattern where, for each point from P1 to P4, the patrol moves to the target point (using pan, tilt, and zoom controls to monitor the target position), and then monitors that point for 10 seconds. This patrol then monitors in the order of P1 → P2 → P3 → P4, before returning to P1 and repeating the same control. Here, P1 to P4 each represent a preset position. A patrol with Patrol ID 2 repeatedly moves between P1 and P2, while periodically performing slow movements. A patrol with Patrol ID 3 follows a pattern where the patrol moves sequentially through the preset positions.

[0148] Patrol ID 97 is a pattern where the patrol visits 36 points in 60-degree increments, with horizontal angles θ = 0 to 360 degrees and vertical angles φ = 0 to 360 degrees, as seen from a wide-angle view. Patrol ID 99 is a pattern where the patrol moves in small increments of 10 degrees, with horizontal angles θ = 0 to 360 degrees and vertical angles φ = 0 to 360 degrees, to view the whole picture. Patrol ID 98 corresponds to a pattern where the route is randomly rearranged, for example, based on patrol ID 3.

[0149] Figure 16B shows the relationship between the day of the week and the patrols performed. For example, patrols are performed four times each day at 0:00, 6:00, 12:00, and 18:00, and the patrol to be performed at each time is indicated by an ID. Note that multiple patrols with different patterns may be performed in a single patrol, such as at 6:00 on Tuesday. For example, since preset locations are considered to have a high need for monitoring, the user of the management terminal device 200 determines the timing of the normal patrol and the ID of the normal patrol to be performed. The control unit 110 may perform automatic patrol control at times when normal patrols are not performed (squares with a colored background in Figure 16B).

[0150] Let's return to the flowchart in Figure 15 and continue the explanation. In step S703, the control unit 110 determines whether the patrol to be performed is a normal patrol or an automated patrol, based on the current day of the week and time, and the schedule shown in Figure 16B.

[0151] If it is determined that it is time to conduct a normal patrol, in step S704, the control unit 110 identifies a route based on the patrol ID and the table shown in Figure 16A, and performs normal patrol control by controlling the camera 150 to monitor according to that route. For example, the control unit 110 continuously transmits moving images to the management terminal device 200. The display unit of the management terminal device 200 displays the moving images on a screen such as the one shown in Figure 3. The processing unit 120 performs object extraction processing based on the captured images. For example, the processing unit 120 may perform processing to find the total number of points for each detected object, similar to the example described above, using Figures 13A and 13B.

[0152] In step S705, the processing result of the processing unit 120 is saved as historical data. As shown in Figure 16B, since the same patrol pattern is executed repeatedly, by storing historical data each time a patrol is performed, data representing the time-series monitoring results can be obtained. Here, we are considering a normal patrol, so the object detection process can be considered to be information that shows the time-series changes of objects present at the preset locations.

[0153] For example, in step S706, the processing unit 120 may determine whether the shape of an object located at a preset position has changed. Since the object located at the preset position is the object being monitored, it will also be referred to as the target below. Specific examples of processing related to target changes will be described later.

[0154] On the other hand, if the determination in step S703 is determined to indicate that it is time to perform an automatic patrol, in step S707, the control unit 110 identifies a route based on the patrol ID and the table shown in Figure 16A, and performs automatic patrol control by controlling the camera 150 to monitor according to that route.

[0155] The saving of history data in step S708 is the same as the process in step S705. However, in automatic patrol control, comprehensive monitoring may be performed regardless of the preset location, as shown in the example of patrol ID = 97 or 99. In this case, monitoring can be extended to locations that were not set as preset locations because no objects to be monitored existed there. As a result, by using the history data of automatic patrol control, the processing unit 120 can detect that an object that did not previously exist has been added, that is, that a significant change has occurred in the monitored environment.

[0156] Therefore, in step S709, the processing unit 120 may perform a process to determine whether or not a new object exists. Specific examples of new object determination will be described later.

[0157] 3.2 Target Change Next, we will explain the process related to target changes shown in step S706 of Figure 15. Figure 17 is a diagram illustrating how the shape of the target changes over time. For example, the target here may be a tree. Since trees grow over time, their height and width may increase, as shown in Figure 17.

[0158] For example, suppose preset information is set assuming the state shown at the far left of Figure 17. This preset information is intended to capture relatively small trees in the appropriate position and size in the captured image. However, if the trees grow to the state shown at the far right, using the above preset information as is will result in the trees appearing larger in the image. As a result, for example, the top of the trees may fall outside the field of view, or the area occupied by the trees in the captured image may become larger, potentially hindering the monitoring of other objects.

[0159] Therefore, the processing unit 120 may determine the shape change of an object present at a given preset position based on the time-series changes in the captured image, and update the preset information corresponding to that preset position if the degree of shape change is greater than or equal to a first threshold. The update here may include adjustment of the field of view and zoom magnification, similar to the processing described above (steps S606-S607) in the preset information correction process.

[0160] Figure 18 shows information illustrating the time-series changes of the target, and is an example of historical data stored in step S705 of Figure 15. As shown in Figure 18, the historical data may be a collection of evaluation values ​​of the main object at a specific preset location. In the example in Figure 18, when object A8 is captured in the image at preset location 5, the total point value of object A8 is continuously acquired.

[0161] For example, on June 11, 2022, the initial monitoring date, the total points for object A8 were 1180. Over time, the total points for object A8 changed to 1240, 1300, 1150, and so on. This is thought to reflect changes such as the growth and wilting of leaves depending on the season. However, in cases where the change is such that it returns to its original state over time, the change in the object's shape is small. Therefore, the processing unit 120 maintains the preset information if the amount of change in the total points relative to the initial value is below the first threshold (for example, around 130 in this case).

[0162] In contrast, the monitoring results on June 11, 2025, show that the total points of object A8 are 1345, and the amount of change is greater than or equal to the first threshold. In this case, it is considered that an irreversible change has occurred, such as the growth of a tree shown in Figure 17. Therefore, the processing unit 120 adjusts the preset information. Specifically, the control unit 110 adjusts the target so that it is in the center of the captured image, similar to step S606, and then adjusts the zoom magnification M so that the total points become an appropriate value. For example, the control unit 110 may adjust the zoom magnification so that the total points calculated by the processing unit 120 are close to the initial value. In the example in Figure 18, by adjusting the drive amount of the pan operation by -1 step, the drive amount of the tilt operation by +1 step, and the drive amount of the zoom operation by -20 steps, the total points become 1180, the same as the initial value. Therefore, the processing unit 120 stores the horizontal angle θ, vertical angle φ, and zoom magnification M corresponding to the adjusted state in the storage unit 140 as new preset information corresponding to the preset position 5.

[0163] Furthermore, the processing unit 120 may perform a process to output alert information indicating the detection of an anomaly at a preset position if the degree of shape change is greater than or equal to a second threshold, which is greater than the first threshold. In other words, the processing unit 120 adjusts the preset information if the degree of shape change is greater than or equal to the first threshold but less than the second threshold, and outputs an alert if the degree of shape change is greater than or equal to the second threshold.

[0164] In the example shown in Figure 18, the monitoring results for December 11, 2025, show that no object corresponding to object A8 was detected, and the total score is 0. This corresponds to a significant change, such as tree felling. Therefore, the processing unit 120 can output an alert to inform the user of this significant change. The alert is output to, for example, the management terminal device 200, but the alert may be output to another device. For example, the electronic device 100 may include a notification unit (not shown in Figure 2), and this notification unit may perform the alert notification process. The notification unit here may be a speaker that provides sound notification, or a light-emitting unit that provides light notification. Furthermore, as will be described later, the monitoring system 10 of this embodiment may perform integrated management that manages the monitoring results of multiple electronic devices 100 together. In this case, not only the electronic device 100 that detected an anomaly, but also the electronic device 100 that did not detect an anomaly may provide notification. In this way, for example, it becomes possible to quickly inform people in the vicinity of an anomaly that is of high urgency. Alternatively, the electronic device 100 that detects an abnormality may provide notification in the first notification mode, while the electronic device 100 that does not detect an abnormality may provide notification in a second notification mode different from the first notification mode. In this way, it becomes possible to notify of the abnormality over a wide area and to provide notification in a manner that allows for the identification of the location where the abnormality occurred. Furthermore, various modifications can be made to the notification mode using the electronic device 100.

[0165] 3.3 New Object Detection Next, we will explain the process related to target changes shown in step S709 of Figure 15. In automatic patrol control, for example, as in patrol ID=99 in Figure 16A, the control is performed to view the whole by moving the horizontal angle θ=0 to 360 degrees and the vertical angle φ=0 to 360 degrees in small increments of 10 degrees. For example, the processing unit 120 may perform object detection processing at each position moved in 10-degree increments.

[0166] Figure 19 shows the object detection results in automatic patrol control. Figure 19 shows the results of one patrol conducted at 18:00 on May 20, 2024. Depending on the combination of horizontal angle θ and vertical angle φ, it is acceptable for the target's position to coincide with a preset position. In the example in Figure 19, the state where horizontal angle θ = 20 degrees and vertical angle φ = 20 degrees corresponds to preset position 1.

[0167] In Figure 19, "no change" indicates that the detected object was identical to an object detected at the same location at a previous time. For example, the processing unit 120 determines the change in the object by reading the data for Patrol ID=99 from the history data saved in step S708, which was executed in the past (specifically, the previous time). Alternatively, since it is assumed that the same object is detected if the horizontal angle θ, vertical angle φ, and zoom magnification M are the same, the processing unit 120 may compare data with the same values ​​but different Patrol IDs.

[0168] The processing unit 120 determines that a "new object has been found" if it detects an object that was not previously detected in the historical data. In the example in Figure 19, a new object is detected at a position corresponding to a horizontal angle θ = 60 degrees and a vertical angle φ = 40 degrees.

[0169] If a new object is detected, the processing unit 120 may add the location corresponding to the new object to the preset locations. In this way, it becomes possible to easily monitor new objects that may require monitoring using the preset information.

[0170] The processing unit 120 may, but is not limited to, use the discovery of a new object as a condition for adding a preset location. For example, the processing unit 120 may add a preset location if a new object is discovered and that object is continuously detected over a predetermined number of patrols. In this way, it becomes less likely that preset information will be set for objects that are only temporarily placed and will be removed or moved in a short time, thus suppressing the execution of monitoring that is not necessary.

[0171] 4.Integrated management Furthermore, the monitoring system 10 of this embodiment may perform integrated management to manage the monitoring results of multiple electronic devices 100 together. For example, the management terminal device 200 may acquire object detection results from each electronic device 100. When an object is found in electronic device 100B, the management terminal device 200 determines whether the object has been found in other electronic devices 100, such as electronic device 100A. The management terminal device 200 may also associate the object with a preset position on each electronic device 100. For example, if an object is detected at preset position B1 on electronic device 100B and the same object is detected at preset position A10 on electronic device 100A, the management terminal device 200 associates the object with both preset positions B1 and A10. The management terminal device 200 may also store the sum or average score calculated from multiple electronic devices 100, associating it with the object.

[0172] Furthermore, the above processing is not limited to being performed by the management terminal device 200, but may also be performed by any of the multiple electronic devices 100 functioning as a server.

[0173] Furthermore, the processing unit 120 of the electronic device 100 may update the first preset information (the preset information it stores) based on the changed preset information when the second preset information (the preset information stored by another electronic device 100) is changed. The change in the second preset information here may be an adjustment of the preset information based on the target change described above using Figures 16A-16B, or an addition of preset information based on the discovery of a new object described above using Figures 18-19.

[0174] In this way, changes to the preset information can be shared among multiple electronic devices 100. For example, if camera 150A detects tree growth that requires adjustment of the preset information, it becomes possible to adjust the preset information of camera 150B, which detects the same object, accordingly. Also, if camera 150A adds preset information corresponding to a new object, it becomes possible to add preset information for monitoring that new object using camera 150B to electronic device 100B.

[0175] In this case, the preset information of camera 150B will be adjusted and added using the preset information of camera 150A, so the processing unit 120B may perform the same processing as the automatic preset information creation process described above. Naturally, the preset information of camera 150A may be updated based on changes to the preset information of camera 150B. Furthermore, there is no preclude other cameras from being targeted as camera 150.

[0176] The process of reflecting changes to the preset information of a given camera 150 in the preset information of other cameras 150 is particularly suitable when performing the integrated management described above. For example, if it is considered ineffective to monitor a single object using a large number of cameras 150, the management terminal device 200 or the electronic device 100, which functions as a server, may perform a process to set an upper limit on the number of cameras 150 that monitor the same object. In addition, the processing unit 120 of the management terminal device 200 or the electronic device 100 may perform a process to prioritize the use of cameras 150 with high total scores for monitoring. In this way, when sharing changes to the preset information described above, it is possible to suppress the execution of unnecessary sharing processes.

[0177] Furthermore, the changes to the preset information here are not limited to the automatic adjustment and addition of preset information as described above, but may also be done manually. For example, the user manually adds or adjusts the preset information for one of the multiple cameras 150 (see Figure 4B for details). Then, the management terminal device 200 or the electronic device 100, which functions as a server, causes the other electronic devices 100 to perform the process of automatically updating the preset information based on the manually changed preset information. In this case, the user incurs a burden for changing the preset information for the first camera 150, but the burden for updating the preset information for the other cameras 150 can be reduced.

[0178] 5. Application Examples The monitoring system 10 described above is not limited to the monitoring of parks (Figure 3, etc.) and trees (Figure 17, etc.) mentioned above, but can be applied to various fields.

[0179] For example, the monitoring system 10 according to this embodiment may be used in medical facilities such as hospitals and nursing homes to detect sudden changes in the condition of patients or residents. By using the monitoring system 10, it becomes possible to detect abnormalities such as when a patient is unable to press the nurse call button. Alternatively, the monitoring system 10 may be used to monitor swimming facilities such as swimming pools or beaches where swimming is permitted. Since a drowning person is expected to disappear underwater, the monitoring system 10 makes it possible to detect water accidents, for example, as a change in an object (Figure 17). From the perspective of the high probability of water accidents, facilities used by children may be targeted. Hot spring facilities may also be included as targets. Furthermore, the monitoring system 10 of this embodiment may be used to detect the occurrence or arrival of tsunamis by monitoring changes in the horizon. In these cases, from the perspective of early detection of abnormalities, it is desirable that each device of the monitoring system 10 has high-performance time-division capability. According to the method of this embodiment, since it is easy to add or move cameras 150, it is easy to construct a system that can monitor all locations prone to abnormalities without fail. Furthermore, by combining the automatic detection of the aforementioned objects (Figure 19) and patrol settings (Figures 16A and 16B), efficient and comprehensive monitoring becomes possible, thereby improving safety in daily life.

[0180] Furthermore, the monitoring system 10 of this embodiment may also perform sports officiating by installing multiple cameras 150 in a multi-purpose sports field or the like. For example, in recent years, it has become known to use VAR (Video Assistant Referee) to support referee decisions, and the monitoring system 10 of this embodiment may be used as VAR. For example, in a soccer match, the monitoring system 10 may perform offside detection based on the movements of players and the ball. In the method of this embodiment, since the coordinates of the object (for example, coordinate values ​​in a given Cartesian coordinate system) are detected, the positional relationship between the offside line and the attacking player can be clearly determined from the coordinates. The monitoring system 10 may also determine the positional relationship between various lines such as touchlines and the ball using coordinates. In a baseball game, the monitoring system 10 can accurately perform safe / out or fair / foul judgments in baseball by monitoring bases and poles. For example, in the patrol settings described above, the monitoring system 10 may patrol the batter object, fielder object, ball object, and first base object. The monitoring system 10 can track the batter's movement by detecting the batter's movement as a change in the shape of the batter object (Figure 17), thereby allowing the monitoring position to follow the batter's movement. Similarly, the monitoring positions of the fielder object and the ball object also change in accordance with the movement of the fielder and the ball. The first base object has a fixed preset position. By performing such patrols at high frequency, the timing of contact between the ball and the fielder with first base, and between the batter and first base, can be detected, making it possible to determine whether the batter is safe or out. The same safe / out determination is possible even if the fielder directly touches the batter. If the fielder (first baseman) drops the ball, the batter will be safe even if the timing of contact between the ball and the fielder with first base is relatively early, so conditions other than timing are also necessary, but by adjusting to the rules of baseball, appropriate determination is possible. Note that this adjustment may be achieved by a learning process.

[0181] Furthermore, in sports judging, the monitoring system 10 only needs to control the pan, tilt, and zoom of the camera 150 so that it can determine the points that require judgment each time, thus reducing the number of cameras 150. In this case, it is efficient to monitor using the camera 150 that is suitable for judging the points, taking into account the angle and distance, so the aforementioned integrated management is particularly suitable.

[0182] Although this embodiment has been described in detail above, it will be readily apparent to those skilled in the art that many modifications are possible without substantially departing from the novel aspects and effects of this embodiment. Therefore, all such modifications are included within the scope of this disclosure. For example, any term that appears at least once in the specification or drawings together with a broader or synonymous term may be replaced with that different term anywhere in the specification or drawings. Furthermore, all combinations of this embodiment and its modifications are also included within the scope of this disclosure. In addition, the configuration and operation of electronic devices, management terminals, monitoring systems, etc., are not limited to those described in this embodiment, and various modifications are possible. [Explanation of Symbols]

[0183] 10...Monitoring system, 100, 100A, 100B...Electronic equipment, 110, 110A, 110B...Control unit, 120, 120A, 120B...Processing unit, 130, 130A, 130B...Communication unit, 140, 140A, 140B...Storage unit, 150, 150A, 150B...Camera, 160, 160A, 160B...Sensor, 200...Management terminal device, 300...Storage, NE...Network, θ...Horizontal angle, φ...Vertical angle, M...Zoom magnification, R...Distance, RE1-RE6...Area, S, T, U...Distance, α, β, γ...Angle

Claims

1. A processing unit that performs analysis processing based on the captured image taken by the first camera, A control unit that controls the imaging range of the first camera, A communication unit that communicates with external devices, Includes, The aforementioned communications unit is When a predetermined preset position is specified as the position to be imaged by the first camera, a second camera positioned at a different location from the first camera acquires second preset information, which is a parameter for imaged at the preset position. The aforementioned processing unit, Based on the second preset information, the first camera determines first preset information which is a parameter for imaging the preset position. The control unit, An electronic device that controls the first camera based on the first preset information.

2. In the electronic device described in claim 1, The aforementioned processing unit, An electronic device that determines the relative positions of the first camera and the second camera, and determines the first preset information from the second preset information based on the relative positions.

3. In the electronic device according to claim 2, The aforementioned processing unit, An electronic device that determines the relative position by measuring the distance and direction between the second camera and the first camera based on the image captured by the second camera, when the image captured by the second camera includes the first camera.

4. In the electronic device according to claim 2, The aforementioned processing unit, An electronic device that determines the relative position when the image captured by the first camera and the image captured by the second camera include one or more reference objects, by measuring at least one of the distance and direction between the first camera and the reference object based on the image captured by the first camera, and by measuring at least one of the distance and direction between the second camera and the reference object based on the image captured by the second camera.

5. In the electronic device according to any one of claims 1 to 4, The aforementioned processing unit, The second camera determines the main object by performing object detection processing on the second image, which is the image captured at the preset position. By performing the object detection process on the first image, which is the image captured by the first camera at the preset position, an evaluation value is determined for evaluating the imaging state of the main object determined from the second image in the first image. An electronic device that corrects the first preset information for the first camera to image the preset position so that the evaluation value satisfies a given condition.

6. In the electronic device according to claim 5, The aforementioned processing unit, An electronic device that performs pattern matching between the main object detected in the object detection process on the second captured image and the object detected in the object detection process on the first captured image, and determines that the object with a high degree of match is the main object in the first captured image.

7. In the electronic device according to claim 5, The aforementioned processing unit, An electronic device that calculates a higher evaluation value when the preset position of the main object detected by the object detection process for the first captured image is close to the center of the first captured image, compared to when the preset position is far from the center.

8. In the electronic device according to claim 5, The aforementioned processing unit, An electronic device that updates the first preset information based on the modified second preset information when the second preset information is changed.

9. A first electronic device having a first camera, A second electronic device having a second camera positioned differently from the first camera, Includes, The first electronic device is, When a preset position is specified as the position to be imaged by the first camera, the second electronic device acquires second preset information, which is the parameters for the second camera to image the preset position. A monitoring system that performs a process to determine first preset information, which is a parameter for the first camera to image the preset position, based on the second preset information.

10. A control method for electronic equipment having a first camera, The aforementioned electronic device When a preset position is specified as the position to be imaged by the first camera, a second camera positioned at a different location from the first camera acquires second preset information, which is a parameter for imaged at the preset position. Based on the second preset information, the first camera performs a process to determine the first preset information, which is a parameter for capturing the preset position. Control method.

Images