Information processing device, information processing system, and information processing method

Abstract

To notify a camera operator of information regarding the change status of the settings, when the camera operator performs an operation to control an imaging device from a remote location.SOLUTION: An information processing device for setting an external device that controls an imaging device that tracks a subject according to a target position of the subject to be tracked in a captured image accepts a setting of a first target position of the subject to be tracked in an image captured by the imaging device, acquires a second target position of the subject to be tracked, which is held in the external device, from the external device, and controls so as to display information regarding the setting of the target position on a display unit. On the basis of the first target position and the second target position, information regarding the setting of the target position is controlled so as to be displayed in a first aspect or a second aspect different from the first aspect.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device that remotely operates an imaging device. [Background technology]

[0002] In recent years, a method called remote production has become popular in the field of video production. This method allows the user to control imaging equipment from a remote location and shoot and create a video without having to go to the location.

[0003] According to this technique, a camera operator can operate a control controller or the like via a network to remotely configure settings related to imaging for the imaging device and a control device that controls the imaging device.

[0004] Patent Document 1 discloses a method in which a camera operator specifies, via a network, the arrangement (composition) of a subject in an image captured by an imaging device, and controls the imaging device. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2023-48874 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the greater the distance between the controller operated by the camera operator and the imaging device, the longer the delay in network communication becomes, which results in a delay in reflecting the results of changes to imaging settings in the imaging device or a control device that controls the imaging device.

[0007] In such a situation, the camera operator may mistakenly believe that the image capture settings have not been changed, which may lead to excessive operation.

[0008] SUMMARY OF THE PRESENT DISCLOSURE An object of the present invention is to provide an information processing device capable of notifying a camera operator of information relating to the status of changes to settings when the camera operator performs operations to control an imaging device from a remote location. [Means for solving the problem]

[0009] One aspect of the present invention is an information processing device that sets an external device that controls an imaging device that tracks a subject according to a target position of the subject to be tracked in an captured image, and has a receiving means that receives a setting of a first target position of the subject to be tracked in an captured image captured by the imaging device, an acquisition means that acquires from the external device a second target position of the subject to be tracked that is stored in the external device, and a display control means that controls information regarding the setting of the target position to be displayed on a display unit, and the display control means controls the information regarding the setting of the target position to be displayed in a first manner or a second manner different from the first manner based on the first target position and the second target position. Effect of the Invention

[0010] According to the present invention, when a camera operator performs an operation to control an image capture device from a remote location, it is possible to notify the camera operator of information relating to the change status of settings. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating a remote control system according to a first embodiment. [Diagram 2] FIG. 2 is a block diagram showing the configurations of a camera 100, an edge AI device 200, and a PC 300 in the first embodiment. [Diagram 3] 10(a) is a flowchart showing the control operation of the AI ​​device 200 in the first embodiment, and FIG. 10(b) is a flowchart showing the operation of the camera 100 in the first embodiment. [Figure 4] FIG. 11 is a diagram showing an operation flow when an operation is performed on the PC 300 of the first embodiment. [Diagram 5] 1A is a diagram showing an example of a target position mark in the PC 300 of the first embodiment, and FIG. 1B is a diagram showing an example of a target position mark in the PC 300 of the first embodiment. [Figure 6] 1A is a diagram showing an operation flow in the PC 300 of the first embodiment, and FIG. 1B is a diagram showing an operation flow in the PC 300 of the first embodiment. [Figure 7] 6A to 6C are diagrams illustrating changes in the display mode of a target position mark in the first embodiment. [Figure 8] FIG. 11 is a diagram illustrating a remote control system according to a second embodiment. [Figure 9] FIG. 11 is a block diagram showing the configuration of each device in the second embodiment. [Figure 10] 13A is a diagram showing an operation flow in the controller 800 of the second embodiment, and FIG. 13B is a diagram showing an operation flow in the controller 800 of the second embodiment. [Figure 11] 13A is a diagram showing an example of a target position mark in the controller 800 of the second embodiment, FIG. 13B is a diagram showing an example of a target position mark in the controller 800 of the second embodiment, and FIG. 13C is a diagram showing an example of a target position mark in the controller 800 of the second embodiment. [Figure 12] FIG. 13 is a diagram illustrating an operation flow for changing a display mode in response to communication latency in the third embodiment. [Figure 13] FIG. 13 is a diagram illustrating an operation flow for changing a display mode in response to communication latency in the fourth embodiment. [Figure 14] 13A to 13C are diagrams illustrating changes in the display mode of a target position mark in the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] <First embodiment> (Configuration of remote control system) FIG. 1 is a diagram showing an example of the configuration of a remote control system for implementing the present invention.

[0013] The remote control system is composed of a camera 100, an edge AI device 200, a PC 300 (Personal Computer), and a controller 400. The camera 100 is connected to the edge AI device 200 via a LAN (Local Area Network) 500, but is not limited to this. For example, they may be connected by SDI (Serial Digital Interface) or HDMI (High-Definition Multimedia Interface) (registered trademark). Similarly, the controller 400 is connected to the PC 300 (information processing device) via a LAN 600, but is not limited to this. For example, they may be connected by SDI, HDMI, or UVC (Universal Serial Bus Video Class).

[0014] Furthermore, the LAN 500 and the LAN 600 are connected via the Internet 700, and each device is capable of communicating with each other.

[0015] In addition, the camera 100 and the edge AI device 200 are installed in locations that are physically close to each other, and the PC 300 is installed in a remote location away from them.

[0016] Therefore, the communication between the PC 300 (information processing device) and the camera 100 or the edge AI device 200 is in a state where latency, that is, a propagation delay (communication delay) of communication data occurs. Also, the communication between the camera 100 and the edge AI device 200 is in a state where no communication delay occurs.

[0017] In this embodiment, the camera 100 and the edge AI device 200 are described as separate entities, but the camera 100 and the edge AI device 200 may be integrated together. Similarly, the PC 300 and the controller 400 are described as separate entities, but the PC 300 and the controller 400 may be integrated together.

[0018] (Configuration of camera 100) FIG. 2 is a block diagram showing the configurations of the camera 100, edge AI device 200, and PC 300 according to this embodiment.

[0019] The camera 100 has a CPU 101, a RAM 102, a ROM 103, an imaging unit 104, a drive unit 105, an imaging optical system 106, an image processing unit 107, an image output interface (I / F) 108, a network I / F 109, and an internal bus 110 that enables mutual communication.

[0020] The camera 100 can output images captured by the imaging unit 104 to an external device via each network or an image cable (not shown). The camera 100 also has a tracking function that automatically tracks a subject.

[0021] The CPU 101 is a central processing unit that controls the camera 100 .

[0022] RAM 102 is a storage device such as DRAM (Dynamic Random Access Memory) that temporarily stores computer programs executed by CPU 101. RAM 102 also provides a work area into which the OS, various programs, and various data are loaded and which is used when CPU 101 executes processing. It is also used as a working area for the OS and various programs.

[0023] The ROM 103 is a non-volatile storage device such as a flash memory, an HDD, an SSD (Solid State Drive), an SD card, etc. The ROM 103 is used as a permanent storage area for storing the OS, various programs, various data, and other programs for the CPU 101 to control the camera 100, and is also used as a short-term storage area for various data.

[0024] The imaging unit 104 includes a driving unit 105, an imaging optical system 106, and an image processing unit 107, which will be described later. The driving unit 105 drives a pan-tilt (PT) mechanism to change the imaging direction. The driving unit 105 also drives the imaging optical system 106 in the optical axis direction to perform a zoom (Z) operation to change the imaging angle of view.

[0025] The driving unit 105 drives the imaging unit 104 so as to obtain pan, tilt, and zoom values ​​according to a PTZ driving command from the CPU 101 .

[0026] The imaging optical system 106 is a lens that collects light from a subject on an imaging surface of an imaging element, and is composed of, for example, a zoom lens, a focus lens, and a blur correction lens. Furthermore, the zoom value can be changed by driving the imaging optical system 106 in the optical axis direction. The imaging element (not shown) of the imaging unit 104 captures an image of the subject and generates a captured image. The imaging element (not shown) of the imaging unit 104 converts the light from the subject collected by the imaging optical system 106 into an electrical signal for each pixel. Furthermore, an amplifier (not shown) amplifies the electrical signal converted by the imaging element and outputs it to an image processing unit 107 described later. The imaging element is, for example, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor.

[0027] The image processing unit 107 converts the electrical signal amplified by an amplifier (not shown) into a predetermined format, compresses it as necessary, and transfers it to the RAM 102. The image processing unit 107 also performs image quality adjustment when acquiring a captured image, and cropping to cut out only a predetermined area of ​​the image data. These are performed according to instructions from external devices such as the edge AI device 200 and PC 300 via the network I / F 109.

[0028] In this embodiment, the imaging unit 104 and the driving unit 105 are described as being integrated together, but the imaging unit 104 and the driving unit 105 may be separate and detachable, like a camera that is mounted on a camera platform to change the imaging direction. Furthermore, the imaging unit 104 and the imaging optical system 106 are described as being integrated together, but the imaging unit 104 and the imaging optical system 106 may be separate and detachable, like a lens-interchangeable camera.

[0029] The image output I / F 108 is an interface for outputting captured images to the outside, and is configured with an SDI (Serial Digital Interface) or an HDMI (High-Definition Multimedia Interface) (registered trademark). Here, the image output I / F 108 is connected to an image input I / F 208 of the edge AI device 200, which will be described later.

[0030] The network I / F 109 is an I / F for connecting to the above-mentioned LAN 500, and is responsible for communication with external devices such as the edge AI device 200 and the PC 300 via a communication medium such as Ethernet (registered trademark). Here, although it has been described that the camera 100 is remotely controlled via the network I / F 105, it may be controlled via another I / F such as a serial communication I / F (not shown).

[0031] (Configuration of Edge AI Device 200) FIG. 2 is a block diagram showing the configurations of the camera 100, edge AI device 200, and PC 300 according to this embodiment.

[0032] The edge AI device 200 has a CPU 201, a RAM 202, a ROM 203, a detection unit 204, a user input I / F 205, an image output I / F 206, a network I / F 207, an image input I / F 208, and an internal bus 209 that enables mutual communication.

[0033] The CPU 201 is a central processing unit that controls the edge AI device 200.

[0034] The RAM 202 is a storage device such as a DRAM, and temporarily stores computer programs executed by the CPU 201. The RAM 202 also provides a work area into which the OS, various programs, and various data are loaded and which the CPU 201 uses when executing processes. It is also used as a working area for the OS and various programs.

[0035] The ROM 203 is a non-volatile storage device such as a flash memory, an HDD, an SSD (Solid State Drive), an SD card, etc. The ROM 203 is used as a permanent storage area for storing the OS, various programs, various data, and other programs for the CPU 201 to control the edge AI device 200, and is also used as a short-term storage area for various data.

[0036] The detection unit 204 estimates the position and presence or absence of a subject from an image received from an image input I / F 208 (described later) or the like. The detection unit 204 is composed of a calculation device specialized for image processing and detection processing, such as a GPU (Graphics Processing Unit). The detection unit 207 has a trained model created using a machine learning method such as deep learning. A GPU is generally effective for use in the learning processing, but a reconfigurable logic circuit such as an FPGA (Field Programmable Gate Array) may achieve the same function. The processing of the detection unit 204 may also be performed by the CPU 201. In this embodiment, the subject will be described as a person, but is not limited to this. For example, the subject may be an object other than a person, such as a car or a dog.

[0037] A user input I / F 205 (reception means) is an interface for connecting to a mouse, keyboard, or other input device, and is configured by a USB (Universal Serial Bus) or the like.

[0038] The image output I / F 206 is an interface for outputting setting information screens of edge AI devices, etc.

[0039] The network I / F 207 is an I / F for connecting to the above-mentioned LAN 500, and is responsible for communication with external devices such as the camera 100 and the PC 300 via a communication medium such as Ethernet.

[0040] The image input I / F 208 is an interface for receiving captured images from the camera 100 described above, and is configured with SDI or HDMI.

[0041] The edge AI device 200 detects the subject by performing AI detection on the captured image delivered from the camera 100. Furthermore, the edge AI device 200 transmits a control signal via the LAN 500 for tracking the subject, such as changing the imaging direction of the camera 100 based on the result of detecting the subject in the acquired captured image.

[0042] The control function performed by the edge AI device 200 described above may be implemented internally by the camera 100.

[0043] (PC300 configuration) FIG. 2 is a block diagram showing the configurations of the camera 100, edge AI device 200, and PC 300 according to this embodiment.

[0044] The PC 300 includes a CPU 301, a RAM 302, a ROM 303, a network I / F 304, a display unit 305, an operation unit 306, a device I / F 307, and an internal bus 308 that enables intercommunication.

[0045] The CPU 301 performs overall control of the PC 300. It also performs control related to display on a display unit 305 (display control), which will be described later.

[0046] The RAM 302 is a storage device such as a DRAM, and temporarily stores computer programs executed by the CPU 301. The RAM 302 also provides a work area into which the OS, various programs, and various data are loaded and which the CPU 301 uses when executing processes. It is also used as a working area for the OS and various programs.

[0047] The ROM 303 is a non-volatile storage device such as a flash memory, HDD, SSD, SD card, etc. The ROM 303 is used as a permanent storage area for storing the OS, various programs, various data, and other programs for the CPU 301 to control the PC 300, and is also used as a short-term storage area for various data. In this embodiment, the ROM 303 is configured as an SSD, but is not limited to this.

[0048] The network I / F 304 is an I / F for connecting to the aforementioned LAN 600, and is responsible for communication with external devices such as the camera 100 and the edge AI device 200 via a communication medium such as Ethernet. Here, communication refers to sending and receiving control commands to the camera 100 and the PC 300, and receiving captured images from the camera 100.

[0049] The display unit 305 is a display unit for displaying captured images of the camera 100 and a setting screen of the PC 300. Although an example in which the PC 300 has a display unit is shown in this embodiment, the present invention is not limited to this configuration. For example, a configuration in which a display monitor that only displays captured images and a controller are each present may be used.

[0050] The operation unit 306 is an interface for accepting operations from a user to the PC 300, and examples of such interfaces include a button, a dial, a joystick, a touch panel, etc. The operation unit 306 accepts input of instructions for pan / tilt control of the camera 100 based on, for example, a user operation.

[0051] The device I / F 307 is an interface for connecting to an input device, and is configured by a USB (Universal Serial Bus) etc. In this embodiment, it is used as a connection I / F with the controller 400 shown in FIG.

[0052] The PC 300 acquires and displays the captured image output from the camera 100 via the Internet 700. Furthermore, the PC 300 accepts the setting operations related to the subject tracking described above from the user, and transmits control instructions for the edge AI device 200 via the Internet 700 based on the operation contents.

[0053] Specific examples of settings related to subject tracking include settings related to the determination of a tracking target, the angle of view and imaging direction of a captured image, and the position of a subject in a captured image.

[0054] The setting operation can be performed by directly operating the PC 300 or via the controller 400 shown in Fig. 1. That is, the setting related to subject tracking may be performed by the operation unit 306 of the PC 300 or may be performed from the operation unit 401 of the controller 400 described later.

[0055] In this embodiment, a user can input tracking settings including various settings for tracking and capturing an image of a tracking target by operating a controller 400 described later. In addition, the following description will be given assuming that the controller 400 outputs the input tracking settings to the PC 300.

[0056] The method of selecting a person as a tracking target is not limited to a specific method as long as the face of the person to be tracked can be selected from one or more detected faces. For example, the PC 300 or the edge AI device 200 may select a face located at coordinates closest to the target position in the captured image, or a person set in advance may be set as the tracking target. In addition, in this embodiment, a case where the tracking target is a person will be described, but the tracking target is not limited to a person, and the following description can be similarly applied even if the tracking target is something other than a person.

[0057] Next, the controller 400 will be described.

[0058] The controller 400 is a controller equipped with an operation unit 401, and is a user interface capable of operating the camera 100, such as a keyboard or a multi-directional input stick controller. In this embodiment, the user performs settings related to subject tracking using a joystick of the controller 400. The controller 400 is also connected to the device I / F 307, and transmits information according to user operations to the PC 300.

[0059] In this embodiment, the controller 400 and the PC 300 are described as separate entities, but the operation unit 306 of the PC 300 may be responsible for the processing of the controller 400 .

[0060] The remote control system having the above configuration enables the camera 100 to automatically track a subject using the edge AI device 200. Furthermore, the user can operate the PC 300 while checking the captured image displayed on the PC 300 to configure various settings related to subject tracking.

[0061] Here, an outline of the operation of the remote control system (information processing system) has been explained, but the basic operation of each device and the characteristic operation of the present invention will be described in detail later.

[0062] <Basic operation of each device> (Operation of the camera 100 tracking a subject detected by the edge AI device 200) Next, the basic operation of the remote control system described above will be described with reference to a flow chart for each device.

[0063] First, the operation of the camera 100 tracking a subject detected by the edge AI device 200 will be described with reference to FIG.

[0064] Fig. 3(a) shows the operation flow of the edge AI device 200, and Fig. 3(b) shows the operation flow of the camera 100, showing a series of steps in which the edge AI device 200 controls the camera 100 in accordance with the subject position detected from a captured image and settings related to tracking. Here, in this embodiment, the settings related to tracking refer to information on a subject to be tracked selected from one or more subjects and the position (target position) of the subject to be tracked in the captured image.

[0065] First, the operation of the edge AI device 200 will be described with reference to the flow in FIG.

[0066] 3(a) shows a loop process in which the edge AI device 200 receives captured images successively transmitted from the camera 100 via the image input I / F 208, identifies the position of the subject within the captured image, and controls to automatically track the subject to be tracked. This control flow starts when the edge AI device 200 connects to the camera 100 via the LAN 500 and acquires the captured image captured by the camera 100.

[0067] In step S101, the CPU 201 of the edge AI device 200 acquires an image captured by the camera 100 via the image input I / F 208. The CPU 201 writes the acquired captured image to the RAM 202 and proceeds to step S202.

[0068] The captured images are sequentially transmitted from the image output I / F 104 of the camera 100 in accordance with a predetermined frame rate, and the CPU 201 sequentially writes the received captured images to the RAM 202. The captured images may also be received via the network I / F 207 and written to the internal RAM 202.

[0069] In step S102, the detection unit 207 detects the position of the subject in the captured image. The CPU 201 reads the captured image from the RAM 202 and inputs it to the detection unit 207. The detection unit 207 writes output data including the detected subject position to the RAM 202 and proceeds to step S103.

[0070] Specifically, the detection unit 207 receives a captured image as input data, and outputs, as output data, the type of a tracking target such as a person, position information of a subject in the captured image, and a score indicating likelihood.

[0071] In step S103, the edge AI device 200 acquires settings related to tracking from the PC 300 via the network 700 and stores them in the RAM 202. Here, the settings related to tracking refer to, for example, information related to a subject to be tracked from among one or more subjects detected in step S102 and information related to a target position. For simplicity, the subject to be tracked will be referred to as the subject below in the description.

[0072] In step S104, the CPU 201 determines whether the subject's position in the captured image matches the target position based on the tracking settings stored in the RAM 202 and the subject's position in the captured image detected in step S102 (target position). Note that information on the target position is acquired in step S103, but is not limited to this. The target position may be loaded into the RAM 202 by the CPU 102 when the edge AI device 200 is started. For example, the subject's position on the captured image at the time of startup (target position) may be automatically set to the center of the captured image, or the position at the time the edge AI device 200 was previously terminated.

[0073] If the CPU 201 determines that the target position and the position of the detected subject are the same, the process skips steps S105 to S107 and returns to the beginning of the loop process. If the CPU 201 determines that the target position and the position of the detected subject are not the same, the process transitions to step S105.

[0074] In step S105, CPU 201 calculates the difference between the position of the subject detected in step S102 and the target position. Furthermore, it calculates at least one angular velocity (control value) in the pan / tilt (zoom) direction according to the calculated difference so that the position of the subject coincides with the target position. It writes the calculated result to RAM 202 and transitions to step S105.

[0075] A specific method for calculating the angular velocity is to multiply the distance, which is the difference between the coordinate values ​​in the pan direction and the tilt direction, by a predetermined coefficient, and then determine the drive direction depending on whether the calculated value is positive or negative. These techniques are well known, so a detailed description will be omitted.

[0076] The above-described method for tracking a subject using the technique for calculating and controlling the driving direction and speed is one example. Any method for tracking a subject may be used, such as a method for calculating a driving position toward a target position and tracking the subject.

[0077] In step S106, the CPU 201 converts the result calculated in step S105 into a control command in accordance with a protocol previously determined as a method for controlling the camera 100, writes the control command into the RAM 202, and the process proceeds to step S107.

[0078] In step S107, the CPU 201 reads out the control command written to the RAM 202 in step S106. Furthermore, the CPU 201 transmits the control command to the camera 100 via the network I / F 207 and returns to the top of the loop process.

[0079] 3(b), a control flow of the camera 100 when a control command is received from the edge AI device 200 will be described. This control flow starts when the camera 100 connects to the edge AI device 200 via the LAN 500 and further receives a control command from the edge AI device 200.

[0080] In step S 201 , the CPU 101 receives a control command via the network I / F 105 .

[0081] In step S202, CPU 101 is notified that communication data, here a control command, has been received from network I / F 105. Upon receiving this notification, CPU 101 reads a value corresponding to the angular velocity in the pan / tilt direction from the control command written to RAM 102 by network I / F 105. After CPU 101 reads the control command, the process proceeds to step S203. At this time, it is sufficient to read at least one value of the pan direction and tilt direction (zoom direction) as the angular velocity in the pan / tilt (zoom) direction.

[0082] In step S203, CPU 101 calculates drive parameters for panning and tilting in a desired direction at a desired speed based on the values ​​read out in step S202, and proceeds to step S204. Specifically, these are parameters for controlling motors (not shown) for the pan direction and tilt direction included in drive unit 106, and may be converted into drive parameters by referring to a conversion table previously stored in ROM 103 based on the amount of operation included in the received control command.

[0083] In step S204, CPU 101 controls driver 106 based on the drive parameters calculated in step S203, and proceeds to step S205. Driver 106 drives camera 100 based on the parameters, causing camera 100 to change the imaging direction, i.e., perform pan / tilt operations.

[0084] As described above, the control flow shown in Fig. 3(b) enables the camera 100 to change the imaging direction based on the control command acquired from the edge AI device 200. In the present embodiment, the case where the control command related to the angular velocity in the pan direction and the tilt direction is acquired from the edge AI device 200 has been described, but the acquired control command is not limited to this. For example, the zoom value may be changed by the driving unit 106 driving the imaging optical system 105 based on a control command related to the zoom value.

[0085] Next, the operational flow when the PC 300 sets the edge AI device 200 for subject tracking will be described with reference to FIG. 4 and FIGS. 5(a) and (b).

[0086] Fig. 4 shows the operation flow of the PC 300. Here, we will explain the flow of transmitting coordinate information (target position) indicating where in a captured image the subject is to be placed when tracking the subject from the PC 300. This control flow starts upon receiving an operation by the user.

[0087] In step S301, the CPU 301 of the PC 300 detects a joystick operation of the controller 400 operated by the user via the device I / F 307, and acquires the operation direction and operation amount of the joystick from the device I / F 307. After detecting and acquiring the operation by the user, the process proceeds to step S302.

[0088] In this embodiment, the joystick will be described as an analog output type that uses voltages output from variable resistors provided in the pan direction and tilt direction, but is not limited to this. For example, operations may be performed by clicking or touching a GUI (Graphical User Interface) displayed on a display unit using a mouse or keyboard. In the controller 400, an A / D conversion unit (not shown) quantizes the voltage input from the joystick at a predetermined sampling period, and outputs the result to the device I / F 307 as information corresponding to the amount of operation.

[0089] It should be noted that in the A / D conversion section, values ​​in a predetermined range, for example 0 to 1023, are quantized as components in the pan direction / tilt direction according to the amount of operation.

[0090] In step S302, the CPU 301 calculates the amount of movement of the coordinate position (target position) indicating where the subject is to be positioned in the captured image, based on the information on the joystick operation direction and the quantized operation amount acquired in step S301. The calculated result is written to the RAM 302, and the process proceeds to step S303.

[0091] Here, the target position when tracking a subject will be described with reference to FIG.

[0092] FIG. 5(a) shows a captured image displayed on the display unit 305. In FIG. 5(a), a subject indicated by diagonal lines and a target position mark 900, which is the position (target position) of the subject in the captured image, are displayed. Note that the PC 300 periodically receives captured images from the camera 100 via the network I / F 304, and is capable of displaying the images on the display unit 305. The target position mark 900 is a mark indicating a target position when tracking the subject described above. When the subject moves, the edge AI device 200 transmits control information to the camera 100 so that the subject is positioned at the position of the target position mark 900 on the captured image, and the driving unit 106 of the camera 100 is driven. In FIG. 5(a), the target position mark 900 is set in the center of the image.

[0093] In this embodiment, the user can move the target position mark 900 by operating the joystick of the controller 400. Specifically, the PC 300 calculates the amount of movement of the target position mark 900 from the amount and direction of operation of the joystick. The PC 300 transmits the calculation result to the edge AI device 200, which then controls the imaging direction of the camera 100 so that the subject is located at the position of the target position mark 900 even if the target position mark 900 is moved.

[0094] Fig. 5(b) shows a captured image after the target position mark 900 has been moved leftward from the position shown in Fig. 5(a). When the target position mark 900 set in the center of the image in Fig. 5(a) is moved leftward, the imaging direction of the camera 100 is changed so that the subject is captured in a position shifted to the left in the captured image.

[0095] In step S303, the CPU 301 calculates coordinate information of the target position based on the value calculated in step S302, and transmits the coordinate information to the edge AI device 200 via the network I / F 105. After the PC 300 transmits the coordinate information to the edge AI device 200, the process proceeds to step S304. The coordinate information transmitted here is stored in the RAM 202 in the edge AI device 200, and is referenced by the CPU 201 as the target position in step S103 of FIG. 3(a). In this embodiment, the coordinate information of the target position is calculated and transmitted continuously while the user is operating the joystick. This makes it possible to adjust the target position of the subject tracking for the edge AI device 200.

[0096] In this embodiment, the edge AI device 200 and the PC 300 are in a situation where communication latency (communication delay) occurs, so after the above-mentioned coordinate information is transmitted, transmission and reception will occur after a delay time due to the communication delay.

[0097] Therefore, the display on the display unit 305 of the PC 300 is updated immediately in response to the operation of the controller 400. However, after the operation, the setting (target position) regarding the tracking position of the edge AI device 200 is updated, and a lag (delay time) occurs until the imaging direction of the camera 100 starts to change.

[0098] In step S304, the CPU 301 calculates the coordinate values ​​at which to superimpose the target position mark on the captured image based on the values ​​calculated in step S302, and instructs the display unit 305 to superimpose and display the target position mark at those coordinate values ​​on the captured image.

[0099] In step S305, the CPU 301 writes the coordinate values ​​on which the target position mark transmitted to the edge AI device is superimposed to the RAM 302, and updates the target position information on the subject's position.

[0100] As described above, the operational flow of PC 300 and camera 100 makes it possible to change the setting of the tracking position of camera 100 in response to a user operation.

[0101] The processes from step S301 to step S305 are executed periodically while the program is being executed, and may be executed periodically, for example, at intervals of about several tens of milliseconds.

[0102] In this case, the controller 400 responds immediately to the user's joystick operation, and the target position is transmitted and the display is updated in response to the direction and amount of operation.

[0103] <Explanation of how to change the display mode of the target position mark> 5(a) and (b), when the target position is changed, a communication delay occurs between the edge AI device 200 and the PC 300, and the information on the new target position is transmitted and the imaging direction is changed after a delay time due to the communication delay. If such a delay time occurs, the user may mistakenly recognize that the target position has not been changed, which may lead to excessive operation.

[0104] In a situation where the above-mentioned delay time occurs, the display unit 305 displays that the user has sent a change instruction for the changed target position to the edge AI device 200. That is, in this embodiment, the display mode is changed based on the information on the target position stored in the RAM 302 of the PC 300 and the information on the target position stored in the RAM 202 of the edge AI device 200. Specifically, in this embodiment, if the difference between the coordinates of the target position in the RAM 302 of the PC 300 and the coordinates of the target position in the RAM 202 of the edge AI device 200 is greater than a set threshold, the color of the target position mark is changed to red, and if it is equal to or less than the threshold, the color of the target position mark is changed to white. Here, if the difference between the coordinates of the target position in the RAM 302 of the PC 300 and the coordinates of the target position in the RAM 202 of the edge AI device 200 is equal to or less than a predetermined threshold, the target positions are considered to match. In this embodiment, while the user continues to operate the controller 400, the CPU 301 continues to update the coordinates of the target position held by the RAM 302. Similarly, the CPU 301 continues to transmit the updated coordinates of the target position to the edge AI device 200, and when the edge AI device 200 acquires the coordinates of the target position, it updates the coordinates of the target position stored in the RAM 202. Furthermore, when the edge AI device 200 can communicate with the PC 300 via the network 700, it continues to transmit the coordinates of the target position stored in the RAM 202 to the PC 300.

[0105] In this embodiment, the set threshold is described as being set in a circular range centered on the target position stored in RAM 302. For example, a range of 2% of the area of ​​the display unit 305 is set as the threshold, centered on the target position stored in RAM 302. If the coordinates of the target position stored in RAM 202 are within a range of 2% of the area of ​​the display unit 305 from the coordinates of the target position stored in RAM 302 in the display unit 305, the color of the target position mark is changed, but this is not limited to this. For example, the threshold may be set centered on the coordinates of the target position stored in RAM 202. In addition, the threshold does not have to be based on the area of ​​the display unit 305, and the threshold may be changed according to the distance to the actual image capture object by reducing the threshold according to the size of the zoom value.

[0106] Using Figures 6(a) and 6(b), we will explain the operation of changing the display mode depending on the coordinates of the target position stored in RAM 202 when displaying information about the target position (target position mark 900) on display unit 305.

[0107] First, the operation flow of the edge AI device 200 shown in Fig. 6(a) will be described. Fig. 6(a) shows a process flow in which the edge AI device 200 transmits the coordinates of a target position stored in the RAM 202 to the PC 300. It is assumed that the CPU 201 in the edge AI device 200 reads out the target position stored in the RAM 202 in the edge AI device 200 referenced in step S103 in Fig. 3(a) and transmits it to the PC 300 at all times.

[0108] In step S401, the CPU 201 reads out the coordinates of a target position in subject tracking stored in the RAM 202.

[0109] In step S 402 , the CPU 201 transmits the coordinates of the target position that have been read out above to the PC 300 via the network I / F 207 .

[0110] In step S403, the CPU 201 sleeps for a predetermined period of time, and then returns to step S401.

[0111] When the PC 300 and the edge AI device 200 are capable of communicating with each other via the network 700, the above-mentioned process is repeated to sequentially transmit the coordinates of the target position stored in the RAM 202 to the PC 300.

[0112] Next, we will explain the operation flow of PC 300 when changing the display mode of information related to the target position (target position mark 900) displayed on display unit 305 in accordance with the coordinates of the target position stored in RAM 202. Fig. 6(b) shows a process in which PC 300 receives the coordinates of the target position from edge AI device 200 and displays the result of the operation of the target position performed by the user via controller 400 on display unit 305 of PC 300.

[0113] In step S501, the CPU 301 receives, via the network I / F 304, the coordinates of the target position transmitted by the edge AI device 200 in step S402 of FIG. 6B, and transitions to step S502.

[0114] In step S502, the CPU 301 reads out the coordinates of the target position written to the RAM 302 in step S305 of Fig. 4, and the process proceeds to step S503. That is, the CPU 301 reads out the coordinates of the target position held in the RAM 302.

[0115] In step S503, CPU 301 compares the coordinates of the target position received in step S501 and held in RAM 202 with the coordinates of the target position held in RAM 302 in step S502. If they match, the process proceeds to step S504, where CPU 301 instructs display unit 305 to display the target position mark displayed on display unit 305 in white. If they do not match, the process proceeds to step S505, where CPU 301 instructs display unit 305 to display the target position mark displayed on display unit 305 in red.

[0116] Here, the state of the image displayed on the display unit 305 when the display color of the target position mark is changed will be described with reference to FIGS. 7(a) to 7(e).

[0117] Figures 7(a) to (e) show the contents displayed on the display unit 305 in chronological order, and will be described assuming that the user is changing the target position by operating the controller 400 from Figure 7(b) to (c).

[0118] Fig. 7(a) shows the display content when a subject is captured at the position of the target position mark 900, and Fig. 7(b) shows the display content when a user operates the target position mark 900. Fig. 7(c) shows the display content when the imaging direction of the camera is being changed in accordance with the movement of the target position mark 900, and Fig. 7(d) shows the display content when the change operation of the target position mark 900 is completed. Fig. 7(e) shows the display content when the change of the imaging direction is completed and the subject is captured again at the position of the target position mark.

[0119] 7(a) and 7(e) show the display contents when the target position information matches in step S504, and the target position mark 900a is displayed in white.

[0120] Figures 7(b), 7(c), and 7(d) show the display content when the target position information does not match in step S505, and the target position mark 900b is displayed in red (diagonal line pattern in the drawings).

[0121] While there is no match, the PC 300 can understand that the edge AI device 200 is unable to acquire the target position due to communication latency.

[0122] The aim of the above-mentioned processing is to take advantage of the difference in target position caused by this communication latency, and by changing the color of the target position mark during that time, to help the user recognize that communication latency is occurring, thereby preventing the user from realizing that the results of their operation are not responding.

[0123] On the other hand, when communication latency is sufficiently low or when the target position is changed slightly, the time during which the coordinates of the target position do not match is shortened.

[0124] In that situation, if a process is performed to change the color of the target position mark 900 while there is no match, the color of the target position mark 900 may change alternately quickly, appearing to flash.

[0125] Therefore, in this embodiment, the color of the target position mark is changed when the difference between the coordinates of the target position held by RAM302 and the coordinates of the target position held by RAM202 is greater than a predetermined threshold value, but this is not limited to the above. For example, the color of the target position mark may be changed only when the mismatch continues for a predetermined period of time. Also, the color of the target position mark may be changed only when the coordinates of the target position held by RAM302 and the coordinates of the target position held by RAM202 completely match without setting a predetermined threshold value.

[0126] In step S506, the CPU 301 sleeps for a predetermined time and then returns to the beginning of the process.

[0127] As described above, by changing the color of the target position mark depending on the comparison result of the coordinates of the target position, the user can visually recognize that communication latency is occurring.

[0128] In this embodiment, an example is described in which communication latency occurs. However, the present invention is also effective when applied to an operation system in which latency is high due to processing other than communication, such as edge device detection processing and image signal processing.

[0129] In this embodiment, an example of changing the display mode of the target position mark is shown in which the color is changed, but it is also possible to use other methods of notifying the user by changing the shape, filling pattern, etc.

[0130] Furthermore, other than the target position mark, it is also possible to inform the user that communication latency is occurring by communication latency time information, a warning message, a voice, a vibration of the controller, etc. In the present embodiment, an example has been described in which the edge AI device 200 controls the camera 100 to track the subject and transmits the target position information to the PC 300, but this is not limited thereto. For example, the camera 100 may be configured to have the functions of these edge AI devices 200 inside.

[0131] <Second embodiment> In the first embodiment, an example in which the display mode of the target position mark is changed by the edge AI device 200 and the PC 300 has been described.

[0132] In this embodiment, unlike the first embodiment, a controller 800 equipped with a joystick and a monitor is used instead of the PC 300. Furthermore, in this embodiment, an example will be described in which the display mode of a rectangular frame (subject frame) indicating a subject detected by an AI is changed instead of a target position mark. In the first embodiment, the joystick operation was accepted via the device I / F 307 of the PC 300, but in this embodiment, the controller 800 directly reads the operation amount of the joystick equipped in the controller 800 and changes the target position.

[0133] In this embodiment, the same processes as those in the first embodiment are denoted by the same reference numerals, and the description thereof will be omitted.

[0134] FIG. 8 is a configuration diagram of a remote control system according to the second embodiment.

[0135] In the first embodiment, the PC 300 is connected to the LAN 600 , but in this embodiment, the controller 800 is connected to the LAN 600 .

[0136] Next, the internal configuration of each device in the second embodiment will be described with reference to Fig. 9. Fig. 9 is a block diagram showing the configuration of each device in the second embodiment.

[0137] The internal configurations of the camera 100 and the edge AI device 200 are omitted since they are the same as those in the first embodiment.

[0138] The controller 800 includes a CPU 301, a RAM 302, a ROM 303, a network I / F 304, a display unit 305, an operation unit 801, and an internal bus 308 that enables intercommunication.

[0139] The operation unit 801 is, for example, a member such as a joystick provided on the controller body. A specific example of the joystick is an analog output type that uses voltages output from variable resistors provided in the pan direction and tilt direction, as in the first embodiment.

[0140] Next, the operation of the controller 800 in this embodiment will be described with reference to FIG.

[0141] FIG. 10(a) shows the operation flow of the controller 800 when the user operates the joystick of the controller 800.

[0142] In the first embodiment, the operation amount of the joystick is received via the device I / F 307 of the PC 300, but in this embodiment, the operation amount of a joystick provided in the controller 800 is directly read by the controller 800. The method for doing this will be described below.

[0143] In step S601, the controller 800 accepts an operation performed using a joystick.

[0144] Specifically, an A / D converter (not shown) quantizes the voltage input from the joystick to the operation unit 806 at a predetermined sampling period, and writes the result and information on the operation direction to the internal RAM 302.

[0145] The CPU 301 reads out the quantized value corresponding to the amount of operation and the information on the operation direction written into the RAM 302 .

[0146] In step S602, the CPU 301 calculates the amount of movement of the coordinate position (target position) where the subject is continuously captured when tracking the subject, based on the joystick operation direction and operation amount read from the RAM 302 in step S601.

[0147] As a result, similarly to the first embodiment, in response to the joystick operation, target position information can be transmitted to the edge AI device 200. Moreover, since steps S303, 304, and 305 are similar to those in the first embodiment, a description thereof will be omitted.

[0148] Next, with reference to Fig. 10(b), an example of changing the mode of tracking position information for notifying the user of the communication latency status on the display unit 305 of the controller 800 will be described. Note that steps S501, 502, 503, and 506 are the same as those in the first embodiment, and therefore the description will be omitted.

[0149] In step S701, CPU 301 instructs display unit 305 to display the subject frame displayed on display unit 305 in white.

[0150] In step S702, CPU 301 instructs display unit 305 to display the subject frame displayed on display unit 305 in red.

[0151] The controller 800 periodically receives information about a subject frame that shows a detected subject as a rectangle from the edge AI device 200 via the network I / F 304. Furthermore, the controller 800 can superimpose the subject frame on the captured image that is periodically received from the camera 100 and display it on the display unit 305.

[0152] Here, the state of the image displayed on display unit 305 when the display color of the subject frame is changed will be described with reference to Figures 11(a) to (c). The description will be given assuming that Figures 11(a) to (c) are arranged in chronological order.

[0153] In FIG. 11(a), the user is not operating the joystick of the controller 800, and the target position mark 900a is in the center of the image.

[0154] In FIG. 11(a), 1000a is a subject frame, and the subject frame received from the above-mentioned edge AI device 200 is superimposed on the captured image.

[0155] Moreover, since the coordinates of the target position received in step S501 match the coordinates of the target position read from the RAM 302 in step S502, the subject frame 1000a is displayed in white.

[0156] Next, FIG. 11(b) shows a situation in which 900b is a target position mark, and the target position mark 900b is being moved as a result of the user operating the joystick of the controller 800.

[0157] In Figure 11 (b), 1000b is the subject frame, and when the target position mark 900b moves, it is determined in step S503 that the coordinates of the target position do not match, and the process transitions to step S702, and the subject frame is displayed in red.

[0158] 11(c), target position mark 900b is located at the same coordinates as in FIG. 11(b), indicating a state in which the control command has been reflected after communication latency. Since the coordinates of the target position received in step S501 match those of the target position read from RAM 302 in step S502, subject frame 1000a is displayed in white.

[0159] As described above, by changing the color of the subject frame according to the comparison result of the target position information, the user can visually recognize that communication latency is occurring. Also, in the second embodiment, by displaying not only the target position mark but also the detection frame that detects the subject, the visibility of the subject to be tracked is improved, which is useful when switching the tracking target.

[0160] In this embodiment, an example has been shown in which the color of the subject frame is changed, but it is also possible to apply a method in which the shape of the frame or the fill pattern is changed, for example.

[0161] <Third embodiment> Communication latency may change dynamically depending on the communication environment. Taking this into consideration, it is also useful to switch between enabling and disabling the display mode change control shown in the first or second embodiment depending on the communication latency situation.

[0162] 12 is a diagram showing a flow in which the display mode change control is switched between enabled and disabled in accordance with the result of measuring the communication latency by the controller 800. Note that the internal configurations of the remote control system and each device are similar to those in the second embodiment.

[0163] In step S801, the CPU 301 of the controller 800 transmits a message to the edge AI device 200 via the network I / F 304 using a ping that allows the response time to be ascertained.

[0164] In step S802, the CPU 301 receives a response from the edge AI device 200 via the network I / F 304.

[0165] In step S803, the CPU 301 calculates an RTT (Round Trip Time) corresponding to a round trip time of communication, that is, a communication time, and in step S804, it is determined whether the RTT is equal to or longer than a predetermined time. In this embodiment, the CPU 301 calculates the communication time, but this is not limited to this. For example, the communication time acquired from the edge AI device 200 may be used.

[0166] When the CPU 301 determines in step S804 that the RTT is greater than the predetermined time, the CPU 301 transitions to step S805 and enables the display mode change control. Specifically, the CPU 301 dynamically changes the process so that the process in step S503 and thereafter in FIG. 10B is executed.

[0167] When the CPU 301 determines in step S804 that the RTT is equal to or shorter than the predetermined time, the CPU 301 transitions to step S806 and disables the display mode change control. Specifically, the CPU 301 dynamically changes the process to skip the process from step S503 onward in FIG. 10B.

[0168] As described above, an example has been shown in which the communication latency between the controller 800 and the edge AI device 200 is measured, and whether or not the display mode is changed is switched depending on the result, that is, an example has been shown in which the display mode is switched between an operation mode that takes communication time into consideration and an operation mode that does not take communication time into consideration. By adding such processing, it becomes possible to automatically suppress the change of display mode in situations where it is not necessary to change the display mode, for example, in situations where no communication latency occurs.

[0169] Furthermore, in the present embodiment, an example has been shown in which the display mode change control is dynamically switched between enabled and disabled depending on the RTT, but this switching may be set by the user, for example.

[0170] Specifically, one example of such a method is that the edge AI device 200 has a web server function, and the device settings operating on the server function can be set to a mode that takes communication latency into consideration.

[0171] The above-mentioned change in the state of the target position mark is performed only when the user checks a checkbox in the main body settings relating to a mode in which communication latency is taken into consideration. In this way, it is possible to configure the device to switch in accordance with a user instruction.

[0172] <Fourth embodiment> In the first embodiment, an example has been described in which the display mode of the target position mark is changed based on the target positions stored in the RAM 202 of the edge AI device 200 and the RAM 302 of the PC 300.

[0173] In this embodiment, an example will be described in which a second target position mark based on a target position stored in the RAM 202 of the edge AI device 200 is displayed in addition to a first target position mark based on a target position stored in the RAM 302 of the PC 300. In the fourth embodiment, when the first target position mark and the second target position mark match, the display mode of the target position mark is changed. Specifically, the first target position mark 1300 stored in the RAM 302 and the second target position mark 1400 stored in the RAM 202 are superimposed on the captured image captured by the camera 100.

[0174] This embodiment will be described with reference to the flow chart of FIG. 13 and FIGS. 14(a) to (d) regarding the state of the image displayed on the display unit 305 when the display color of the target position mark is changed.

[0175] In the flow of FIG. 13, the processes from step S301 to step S305 are the same as those in the above-mentioned flow, and therefore the description thereof will be omitted.

[0176] In step S901, the PC 300 acquires the coordinates of the target position stored in the RAM 202 from the edge AI device 200.

[0177] In step S902, the CPU 301 controls the display unit 305 to display a mark indicating the target position based on the coordinates of the target position acquired in step S901.

[0178] 14(a) to (d) show the contents displayed on the display unit 305 in chronological order, and from Fig. 14(b) to (c) it will be explained that the user is changing the target position by operating the controller 400. Furthermore, the first target position mark 1300 indicates the target position changed by the user, i.e., the target position stored in the RAM 302, and the second target position mark 1400 indicates the target position stored in the RAM 202.

[0179] FIG. 14(a) shows the display contents when the positions of the first target position mark 1300 and the second target position mark 1400 match. In this embodiment, when the first target position mark 1300 and the second target position mark 1400 match as in FIG. 14(a), the second target position mark 1400 is not displayed, but this is not limited to the above. For example, the first target position mark may be displayed so as to overlap the second target position mark 1400. FIG. 14(b) shows the display contents when the user operates the first target position mark 1300. FIG. 14(c) shows the display contents when the imaging direction of the camera is changed according to the second target position mark 1400 stored in the RAM 202, and the user completes the movement operation of the first target position mark 1300. FIG. 14(d) shows the display contents when the position of the first target position mark 1300 and the position of the second target position mark 1400 match due to the change in the imaging direction of the camera.

[0180] Here, the target position mark 1300a and the target position mark 1300b are marks of different colors, and when the position coordinates in the image of the first target position mark 1300 do not match the position coordinates in the image of the second target position mark 1400, they are displayed in red.

[0181] As described above, an example has been shown in which two target position marks are displayed when the target positions do not match, based on the target positions stored in the RAM 202 of the edge AI device 200 and the RAM 302 of the PC 300. By adding such processing, it becomes possible for the user to visually check on the display unit 305 how much communication latency is occurring.

[0182] In addition, in this embodiment, an example in which the target positions match has been shown, but this is not limiting. For example, the color may be changed when the target positions approximately match below a set threshold value as in the first embodiment. Furthermore, control may be performed so that the target position mark 1300 or the target position mark 1400 is not displayed when the target positions approximately match. [Explanation of symbols]

[0183] 100 Cameras 200 Edge AI Devices 300 PC 301 CPU 302 RAM 303 ROM 400 Controller 305 Display section

Claims

1. An information processing device that performs settings of an external device that controls an imaging device that tracks a subject according to a target position of the subject to be tracked in a captured image, a receiving unit for receiving a setting of a first target position of a subject to be tracked in an image captured by the imaging device; an acquisition means for acquiring a second target position of a subject to be tracked, which is held in the external device, from the external device; a display control means for controlling information regarding the setting of the target position to be displayed on a display unit, The information processing device characterized in that the display control means controls information regarding the setting of the target position to be displayed in a first manner or in a second manner different from the first manner based on the first target position and the second target position.

2. When a distance between the first target position and the second target position is greater than a predetermined threshold, the display control means controls to display information related to setting of the target position in the first manner, When a distance to the first target position or the second target position is equal to or less than the threshold value, the display control means controls to display information related to setting of the target position in the second manner.

2. The information processing apparatus according to claim 1,

3. 3. The information processing apparatus according to claim 2, wherein the information relating to the setting of the target position is a mark indicating the first target position.

4. 3. The information processing apparatus according to claim 2, wherein the information regarding the setting of the target position is a color of a frame surrounding the subject.

5. 2. The information processing apparatus according to claim 1, wherein the display control means superimposes information relating to the setting of the target position on the captured image.

6. The accepting means accepts an operation by a user, the display control means controls a display such that a mark indicating the first target position moves in the captured image based on an operation by the user.

4. The information processing apparatus according to claim 3.

7. a calculation unit that calculates a movement amount of a mark indicating the first target position in response to an operation amount of the operation by the user; the display control means moves and displays a mark indicating the first target position in the captured image based on the amount of movement calculated by the calculation means.

4. The information processing apparatus according to claim 3.

8. A calculation means for calculating a communication time between the external device and the information processing device; The information processing device according to claim 1, characterized in that the display control means determines whether to display information regarding the target position in a first manner or in a second manner different from the first manner depending on the time measured by the measurement means.

9. when the time measured by the measuring means is longer than a predetermined time and when the distance between the first target position and the second target position is longer than a predetermined threshold, the display control means controls to display information about the first target position in the first manner; a first operation mode in which, when the time measured by the measuring means is longer than a predetermined time and the distance to the first target position or the second target position is equal to or less than the threshold value, the display control means controls to display information about the first target position in a second manner; When the time measured by the measuring means is equal to or shorter than a predetermined time, a second operation mode in which information regarding the first target position is not displayed in the first manner; 2. The information processing apparatus according to claim 1, further comprising:

10. the display control means superimposes information indicating the first target position and the second target position on an image captured by an imaging device; The information processing device according to claim 1, characterized in that the display control means controls the display of information indicating the first target position in a second manner when the distance between the first target position and the second target position is equal to or less than the threshold value.

11. 2. The information processing apparatus according to claim 1, wherein the first embodiment and the second embodiment have different shapes.

12. 2. The information processing apparatus according to claim 1, wherein the first aspect and the second aspect are different in color.

13. An information processing system including an external device that controls an imaging device having a subject tracking function and an information processing device that sets the external device, The external device is a setting means for acquiring information regarding a target position of a subject from the information processing device and setting the information regarding the target position; a transmitting means for transmitting information regarding the target position set in the setting means; having The information processing device includes: a receiving unit for receiving a setting of a first target position of a subject to be tracked in an image captured by the imaging device; an acquisition means for acquiring a second target position of the subject to be tracked from the external device; a display control means for controlling the display of information indicating the target position on a display unit, The information processing system is characterized in that the display control means controls information regarding the target position to be displayed in a first manner or in a second manner different from the first manner based on the first target position and the second target position.

14. An information processing method of an information processing device that sets an external device that controls an imaging device having a subject tracking function, comprising: a receiving step of receiving a setting of a first target position in an image captured by the external device of a subject to be tracked; an acquisition step of acquiring a second target position from the external device; a display control step of controlling a display unit to display information indicating the target position, An information processing method characterized in that, in the display control process, based on the first target position and the second target position, information indicating the target position is controlled to be displayed in a first aspect or a second aspect different from the first aspect.

15. A program for causing a computer to execute the information processing method according to claim 14.

Images