Camera control program, camera control method, and information processing device

Abstract

This camera control program causes a computer to execute a process in which an image captured by a camera is acquired, and when panning of the camera is to be controlled in accordance with the position of an object to be captured in the captured image, panning control for moving the camera at a speed stipulated in advance is executed in a case in which the panning coordinates of the camera are within a prescribed angle range.

Description

Camera control program, camera control method, and information processing device

[0001] The present invention relates to a camera control program, a camera control method, and an information processing device.

[0002] A PTZ camera is a type of camera that has pan, tilt, and zoom functions. For example, when a moving object is being photographed, the PTZ camera may be combined with object detection using artificial intelligence (AI) to perform pan control based on the position of the object in the captured image.

[0003] JP 2013-162502 A

[0004] However, when AI-based object detection is used in conjunction with the AI, as the moving speed of the object increases, the relative position between the object's position in the captured image and the target position for capturing the object in the captured image changes frequently in relation to the object's moving direction. As a result, the panning direction changes frequently, making it difficult to perform smooth panning control.

[0005] In one aspect, the present invention aims to provide a camera control program, a camera control method, and an information processing device that can realize smooth pan control.

[0006] One aspect of the camera control program causes a computer to execute a process in which, when acquiring an image captured by a camera and controlling the pan of the camera according to the position of the object being photographed in the captured image, if the coordinates of the pan of the camera are within a predetermined angle range, pan control is executed to move at a predetermined speed.

[0007] According to one embodiment, smooth pan control can be achieved.

[0008] FIG. 1 is a diagram showing an example of the configuration of a camera control system. FIG. 2 is a diagram (1) showing one aspect of a problem-solving approach. FIG. 3 is a diagram (2) showing one aspect of a problem-solving approach. FIG. 4 is a block diagram showing an example of the functional configuration of an information processing device. FIG. 5 is a schematic diagram showing an example of section division. FIG. 6 is a diagram showing an example of section setting data. FIG. 7 is a schematic diagram showing an example of object detection. FIG. 8 is a schematic diagram showing an example of control parameter calculation. FIG. 9 is a flowchart showing the procedure of camera control processing. FIG. 10 is a schematic diagram showing an application example of section division. FIG. 11 is a diagram showing an example of hardware configuration.

[0009] Hereinafter, embodiments of a camera control program, a camera control method, and an information processing device according to the present disclosure will be described with reference to the accompanying drawings. Note that these embodiments merely illustrate examples and aspects, and the structures, actions, functions, properties, characteristics, methods, uses, and the like according to the present disclosure are not limited by these examples.

[0010] <First Embodiment> <System Configuration> Fig. 1 is a diagram showing an example of the configuration of a camera control system. Fig. 1 shows, as an example, a camera control system 1 that provides a camera control function for causing a camera 30 to capture an image of a moving object moving on a course C.

[0011] An example of a usage scenario for such a camera control system 1 is when a horse racing broadcast is being filmed, but this is just one example, and usage scenarios to which the camera control system 1 can be applied are not limited to specific races, specific courses, or specific subjects.

[0012] For example, the camera control system 1 may be applied to other sports besides horse racing, such as track and field events, bicycle racing, boat racing, car racing, and triathlons. Furthermore, the camera control system 1 does not necessarily have to be applied to a course determined for a sport. For example, the "course" may be a course determined for training, such as hill training or training tests, or may be a course formed in a facility or area to be monitored.

[0013] In this way, the camera control system 1 can capture any subject, such as a racehorse, a person, a vehicle, an aircraft, a robot, or other moving object, as a subject of photography depending on the usage scene.

[0014] 1, the camera control system 1 may include an information processing device 10 and a camera 30. The information processing device 10 and the camera 30 may be connected to each other so as to be able to communicate with each other, whether by wire or wirelessly.

[0015] The information processing device 10 is an example of a computer that provides the camera control function. For example, the information processing device 10 can be implemented by a computer on which a camera control program that implements the camera control function is installed. Note that the camera control function can be implemented as a software as a service (SaaS) application, thereby providing it as a cloud service, or as a web server that provides the camera control function on-premise.

[0016] The camera 30 is an example of an imaging device that captures images. For example, the camera 30 may be realized by a PTZ camera that can remotely control pan (horizontal rotation around the Y axis), tilt (vertical rotation around the X axis), and optical zoom. Such a camera 30 is installed so that it can capture all or part of the course C. This allows for capturing images of moving objects such as racehorses moving along the course C. Note that while a PTZ camera that combines three elements for controlling the camera's field of view has been exemplified as the camera 30 here, a camera that can control at least one or more elements may also be used.

[0017] In one aspect, images captured by the camera 30 may be transmitted frame by frame between the information processing device 10 and the camera 30. For example, communication from the information processing device 10 to the camera 30 may be realized by any interface standard, such as HDMI (registered trademark) (High-Definition Multimedia Interface). However, this is not limited to this, and communication may be realized by any communication technology, such as an intranet, the Internet, or a low-power wireless communication standard for the Internet of Things (IoT).

[0018] In another aspect, commands for remotely operating the camera 30 may be transmitted between the information processing device 10 and the camera 30. For example, communication from the information processing device 10 to the camera 30 may be realized by a protocol for transmitting camera control commands, such as VISCA (Video System Control Architecture).

[0019] <One Aspect of the Problem> As explained in the Background Art section above, the following problem occurs when AI-based object detection is coordinated. That is, as the moving speed of the object to be photographed increases, the relative position between the position of the object to be photographed in the photographed image and the target position for photographing the object to be photographed in the photographed image changes more frequently in front of or behind the moving direction of the object to be photographed. As a result, the pan movement direction changes frequently, making it difficult to perform smooth pan control.

[0020] <One aspect of the problem-solving approach (1)> Therefore, the camera control function according to this embodiment allows pan control only in a predefined direction when controlling the pan of the camera 30 in accordance with the position of the subject in the image captured by the camera 30.

[0021] Fig. 2 is a diagram (1) illustrating one aspect of the problem-solving approach. For example, Fig. 2 shows the results of performing object detection on a photographed image 20 captured by a camera 30. Fig. 2 also shows an example in which control parameters are determined to control the pan direction of the camera 30, with the position of the leading racehorse in the photographed image 20 as the target.

[0022] 2, as a result of executing object detection using AI, three bounding boxes BB1 to BB3 are detected from the captured image 20. In this case, of the three bounding boxes BB1 to BB3, the bounding box BB1 corresponding to the leading edge of the "left direction" that corresponds to the traveling direction of the captured object is targeted.

[0023] For example, if the target position for photographing the subject is the center of the screen, a specific position of the bounding box BB1, for example, the two-dot chain line corresponding to the left edge, is used as the target position, and control parameters such as the relative pan movement amount and absolute pan position are set to align the target position, for example, the dashed line corresponding to the center of the screen.

[0024] Here, when the pan control parameters are set based solely on the results of object detection by AI, as in the above-described conventional technology, the pan movement direction is set to the "right direction," which is the opposite direction to the traveling direction of the subject in the example shown in Figure 2. That is, in the example shown in Figure 2, when the aiming position is located ahead of the target position in the traveling direction of the subject, the pan movement direction of the camera 30 is set to the "right direction," which is the opposite direction to the traveling direction of the subject. As a result, the pan movement direction switches between the traveling direction and the opposite direction.

[0025] On the other hand, the camera control function according to this embodiment allows pan control only in a predetermined direction, for example, "leftward" in the direction of travel of the subject. In other words, if the aiming position is located ahead of the target position in the direction of travel of the subject, setting the pan movement direction of the camera 30 to "rightward," which is the opposite direction to the direction of travel of the subject, is prohibited. This prevents the pan movement direction from switching between the travel direction and the opposite direction.

[0026] Therefore, the camera control function according to this embodiment makes it possible to realize smooth pan control of the camera 30. Furthermore, the camera control function according to this embodiment realizes automatic shooting that allows smooth pan control of the camera 30, thereby eliminating the need for personnel for shooting. Therefore, the location where the camera 30 is installed is not restricted by factors such as the ease of access for the photographer or the space required for the photographer to take photos. As a result, it is possible to install the camera 30 in a location where the camera angle is favorable or where occlusion by competition or spectator facilities is unlikely to occur.

[0027] <One aspect of the problem-solving approach (2)> Furthermore, when controlling the pan of the camera 30 in accordance with the position of the object to be photographed in the image captured by the camera 30, the camera control function according to this embodiment executes pan control to move at a predefined speed if the pan coordinates of the camera 30 are within a predetermined angle range.

[0028] Fig. 3 is a diagram (2) showing one aspect of the problem-solving approach. Fig. 3 shows a clockwise course C as an example of a course. As shown in Fig. 3, when the pan angle of the camera 30 is within the angle ranges A1, A2, and A3, pan control is permitted to move in the pan direction at a predefined speed V1, V2, or V3.

[0029] For example, in pan angle range A1, the racehorse, which is the subject of the image capture, is closer to camera 30 than in other angle ranges. Therefore, the speed at which the racehorse moves in the captured image is also relatively faster. As the moving speed of the subject of the image capture increases in this way, the frequency at which the relative position between the position of the subject of the image capture in the captured image and the target position at which to capture the subject of the image in the captured image switches between being forward or backward in the direction of travel of the subject of the image capture increases. The moving speed of a racehorse is largely fixed, and by setting a fixed pan speed V1 in such pan angle range A1 as a control parameter, it is possible to prevent the pan movement direction from switching between the forward direction and the opposite direction.

[0030] Furthermore, in pan angle ranges A2 and A3, the racehorse, which is the subject of the filming, is farther from camera 30 than in other angle ranges. As the subject of the filming moves farther from camera 30, the accuracy of object detection by the AI ​​decreases or becomes unstable. As a result, the bounding box corresponding to the leading horse is detected or lost, causing the aim that determines the pan control parameters to switch frequently. By setting a fixed pan speed V2 or speed V3 as the control parameter in pan angle ranges A2 and A3, it is possible to prevent the pan movement direction from switching between the forward direction and the opposite direction.

[0031] Therefore, the camera control function according to this embodiment makes it possible to realize smooth pan control of the camera 30. Furthermore, the camera control function according to this embodiment realizes automatic shooting that allows smooth pan control of the camera 30, thereby eliminating the need for personnel for shooting. Therefore, the location where the camera 30 is installed is not restricted by factors such as the ease of access for the photographer or the space required for the photographer to take photos. As a result, it is possible to install the camera 30 in a location where the camera angle is favorable or where occlusion by competition or spectator facilities is unlikely to occur.

[0032] <Configuration of information processing device> Next, a functional configuration of the information processing device 10 according to this embodiment will be described. Fig. 4 is a block diagram showing an example of the functional configuration of the information processing device 10. Fig. 4 is a block diagram showing an example of the functional configuration of the information processing device 10. Fig. 4 schematically shows blocks related to the camera control function of the information processing device 10.

[0033] As shown in Fig. 4, the information processing device 10 has a communication control unit 11, a storage unit 13, and a control unit 15. Note that Fig. 4 only shows an excerpt of functional units related to the above-described camera control function, and the information processing device 10 may also be provided with functional units other than those shown, such as an input unit and a display unit.

[0034] The communication control unit 11 is a functional unit that controls communication with other devices such as the camera 30. For example, the communication control unit 11 can be realized by a network interface card. In one aspect, the communication control unit 11 can accept captured images from the camera 30. In another aspect, the communication control unit 11 can output to the camera 30 commands for controlling the camera 30, control parameters specified by the commands, and the like.

[0035] The storage unit 13 is a functional unit that stores various types of data. For example, the storage unit 13 is realized by internal, external, or auxiliary storage of the information processing device 10. In one embodiment, the storage unit 13 stores information such as section setting data 13A. Note that the storage unit 13 may also store information other than the section setting data 13A, such as photographed images, programs in which races are held, and information on the jockeys and racehorses participating in each race.

[0036] The section setting data 13A is data in which definitions related to camera control are set for each section. The term "section" here refers to the segments into which the course C is divided. FIG. 5 is a schematic diagram showing an example of section division. FIG. 5 shows a clockwise course C as an example of a course. As shown in FIG. 5, the course C is divided into eight sections, sections 1 to 8, which allow the movement direction of the subject to be photographed within a section to be narrowed down to only one direction. For example, in each of sections 1 to 8, the movement direction of a racehorse can be limited to left, left, left, right, right, down, down, and left.

[0037] For example, the section setting data 13A may be data in which, for each section, settings such as a definition of the condition for transitioning from the section to the next section and a definition of the direction in which movement is permitted or prohibited in the section are associated.

[0038] 6 is a diagram showing an example of the section setting data 13A. For example, the PTZ coordinates are expressed in the PTZ coordinate system described below. For example, the pan coordinate system is expressed in a range from the coordinate "-8704" corresponding to 170 degrees left to the coordinate "8704" corresponding to 170 degrees right. The tilt coordinate system is expressed in a range from the coordinate "-1024" corresponding to 20 degrees down to the coordinate "4608" corresponding to 90 degrees up. The zoom coordinate system is expressed in a range from the coordinate "0" corresponding to the minimum zoom out to the coordinate "16384" corresponding to the maximum zoom in.

[0039] Furthermore, the PTZ control parameters can be specified within the PTZ speed range described below. For example, the pan speed can be specified within the range of -24 to 24, with a positive sign representing "left" and a negative sign representing "right." The tilt speed can be specified within the range of -24 to 24, with a positive sign representing "up" and a negative sign representing "down." The zoom speed can be specified within the range of -7 to 7, with a positive sign representing "zoom in" and a negative sign representing "zoom out."

[0040] FIG. 6 shows an excerpt of the settings for section 1 and section 2 out of the eight sections shown in FIG.

[0041] For example, in the case of section 1, the pan coordinate condition for transitioning to section 2, the next section after section 1, is that the pan coordinate must move up to 4100. Also, the tilt coordinate condition is that the tilt coordinate must move up to -19. Furthermore, the zoom coordinate condition is that the zoom coordinate must move up to 16384.

[0042] Furthermore, in section 1, it is defined that the aim used to calculate the pan control parameters is set based on the left direction, while the aim used to calculate the tilt and zoom control parameters is not set. Furthermore, in section 1, it is defined that the traveling direction of the object to be photographed is the left direction.

[0043] Furthermore, Section 1 defines that operations within the following speed ranges are permitted. For example, pan speed is permitted within a range from 0 to 3. This is equivalent to defining the left direction as the permitted direction of pan rotation. Tilt speed is permitted within a range from -3 to 0. This is equivalent to defining the downward direction as the permitted direction of tilt rotation. Zoom speed is permitted within a range from -2 to 0. This is equivalent to defining zoom out as the permitted direction of zoom. Furthermore, it is defined that zooming out up to 14921 and zooming in up to 16384 are permitted. Note that although FIG. 6 shows an example in which a speed range for permitted operations is defined, a speed range for prohibited operations may also be defined.

[0044] Furthermore, Section 1 defines the center of the screen as the target used for calculating the pan control parameters. Furthermore, Section 1 defines that bounding boxes obtained by object detection using AI with a confidence level of 0.2 or higher are used.

[0045] Next, in the example of section 2, the pan coordinate condition for transitioning to section 3, the section next to section 2, is that the pan coordinate moves to 3462. Also, the tilt coordinate condition is that the tilt coordinate moves to -87. Furthermore, the zoom coordinate condition is that the zoom coordinate moves to 14921.

[0046] Furthermore, section 2 defines that the aim used to calculate the pan control parameters is set based on the left direction. Furthermore, it defines that the aim used to calculate the tilt control parameters is set based on the downward direction. Furthermore, it defines that the aim used to calculate the zoom control parameters is set based on zoom-out. Furthermore, section 2 defines that the traveling direction of the object to be photographed is leftward.

[0047] Furthermore, Section 2 defines that operations within the following speed ranges are permitted. For example, pan speed is permitted in the range of 1 to 7. This has the aspect of being equivalent to defining the left direction as the permitted direction of pan rotation. Tilt speed is permitted in the range of -3 to -1. This has the aspect of being equivalent to defining the downward direction as the permitted direction of tilt rotation. Zoom speed is permitted in the range of -3 to -1. This has the aspect of being equivalent to defining zoom out as the permitted direction of zoom. Furthermore, it is defined that zooming out up to 8931 and zooming in up to 14921 are permitted.

[0048] Furthermore, Section 2 defines that the target used in calculating the pan control parameters is the center of the screen in the height direction and 50 pixels to the left of the center of the screen in the width direction. Furthermore, Section 2 defines that bounding boxes obtained by object detection using AI with a confidence level of 0.3 or higher should be used.

[0049] While Figure 6 only illustrates definitions for Sections 1 and 2, it goes without saying that definitions can also be set for the other sections. For example, since Sections 3, 5, and 6 correspond to the angle ranges A1 to A3 shown in Figure 3, the PTZ speed range can be set to a narrower range than the speed ranges defined for the other sections, or a single value, such as a value corresponding to V1 to V3, can be set. This allows the PTZ speed to be fixed at a substantially constant value. Furthermore, for Section 8, which corresponds to the goal of Course C, the movement of the PTZ can be stopped by defining a coordinate end point for each function of changing the PTZ's field of view instead of a transition condition to the next section.

[0050] Returning to the explanation of Fig. 4, the control unit 15 is a functional unit that performs overall control of the information processing device 10. For example, the control unit 15 can be realized by a hardware processor. As shown in Fig. 4, the control unit 15 has an image acquisition unit 15A, an object detection unit 15B, a control amount calculation unit 15C, a camera control unit 15E, and an output control unit 15F. Note that the control unit 15 may also be realized by hardwired logic or the like.

[0051] The image acquisition unit 15A is a processing unit that acquires a captured image. In one embodiment, the image acquisition unit 15A can acquire a captured image frame by frame from the camera 30. Note that, although an example of acquiring a captured image transmitted from the camera 30 has been given here, the captured image may also be acquired via an external device, such as a file server or a storage medium, such as removable media.

[0052] The object detection unit 15B is a processing unit that detects objects from captured images. Such object detection may be realized by a machine learning model that executes an object detection task that receives an image as input and outputs the position and category of an object in the image. For example, the machine learning model may be YOLO (You Only Look Once), Faster-RCNN (Regions with Convolutional Neural Network), DETR (End-to-End Object Detection with Transformers), or the like.

[0053] To train such a machine learning model, a training dataset can be used, which is training data that matches images with bounding boxes that indicate the location of objects in the images and ground truth labels that include classes that indicate the object categories.

[0054] In one aspect, in the training phase, a machine learning model can be trained using a machine learning algorithm, such as deep learning, with the images as explanatory variables and the correct labels as objective variables, thereby obtaining a trained machine learning model.

[0055] In another aspect, in the prediction phase, the captured image is input to a trained machine learning model, which then outputs a bounding box, a class, and a confidence score.

[0056] Hereinafter, we may refer to a trained machine learning model that performs object detection tasks as an "object detection model."

[0057] In one embodiment, when a differential image of a new frame is acquired by the image acquisition unit 15A, the object detection unit 15B inputs the acquired image into the object detection model to obtain the object detection result (prediction result) output by the object detection model.

[0058] FIG. 7 is a schematic diagram showing an example of object detection. As shown in FIG. 7, the object detection unit 15B inputs a captured image 20 to an object detection model. The object detection model to which the captured image 20 has been input outputs an object detection result including a bounding box, a class, and a confidence level for each object detected from the captured image 20. For example, in the example shown in FIG. 7, a "class name" such as horse and a "confidence level" ranging from 0 to 1 are predicted for each of the bounding boxes BB1 to BB3 detected from the captured image 20.

[0059] The control amount calculation unit 15C is a processing unit that calculates control parameters for controlling the PTZ of the camera 30. Below, an example in which the speed of the PTZ is specified will be given as an example of the control parameters, but it goes without saying that the control parameters are not limited to specifying a relative movement from the current position, and the coordinates of the PTZ can also be specified as absolute values.

[0060] In one embodiment, the control amount calculation unit 15C executes the function of changing the field of view of the camera 30, for example, the following processing for each PTZ. That is, the control amount calculation unit 15C calculates control parameters in accordance with the settings of the aim and target defined for the section corresponding to the progress of the race by the object being photographed, in the section setting data 13A stored in the storage unit 13. For example, the progress of the race can be managed by updating the loop counter for the section each time a photographed image of a new frame is acquired, based on the monitoring result of whether the coordinates of the PTZ observed in the new frame satisfy the transition condition to the next section.

[0061] Fig. 8 is a schematic diagram showing an example of calculating control parameters. For example, Fig. 8 shows an example of calculating PTZ control parameters from object detection results for the captured image 20 shown in Fig. 7. Fig. 8 also shows an example of calculating PTZ control parameters in accordance with the aim setting defined in section 2 of the section included in the section setting data 13A. Fig. 8 also shows an example of calculating pan speed, one of the control parameters.

[0062] As shown in Figure 8, as a result of executing the object detection model, three bounding boxes BB1 to BB3 are detected from the captured image 20. In this case, of the three bounding boxes BB1 to BB3, the bounding box BB1 corresponding to the beginning of the "left direction" that corresponds to the traveling direction of the captured object is set as the target. At this time, the bounding box to be set as the target may be determined after narrowing down the bounding boxes to those with a certainty factor equal to or greater than the threshold value of "0.3" in accordance with the definition of certainty factor in Section 2.

[0063] Here, in section 2, as explained using Figure 6, the target used to calculate the pan control parameters is defined as position T1, which is the center of the screen in the height direction and 50 pixels to the left of the center of the screen in the width direction.

[0064] In this case, a specific position of the bounding box BB1, for example, a two-dot chain line corresponding to the left edge, is used as the target, and a pan speed is calculated to align with a target position, for example, position T1 where the dashed line and the one-dot chain line intersect. For example, a larger pan speed is calculated as the distance D1 from the target to the target increases, while a smaller pan speed is calculated as the distance D1 from the target to the target decreases. Furthermore, when the target position is located ahead of the target position in the direction of travel of the subject, a negative sign, i.e., a rightward direction, is set for the pan speed, whereas when the target position is located behind the target position in the direction of travel of the subject, a positive sign, i.e., a leftward direction, is set for the pan speed. Note that while an example of calculating the pan speed has been described here, tilt speed and zoom speed can also be calculated in a similar manner.

[0065] The camera control unit 15E is a processing unit that controls the PTZ of the camera 30. In one embodiment, the camera control unit 15E performs the following processing for each field-of-view change function of the camera 30, for example, for each PTZ. That is, the camera control unit 15E determines whether the control parameter calculated by the control amount calculation unit 15C is an allowed action defined for a section in the section setting data 13A that corresponds to the progress of the race by the subject being photographed. If the control parameter is not an allowed action, the camera control unit 15E specifies, as a control parameter for a remote control command, a boundary value corresponding to the sign of the control parameter calculated by the control amount calculation unit 15C, among the boundary values ​​of the range of control parameters defined as allowed actions for the section. On the other hand, if the control parameter is an allowed action, the camera control unit 15E specifies the control parameter calculated by the control amount calculation unit 15C as a control parameter for a remote control command for the camera 30. Then, the camera control unit 15E transmits to the camera 30 a remote control command for the PTZ, for which a control parameter is specified for each field-of-view change function of the camera 30.

[0066] The output control unit 15F is a processing unit that controls the output of the captured images acquired by the image acquisition unit 15A. In one embodiment, the output control unit 15F can output the captured images acquired by the image acquisition unit 15A frame by frame. For example, captured images captured in chronological order may be output as a video. Such output may be implemented in any form, such as television broadcasting, internet distribution, or live broadcasting using a social networking service (SNS).

[0067] <Processing Flow> Next, a processing flow of the information processing device 10 according to this embodiment will be described. Fig. 9 is a flowchart showing the procedure of the camera control processing. This processing can be started when the above-mentioned camera control function is activated.

[0068] As shown in FIG. 9, the camera control unit 15E moves the camera 30 to an initial position (step S101), and the camera control unit 15E sets an initial value, for example, "1", to a loop counter m for the section (step S102).

[0069] Thereafter, the image acquisition unit 15A acquires a captured image of a new frame (step S103). Then, the object detection unit 15B inputs the captured image acquired in step S103 into an object detection model, thereby acquiring an object detection result (prediction result) output by the object detection model (step S104).

[0070] Next, the control amount calculation unit 15C identifies the bounding box of the aim defined in the section corresponding to the value of the loop counter m from among the bounding boxes obtained as the object detection result in step S104 (step S105).

[0071] Thereafter, loop processing 1 is executed in which the processing from step S106 to step S108 below is repeated a number of times corresponding to the number N of field-of-view changing functions of camera 30. Note that, although an example in which the processing from step S106 to step S108 below is repeated is given here, the processing from step S106 to step S108 below may also be executed in parallel.

[0072] That is, the control amount calculation unit 15C calculates a control parameter related to the function of the nth field of view change based on the position of the bounding box identified as the aim in step S105 and the position of the target defined in the section corresponding to the value of the loop counter m in the section setting data 13A (step S106).

[0073] Then, the camera control unit 15E determines whether or not the control parameter calculated in step S106 is a permitted operation defined in the section corresponding to the value of the loop counter m in the section setting data 13A (step S107).

[0074] Here, if the control parameter related to the nth field of view change function is not an allowed operation (No in step S107), the camera control unit 15E designates the boundary value of the range of control parameters defined as the allowed operation of the section corresponding to the value of the loop counter m, which corresponds to the sign of the control parameter calculated in step S106, as the control parameter of the remote control command (step S108).

[0075] On the other hand, if the control parameters are not the permitted operation (Yes in step S107), the process of step S108 is skipped. In this case, the control parameters calculated in step S106 are specified as the control parameters of the command for remotely controlling the camera 30.

[0076] By repeating this loop process 1, control parameters are specified for each function of changing the field of view of the camera 30.

[0077] Thereafter, the camera control unit 15E transmits to the camera 30 a remote control command in which control parameters are specified for each function of changing the field of view of the camera 30 (step S109).

[0078] Next, the camera control unit 15E determines whether the coordinates of the PTZ satisfy the transition condition to the next section defined in the section corresponding to the value of the loop counter m in the section setting data 13A (step S110).

[0079] At this time, if the coordinates of the PTZ satisfy the transition condition to the next section (Yes in step S110), the camera control unit 15E increments the loop counter m (step S111). After that, the camera control unit 15E acquires the setting of the section corresponding to the value of the loop counter m from the section setting data 13A (step S112), and proceeds to the processing of step S103.

[0080] If the coordinates of the PTZ do not satisfy the transition condition to the next section (No in step S110), the processes in steps S111 and S112 are not executed, and the process proceeds to step S103.

[0081] <One Aspect of Effect> In one aspect, the information processing device 10 according to this embodiment allows pan control only in a predetermined direction when controlling the pan of the camera 30 in accordance with the position of a subject in an image captured by the camera 30. This prevents the pan movement direction from switching between the traveling direction and the opposite direction. Therefore, the camera control function according to this embodiment makes it possible to achieve smooth pan control of the camera 30.

[0082] In another aspect, when controlling the pan of the camera 30 according to the position of the subject in the image captured by the camera 30, the information processing device 10 according to this embodiment allows pan control to move at a predetermined speed if the pan coordinates of the camera 30 are within a predetermined angle range. This makes it possible to prevent the pan movement direction from switching between the forward direction and the opposite direction. Therefore, the camera control function according to this embodiment makes it possible to achieve smooth pan control of the camera 30.

[0083] Furthermore, the camera control function according to this embodiment realizes automatic shooting that allows smooth pan control of the camera 30, eliminating the need for personnel for shooting. Therefore, the location where the camera 30 is installed is not restricted by factors such as ease of access for the photographer or the space required for the photographer to take photos. As a result, it is possible to install the camera 30 in a location where the camera angle is favorable or where occlusion by the facilities for the competition or spectators is unlikely to occur.

[0084] Second Embodiment Although the embodiments of the present disclosure have been described above, various applications are possible, and further, the present disclosure may be implemented in various different forms other than the first embodiment described above.

[0085] <Numeric Values, etc.> The matters described in the first embodiment, such as the types of control parameters and the number of cameras 30, are merely examples and may be changed. In addition, the order of processing in the flowcharts described in the embodiments may also be changed within a consistent range.

[0086] <Application Example> For example, in the above-described first embodiment, one camera 30 is used to film course C, but two or more cameras 30 can also be used to film a horse race broadcast. FIG. 10 is a schematic diagram showing an application example of section division. FIG. 10 shows an example in which two cameras 30, camera 30A and camera 30B, are used to film a horse race broadcast. In this case, as shown in FIG. 10, camera 30A can be assigned to film sections 1 to 4 and sections 7 and 8 out of sections 1 to 8, and camera 30B can be assigned to film sections 5 and 6. Allocating sections in this manner allows for close-up filming of racehorses throughout course C and also reduces occlusion caused by racing and spectator facilities, such as large display boards.

[0087] <System> The information including the processing procedures, control procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, one or more of the functional units among the image acquisition unit 15A, the object detection unit 15B, the control amount calculation unit 15C, the camera control unit 15E, and the output control unit 15F may be configured as separate devices.

[0088] Furthermore, the components of each device shown in the figure are functional concepts and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown. In other words, all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Note that each configuration may also be a physical configuration.

[0089] Furthermore, each processing function performed by each device can be realized, in whole or in part, by a CPU (Central Processing Unit) and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0090] <Hardware> Next, an example of the hardware configuration of the computer described in the above embodiment will be described. Fig. 11 is a diagram showing an example of the hardware configuration. As shown in Fig. 11, the information processing device 10 has a communication device 10a, a storage device 10b, a memory 10c, and a processor 10d. Note that the components shown in Fig. 11 may be connected to each other via a bus or the like.

[0091] The communication device 10a is a network interface card, etc. The storage device 10b is a storage device such as a hard disk drive (HDD) or a solid state drive (SSD). For example, the storage device 10b stores a program, a database, etc. that realizes the functions shown in FIG. 4.

[0092] The processor 10d reads out a program that executes the same processing as the processing unit shown in FIG. 4 from the storage device 10b or the like and loads it into the memory 10c, thereby operating a process that executes the functions described in FIG.

[0093] Such a process realizes the same functions as the processing units of the information processing device 10. For example, the processor 10d reads out a program having the same functions as the image acquisition unit 15A, the object detection unit 15B, the control amount calculation unit 15C, the camera control unit 15E, the output control unit 15F, etc. from the storage device 10b, etc. Then, the processor 10d executes a process that executes the same processing as the image acquisition unit 15A, the object detection unit 15B, the control amount calculation unit 15C, the camera control unit 15E, the output control unit 15F, etc.

[0094] In this way, the information processing device 10 operates as an information processing device that executes a calculation method by reading and executing a program. The information processing device 10 can also realize functions similar to those of the above-described embodiment by reading the program from a recording medium using a medium reading device and executing the read program. Note that the program in these other embodiments is not limited to being executed by the information processing device 10. For example, the present invention can also be applied in the same way to cases where another computer or server executes the program, or where these execute the program in cooperation with each other.

[0095] The above program can be distributed via a network such as the Internet. The above program can also be recorded on any recording medium and executed by a computer by reading it from the recording medium. For example, the recording medium can be a hard disk, a flexible disk (FD), a CD-ROM, a magneto-optical disk (MO), a digital versatile disk (DVD), or the like.

[0096] REFERENCE SIGNS LIST 1 camera control system 10 information processing device 11 communication control unit 13 storage unit 13A section setting data 15 control unit 15A image acquisition unit 15B object detection unit 15C control amount calculation unit 15E camera control unit 15F output control unit 30 camera

Claims

1. When acquiring a captured image of a camera and controlling the pan of the camera according to the position of a subject in the captured image, when the coordinates of the pan of the camera are within a predetermined angular range, execute a pan control that moves at a previously defined speed, and cause a computer to execute the process. A camera control program characterized by this.

2. The process to be executed includes, among setting data in which the angular range of the pan corresponding to a specific section among sections into which the course along which the subject moves is divided and the speed of the pan of the camera are defined, when the coordinates of the pan of the camera correspond to the angular range of the pan corresponding to the specific section, execute a pan control that moves at the speed associated with the specific section. The camera control program according to claim 1, characterized by this.

3. The process to be executed includes, when controlling the tilt of the camera according to the position of a subject in the captured image, when the coordinates of the tilt of the camera are within a predetermined angular range, execute a tilt control that moves at a previously defined speed. The camera control program according to claim 1, characterized by this.

4. The process to be executed includes, when controlling the zoom of the camera according to the position of a subject in the captured image, when the coordinates of the zoom of the camera are within a predetermined angular range, execute a zoom control that moves at a previously defined speed. The camera control program according to claim 1, characterized by this.

5. When acquiring a captured image of a camera and controlling the pan of the camera according to the position of a subject in the captured image, when the coordinates of the pan of the camera are within a predetermined angular range, execute a pan control that moves at a previously defined speed, and a camera control method characterized by a computer executing the process.

6. An information processing apparatus having a control unit that executes a process of acquiring a captured image of a camera and, when controlling the pan of the camera according to the position of a subject in the captured image, when the coordinates of the pan of the camera are within a predetermined angular range, executes a pan control that moves at a previously defined speed.

Images