Camera control program, camera control method, and information processing device

Abstract

To allow smooth panning control.SOLUTION: A camera control program causes a computer to execute the processing of: acquiring an image captured by a camera; and when controlling panning of the camera in accordance with a position of an imaged object in the captured image, executing panning control for moving the camera at a speed stipulated in advance in a case in which panning coordinates of the camera are within a prescribed angle range.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a camera control program, a camera control method, and an information processing apparatus.

Background Art

[0002] As a type of camera, a PTZ camera having pan, tilt, and zoom functions is known. For example, when the object to be photographed is a moving object, pan control may be performed based on the position of the object to be photographed in the photographed image by linking the PTZ camera with object detection by AI (Artificial Intelligence).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, when linked with object detection by AI, as the moving speed of the object to be photographed increases, the frequency with which the relative position between the position of the object to be photographed in the photographed image and the target position for photographing the object to be photographed in the photographed image switches back and forth in front of and behind the traveling direction of the object to be photographed increases. As a result, since the moving direction of the pan frequently switches, there is an aspect that it is difficult to perform smooth pan control.

[0005] In one aspect, an object of the present invention is to provide a camera control program, a camera control method, and an information processing apparatus capable of realizing smooth pan control.

Means for Solving the Problems

[0006] A camera control program for one side acquires a captured image of a camera, and when controlling the pan of the camera according to the position of a subject in the captured image, when the coordinates of the pan of the camera are within a predetermined angle range, it causes a computer to execute a pan control that moves at a previously defined speed.

Advantages of the Invention

[0007] According to one embodiment, smooth pan control can be realized.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Modes for Carrying Out the Invention

[0009] Hereinafter, examples of a camera control program, a camera control method, and an information processing apparatus according to the present disclosure will be described with reference to the accompanying drawings. Note that this example merely shows one example or aspect, and the structure, operation, function, nature, characteristics, method, use, etc. according to the present disclosure are not limited by such an exemplification.

[0010] <Example 1> <System Configuration> FIG. 1 is a diagram showing a configuration example of a camera control system. In FIG. 1, as an example, a camera control system 1 that provides a camera control function for causing a moving body moving on a course C to be photographed by a camera 30 is illustrated.

[0011] As a usage scene of such a camera control system 1, a case where horse racing relay is photographed is taken as an example, but this is just an example, and the usage scenes to which the camera control system 1 can be applied are not limited to specific sports, specific courses, or specific subjects.

[0012] For example, the camera control system 1 may be applied to other sports such as track and field, motorcycle racing, motorboat racing, car racing, and triathlon in addition to horse racing. Further, the camera control system 1 does not necessarily have to be applied to a course defined in a sport. For example, the "course" may be a course defined in training, such as hill training or a training test, or a course formed in a facility or area to be monitored.

[0013] In this way, the camera control system 1 can use any subject, for example, a moving body such as a racehorse, a person, a vehicle, an aircraft, or a robot, as a photographing object according to the usage scene.

[0014] As shown in FIG. 1, the camera control system 1 may include an information processing apparatus 10 and a camera 30. These information processing apparatus 10 and camera 30 may be communicably connected to each other regardless of whether they are wired or wireless.

[0015] The information processing apparatus 10 is an example of a computer that provides the above-described camera control function. For example, the information processing apparatus 10 can be realized by a computer installed with a camera control program that realizes the above-described camera control function. Note that the above-described camera control function can be provided as a cloud service by being realized as a SaaS (Software as a Service) type application, or can be realized as a web server that provides the above-described camera control function on-premises without being hindered.

[0016] The camera 30 is an example of an imaging device that captures images. For example, the camera 30 may be realized by a PTZ camera capable of remotely controlling panning (horizontal rotation around the Y axis), tilting (vertical rotation around the X axis), and optical zoom. Such a camera 30 is installed so as to be able to photograph all or part of the course C. Thereby, a composition in which a moving object such as a racehorse moves on the course C can be photographed. Here, as an example of the camera 30, a PTZ camera having three elements for controlling the field of view of the camera is illustrated, but a camera capable of controlling at least one or more elements may be used.

[0017] On one side, images captured by the camera 30 in frame units may be transmitted between the information processing apparatus 10 and the camera 30. For example, communication from the information processing apparatus 10 to the camera 30 may be realized by an arbitrary interface standard, such as a standard such as HDMI (registered trademark) (High-Definition Multimedia Interface). Note that the present invention is not limited to this, and it is not hindered from being realized by an arbitrary communication technology such as an intranet, the Internet, or a power-saving wireless communication standard for IoT (Internet of Things).

[0018] As another aspect, a command for remotely operating the camera 30 may be transmitted between the information processing apparatus 10 and the camera 30. For example, communication from the information processing apparatus 10 to the camera 30 may be realized by a protocol for transmitting a command for camera control, such as VISCA (Video System Control Architecture).

[0019] <One aspect of the problem> As described in the background art section above, there are the following problems when cooperating object detection by AI. That is, as the moving speed of the object to be photographed increases, the frequency with which the relative position between the position of the object to be photographed on the photographed image and the position targeted for photographing the object to be photographed on the photographed image switches before and after the traveling direction of the object to be photographed increases. As a result, since the pan direction frequently switches, there is an aspect in which it is difficult to perform smooth pan control.

[0020] <One aspect of the problem-solving approach (1)> Therefore, when the camera control function according to the present embodiment controls the pan of the camera 30 according to the position of the object to be photographed in the photographed image of the camera 30, pan control is permitted only in a direction defined in advance.

[0021] FIG. 2 is a diagram (1) showing one aspect of the problem-solving approach. For example, FIG. 2 shows the execution result of object detection for the photographed image 20 photographed by the camera 30. Further, FIG. 2 shows an example in which control parameters for controlling the pan direction of the camera 30 are determined with the position of the leading racehorse on the photographed image 20 as the aiming point.

[0022] As shown in FIG. 2, as an execution result of object detection by AI, three bounding boxes BB1 to BB3 are detected from the photographed image 20. In this case, among the three bounding boxes BB1 to BB3, the bounding box BB1 corresponding to the "left direction" corresponding to the leading edge in the traveling direction of the object to be photographed is used as the aiming point.

[0023] For example, take the case where the target position for photographing the object to be photographed is the center of the screen. In this case, control parameters are set to align with the target position, for example, the dashed line corresponding to the center of the screen, using a specific position of the bounding box BB1, for example, the two-dot chain line corresponding to the left side as a sighting reference. Examples of the control parameters include the relative movement amount of panning and the specification of the absolute position of panning.

[0024] Here, when the control parameter of panning is set only based on the execution result of object detection by AI as in the above prior art, the moving direction of panning is set to the "right direction", which is opposite to the moving direction of the object to be photographed, in the case shown in FIG. 2. That is, in the example shown in FIG. 2, when the aiming position is ahead of the target position in the moving direction of the object to be photographed, the moving direction of panning of the camera 30 is set to the "right direction", which is opposite to the moving direction of the object to be photographed. For this reason, the moving direction of panning switches between the moving direction and its opposite direction.

[0025] On the other hand, in the camera control function according to the present embodiment, panning control is permitted only in a predefined direction, for example, the "left direction" which is the moving direction of the object to be photographed. In other words, when the aiming position is ahead of the target position in the moving direction of the object to be photographed, it is prohibited to set the moving direction of panning of the camera 30 to the "right direction", which is opposite to the moving direction of the object to be photographed. For this reason, it is possible to suppress the switching of the moving direction of panning between the moving direction and its opposite direction.

[0026] Therefore, according to the camera control function according to the present embodiment, it is possible to realize smooth panning control of the camera 30. Furthermore, as a result of realizing automatic photographing capable of smooth panning control of the camera 30 by the camera control function according to the present embodiment, the need for personnel for photographing can be eliminated. Therefore, the location where the camera 30 is installed is not restricted by factors such as the ease of access for the photographer and the space for the photographer to take pictures. As a result, it is also possible to install the camera 30 at a location where the camera angle is in good condition or where occlusion due to competition or viewing facilities is less likely to occur.

[0027] <One aspect of the problem-solving approach (2)> In addition, when the camera control function according to this embodiment controls the pan of the camera 30 according to the position of the object to be photographed in the photographed image of the camera 30, when the coordinates of the pan of the camera 30 are within a predetermined angle range, pan control that moves at a predefined speed is executed.

[0028] FIG. 3 is a diagram (2) showing one aspect of the problem-solving approach. In FIG. 3, as an example of a course, a clockwise course C is shown. As shown in FIG. 3, when the pan angle of the camera 30 is within the angle ranges A1, A2, and A3, pan control that moves in the pan direction at predefined speeds V1, V2, or V3 is permitted.

[0029] For example, in the pan angle range A1, the racehorse that is the object to be photographed approaches the camera 30 compared to other angle ranges. Therefore, the moving speed of the racehorse moving on the photographed image also becomes relatively fast. As the moving speed of the object to be photographed increases in this way, the frequency at which the relative position between the position of the object to be photographed on the photographed image and the target position for photographing the object to be photographed on the photographed image switches before and after the traveling direction of the object to be photographed increases. The moving speed of the racehorse is generally determined, and by setting the fixed pan speed V1 in such a pan angle range A1 as a control parameter, it is possible to suppress the pan moving direction from switching between the traveling direction and the opposite direction.

[0030] In addition, in the pan angle range A2 and the pan angle range A3, the racehorse that is the object to be photographed moves away from the camera 30 compared to other angle ranges. As the object to be photographed moves away from the camera 30 in this way, the accuracy of object detection by AI decreases or becomes unstable. As a result, the bounding box corresponding to the leading horse is detected or lost, so the aiming for determining the pan control parameter frequently switches. By setting the fixed pan speed V2 or speed V3 in these pan angle ranges A2 and A3 as control parameters, it is possible to suppress the pan moving direction from switching between the traveling direction and the opposite direction.

[0031] Therefore, according to the camera control function according to this embodiment, smooth pan control of the camera 30 can be realized. Furthermore, as a result of realizing automatic shooting capable of smooth pan control of the camera 30 by the camera control function according to this embodiment, the need for personnel for shooting can be eliminated. Therefore, the location where the camera 30 is installed is not restricted by factors such as the ease of access for the photographer and the space for the photographer to shoot. As a result, it is also possible to install the camera 30 at a location where the camera angle is favorable or at a location where occlusion due to competition or viewing facilities is unlikely to occur.

[0032] <Configuration of the information processing device> Next, the functional configuration of the information processing device 10 according to this embodiment will be described. FIG. 4 is a block diagram showing an example of the functional configuration of the information processing device 10. FIG. 4 is a block diagram showing an example of the functional configuration of the information processing device 10. In FIG. 4, the blocks related to the camera control function of the information processing device 10 are schematized.

[0033] As shown in FIG. 4, the information processing device 10 includes a communication control unit 11, a storage unit 13, and a control unit 15. Note that in FIG. 4, only the functional units related to the above camera control function are shown in an extract, and it is also possible that the information processing device 10 is provided with functional units other than those shown, such as an input unit and a display unit.

[0034] The communication control unit 11 is a functional unit that controls communication with other devices such as the camera 30. For example, the communication control unit 11 can be realized by a network interface card. On one hand, the communication control unit 11 can receive a captured image from the camera 30. On the other hand, the communication control unit 11 can output a command for controlling the camera 30, control parameters specified by the command, etc. to the camera 30.

[0035] The storage unit 13 is a functional unit that stores various types of data. For example, the storage unit 13 is realized by an internal, external, or auxiliary storage of the information processing apparatus 10. As one aspect, the storage unit 13 stores information such as the section setting data 13A. Note that the storage unit 13 may store other information other than the section setting data 13A, such as a photographed image, a program in which a race is held, information on a jockey and a racehorse participating in each race, and the like.

[0036] The section setting data 13A is data in which definitions regarding camera control are set for each section. The "section" referred to here refers to a segment into which the course C is divided. FIG. 5 is a schematic diagram showing an example of the division of sections. In FIG. 5, as an example of a course, a right-turn course C is shown. As shown in FIG. 5, the course C is divided into eight sections 1 to 8 that can narrow down the moving direction of the object to be photographed within the section to only one direction. For example, in each of sections 1 to 8, the moving direction of the racehorse can be limited to the left direction, the left direction, the left direction, the right direction, the right direction, the downward direction, the downward direction, and the left direction.

[0037] For example, the section setting data 13A may be data in which settings such as definitions of conditions for transitioning to the next section of the section and definitions of directions in which operations are permitted or prohibited in the section are associated for each section.

[0038] FIG. 6 is a diagram showing an example of the section setting data 13A. For example, the PTZ coordinates are expressed in the PTZ coordinate system described below. For example, the pan coordinate system is expressed in the range from the coordinate "-8704" corresponding to 170 degrees to the left to the coordinate "8704" corresponding to 170 degrees to the right. Also, the tilt coordinate system is expressed in the range from the coordinate "-1024" corresponding to 20 degrees downward to the coordinate "4608" corresponding to 90 degrees upward. Further, the zoom coordinate system is expressed in the range from the coordinate "0" corresponding to the minimum zoom out to the coordinate "16384" corresponding to the maximum zoom in.

[0039] Furthermore, the PTZ control parameters can be specified within the speed range of the PTZ described below. For example, the pan speed can be specified in the range from -24 to 24, with the positive sign representing "left direction" and the negative sign representing "right direction". Also, the tilt speed can be specified in the range from -24 to 24, with the positive sign representing "upward direction" and the negative sign representing "downward direction". Furthermore, the zoom speed can be specified in the range from -7 to 7, with the positive sign representing "zoom in" and the negative sign representing "zoom out".

[0040] Here, FIG. 6 excerptedly shows the settings related to Section 1 and Section 2 among the eight sections shown in FIG. 5.

[0041] For example, in the case of Section 1, as the condition for the pan coordinates to transition to Section 2, which is the next section of Section 1, it is defined that the pan coordinates move up to 4100. Also, as the condition for the tilt coordinates, it is defined that the tilt coordinates move up to -19. Furthermore, as the condition for the zoom coordinates, it is defined that the zoom coordinates move up to 16384.

[0042] Furthermore, in Section 1, while the aiming used for calculating the control parameter of the pan is set based on the left direction, it is defined that no aiming is set for calculating the control parameters of the tilt and zoom. Furthermore, in Section 1, it is defined that the traveling direction of the object to be photographed is the left direction.

[0043] Furthermore, in Section 1, it is defined to permit operations within the speed ranges described below. For example, the pan speed is permitted in the range from 0 to 3. This has an aspect equivalent to defining the left direction as the direction permitting the rotation of the pan. Also, the tilt speed is permitted in the range from -3 to 0. This has an aspect equivalent to defining the downward direction as the direction permitting the rotation of the tilt. Furthermore, the zoom speed is permitted in the range from -2 to 0. This has an aspect equivalent to defining the zoom-out as the direction permitting the zoom. Additionally, for zoom, it is defined to permit reduction up to 14921 and enlargement up to 16384. Note that in FIG. 6, an example where the speed ranges permitting operations are defined is given, but it may also be defined as the speed ranges prohibiting operations.

[0044] Furthermore, in Section 1, the screen center is defined as the target used for calculating the control parameters of the pan. Additionally, in Section 1, it is defined to use the bounding boxes with a confidence level of 0.2 or more among the bounding boxes obtained by object detection by AI.

[0045] Next, taking the example of Section 2, as the condition for the pan coordinates to transition to Section 3, which is the next section of Section 2, it is defined that the pan coordinates move up to 3462. Also, as the condition for the tilt coordinates, it is defined that the tilt coordinates move up to -87. Furthermore, as the condition for the zoom coordinates, it is defined that the zoom coordinates move up to 14921.

[0046] Furthermore, in Section 2, it is defined that the aiming used for calculating the control parameters of the pan is set based on the left direction. Additionally, it is defined that the aiming used for calculating the control parameters of the tilt is set based on the downward direction. Furthermore, it is defined that the aiming used for calculating the control parameters of the zoom is set based on the zoom-out. Additionally, in Section 2, it is defined that the traveling direction of the object to be photographed is the left direction.

[0047] Furthermore, in Section 2, it is defined to permit operations within the speed ranges described below. For example, the pan speed is permitted in the range from 1 to 7. This has an aspect that is equivalent to defining the left direction as the direction permitting the rotation of the pan. Also, the tilt speed is permitted in the range from -3 to -1. This has an aspect that is equivalent to defining the downward direction as the direction permitting the rotation of the tilt. Furthermore, the zoom speed is permitted in the range from -3 to -1. This has an aspect that is equivalent to defining the zoom-out as the direction permitting the zoom. Furthermore, for zoom, it is defined to permit reduction up to 8931 and enlargement up to 14921.

[0048] Furthermore, in Section 2, as a target used for calculating the control parameters of the pan, it is defined to be the center in the height direction of the screen and the position 50 pixels to the left from the center in the width direction of the screen. Furthermore, in Section 2, it is defined to use the bounding boxes with a confidence level of 0.3 or higher among the bounding boxes obtained by object detection by AI.

[0049] Note that although the definitions regarding Section 1 and Section 2 are excerpted and explained in FIG. 6, it goes without saying that definitions are also set for other sections. For example, for Section 3, Section 5, and Section 6, since they correspond to the angular ranges A1 to A3 shown in FIG. 3, the speed range of the PTZ can be set narrower than the speed ranges defined in other sections, or a value corresponding to one value, for example, V1 to V3, can be set. Thereby, the speed of the PTZ can be fixed to a substantially constant value. In addition, in Section 8 corresponding to the goal of Course C, by defining the end point of the coordinates for each function of changing the field of view of the PTZ instead of the transition condition to the next section, the movement of the PTZ can be stopped.

[0050] Returning to the description of FIG. 4, the control unit 15 is a functional unit that performs overall control of the information processing apparatus 10. For example, the control unit 15 can be realized by a hardware processor. As shown in FIG. 4, the control unit 15 includes an image acquisition unit 15A, an object detection unit 15B, a control amount calculation unit 15C, a camera control unit 15E, and an output control unit 15F. Note that the control unit 15 may be realized by hard-wired logic or the like.

[0051] The image acquisition unit 15A is a processing unit that acquires a captured image. As one embodiment, the image acquisition unit 15A can acquire a captured image from the camera 30 in units of frames. Here, an example of acquiring a captured image transmitted from the camera 30 is given, but it is also possible to acquire a captured image via an external device, such as a file server, or a storage medium, such as a removable medium.

[0052] The object detection unit 15B is a processing unit that detects an object from a captured image. Such object detection may be realized by a machine learning model that executes an object detection task that takes an image as input and outputs the position and category of the object in the image. For example, the machine learning model may be realized by YOLO (You Only Look Once), Faster-RCNN (Regions with Convolutional Neural Network), DETR (End-to-End Object Detection with Transformers), or the like.

[0053] For training such a machine learning model, a training dataset of training data in which an image is associated with a ground truth label including a bounding box indicating the position of the object in the image and a class indicating the category of the object can be used.

[0054] As one aspect, in the training phase, the machine learning model can be trained according to a machine learning algorithm, such as deep learning, with the image as the explanatory variable and the ground truth label as the objective variable. Thereby, a trained machine learning model is obtained.

[0055] As another aspect, in the prediction phase, the captured image is input into a trained machine learning model. The machine learning model into which the input scene is thus input outputs a bounding box, a class, and a confidence level.

[0056] Hereinafter, the trained machine learning model for realizing the object detection task may be referred to as an "object detection model".

[0057] As one embodiment, when the image acquisition unit 15A acquires a differential captured image of a new frame, the object detection unit 15B inputs the captured image into the object detection model, thereby obtaining an object detection result (prediction result) output by the object detection model.

[0058] FIG. 7 is a schematic diagram showing an example of object detection. As shown in FIG. 7, the object detection unit 15B inputs the captured image 20 into the object detection model. The object detection model into which the captured image 20 is thus input outputs an object detection result including a bounding box, a class, and a confidence level for each object detected from the captured image 20. For example, in the example shown in FIG. 7, for each of the bounding boxes BB1 to BB3 detected from the captured image 20, a "class name" such as a horse and a "confidence level" of 0 to 1 are predicted.

[0059] The control amount calculation unit 15C is a processing unit that calculates control parameters for controlling the PTZ of the camera 30. Hereinafter, as an example of the control parameters, an example in which the speed of the PTZ is specified will be given. Needless to say, not only when specifying the relative movement from the current position, but also the coordinates of the PTZ can be specified by absolute values.

[0060] As an example of an embodiment, the control amount calculation unit 15C executes the following processing for each PTZ as a function of changing the field of view of the camera 30. That is, the control amount calculation unit 15C calculates control parameters according to the aiming and target settings defined in the section corresponding to the progress state of the race by the imaging object among the section setting data 13A stored in the storage unit 13. For example, the progress state of the race can be managed by updating the section loop counter based on the monitoring result of whether the coordinates of the PTZ observed in the new frame satisfy the transition condition to the next section each time a captured image of a new frame is acquired.

[0061] FIG. 8 is a schematic diagram showing an example of calculating control parameters. For example, FIG. 8 shows an example of calculating the control parameters of the PTZ from the object detection result for the captured image 20 shown in FIG. 7. Further, FIG. 8 shows an example of calculating the control parameters of the PTZ according to the aiming setting defined in section 2 among the sections included in the section setting data 13A. Further, FIG. 8 shows an example of calculating the pan speed among the control parameters.

[0062] As shown in FIG. 8, as a result of executing the object detection model, three bounding boxes BB1 to BB3 are detected from the captured image 20. In this case, among the three bounding boxes BB1 to BB3, the bounding box BB1 corresponding to the head in the "left direction" corresponding to the traveling direction of the imaging object is used as the aiming point. At this time, the bounding box to be used as the aiming point may be determined after narrowing down to the bounding boxes with a confidence level of "0.3" or more according to the definition of the confidence level in section 2.

[0063] Here, in section 2, as described with reference to FIG. 6, as a target used for calculating the control parameter of the pan, a position T1 that is the center in the height direction of the screen and 50 pixels to the left from the center in the width direction of the screen is defined.

[0064] In this case, a pan speed is calculated to align with a target position, for example, the position T1 where the dashed line and the dotted line intersect, using as a reference a two-dot chain line corresponding to a specific position of the bounding box BB1, such as the left side. For example, as the distance D1 from the reference to the target increases, a larger pan speed is calculated, while as the distance D1 from the reference to the target decreases, a smaller pan speed is calculated. Further, for the pan speed, a negative sign, i.e., the right direction, is set when the reference position is ahead of the target position in the traveling direction of the object to be photographed, while a positive sign, i.e., the left direction, is set when the reference position is behind the target position in the traveling direction of the object to be photographed. Here, an example of calculating the pan speed has been described, but the tilt speed and the zoom speed can be calculated in the same way.

[0065] The camera control unit 15E is a processing unit that controls the PTZ of the camera 30. As an embodiment, the camera control unit 15E executes the following processing for each of the PTZ for the function of changing the field of view of the camera 30. That is, the camera control unit 15E determines whether the control parameter calculated by the control amount calculation unit 15C is a permitted operation defined in the section corresponding to the progress state of the race by the object to be photographed in the section setting data 13A. At this time, when the control parameter is not a permitted operation, the camera control unit 15E designates, as the control parameter of the remote operation command, the boundary value on the side corresponding to the sign of the control parameter calculated by the control amount calculation unit 15C among the boundary values of the range of the control parameter defined as the permitted operation in the section. On the other hand, when the control parameter is a permitted operation, the camera control unit 15E designates the control parameter calculated by the control amount calculation unit 15C as the control parameter of the remote operation command of the camera 30. Thereafter, the camera control unit 15E transmits, to the camera 30, the remote operation command of the PTZ for which the control parameter is designated for each function of changing the field of view of the camera 30.

[0066] The output control unit 15F is a processing unit that executes output control of the captured image acquired by the image acquisition unit 15A. As one embodiment, the output control unit 15F can output the captured image acquired by the image acquisition unit 15A in frame units. For example, the captured images captured in time series may be output as a video. Such output may be implemented in any form such as television broadcasting, Internet distribution, live broadcasting using SNS (Social Networking Service), etc.

[0067] <Flow of processing> Next, the flow of processing of the information processing apparatus 10 according to the present embodiment will be described. FIG. 9 is a flowchart showing the procedure of camera control processing. This processing can be started when the above-described camera control function is activated.

[0068] As shown in FIG. 9, the camera control unit 15E moves the camera 30 to the initial position (step S101), and the camera control unit 15E sets an initial value, for example, "1", to the loop counter m of the section (step S102).

[0069] Thereafter, the image acquisition unit 15A acquires a captured image of a new frame (step S103). Then, the object detection unit 15B inputs the captured image acquired in step S103 to the object detection model, and acquires the object detection result (prediction result) output by the object detection model (step S104).

[0070] Subsequently, the control amount calculation unit 15C identifies the bounding box of the aiming defined in the section corresponding to the value of the loop counter m among the bounding boxes obtained as the object detection result in step S105 (step S105).

[0071] Thereafter, loop process 1 that repeats the processes from step S106 to step S108 below is executed for the number of times corresponding to the number N of functions for changing the field of view of camera 30. Here, an example in which the processes from step S106 to step S108 below are repeated is given, but the processes from step S106 to step S108 below may be executed in parallel.

[0072] That is, the control amount calculation unit 15C calculates a control parameter regarding the n-th field of view change function based on the position of the bounding box identified as the aiming in step S105 and the position of the target defined in the section corresponding to the value of the loop counter m in the section setting data 13A (step S106).

[0073] Then, the camera control unit 15E determines whether the control parameter calculated in step S106 is a permitted operation defined in the section corresponding to the value of the loop counter m in the section setting data 13A (step S107).

[0074] Here, when the control parameter regarding the n-th field of view change function is not a permitted operation (step S107 No), the camera control unit 15E designates, as the control parameter of the remote operation command, the boundary value on the side corresponding to the sign of the control parameter calculated in step S106 among the boundary values of the range of the control parameter defined as the permitted operation in the section corresponding to the value of the loop counter m (step S108).

[0075] On the other hand, when the control parameter is not a permitted operation (step S107 Yes), the process of step S108 is skipped. In this case, the control parameter calculated in step S106 is designated as the control parameter of the remote operation command of camera 30.

[0076] By repeating such loop process 1, a control parameter is designated for each function for changing the field of view of camera 30.

[0077] After that, the camera control unit 15E transmits a remote operation command with control parameters specified for each function of changing the field of view of the camera 30 to the camera 30 (step S109).

[0078] Subsequently, the camera control unit 15E determines whether or not the coordinates of the PTZ satisfy the transition condition to the next section defined in the section corresponding to the value of the loop counter m in the section setting data 13A (step S110).

[0079] At this time, when the coordinates of the PTZ satisfy the transition condition to the next section (step S110 Yes), the camera control unit 15E increments the loop counter m (step S111). After that, the camera control unit 15E acquires the setting of the section corresponding to the value of the loop counter m in the section setting data 13A (step S112), and proceeds to the process of step S103.

[0080] Note that when the coordinates of the PTZ do not satisfy the transition condition to the next section (step S110 No), the processes of step S111 and step S112 are not executed, and the process proceeds to the process of step S103.

[0081] <One aspect of the effect> As one aspect, when controlling the pan of the camera 30 according to the position of the object to be photographed in the photographed image of the camera 30, the information processing apparatus 10 according to the present embodiment permits pan control only in a previously defined direction. For this reason, it is possible to suppress the switching of the moving direction of the pan between the traveling direction and the opposite direction. Therefore, according to the camera control function according to the present embodiment, it is possible to realize smooth pan control of the camera 30.

[0082] As another aspect, when controlling the pan of the camera 30 according to the position of the object to be photographed in the photographed image of the camera 30, if the coordinates of the pan of the camera 30 are within a predetermined angular range, pan control that moves at a previously defined speed is permitted. For this reason, it is possible to suppress the switching of the pan movement direction between the traveling direction and the opposite direction. Therefore, according to the camera control function according to the present embodiment, it is possible to realize smooth pan control of the camera 30.

[0083] Furthermore, since automatic shooting enabling smooth pan control of the camera 30 is realized by the camera control function according to the present embodiment, it is possible to eliminate the need for personnel for shooting. Therefore, the location where the camera 30 is installed is not restricted by factors such as the ease of access for the photographer and the space for the photographer to take pictures. As a result, it is also possible to install the camera 30 at a location where the camera angle is in good condition or at a location where occlusion due to competition or viewing facilities is unlikely to occur.

[0084] <Example 2> Now, although the embodiments of the present disclosure have been described so far, various applications are possible, and furthermore, in addition to the above-described Example 1, it may be implemented in various different forms.

[0085] <Numerical values, etc.> The matters described in the above Example 1, such as specific examples of the types of control parameters and the number of cameras 30, are merely examples and can be changed. Also, the flowcharts described in the examples can have the processing order changed within a non-contradictory range.

[0086] <Application examples> For example, in the above-described Example 1, an example where the number of cameras 30 used for photographing the course C is one was given. However, it is also possible to perform photographing of horse race relay using two or more cameras 30. FIG. 10 is a schematic diagram showing an application example of section division. FIG. 10 shows an example in which two cameras 30, namely camera 30A and camera 30B, are used for photographing a horse race relay. In this case, as shown in FIG. 10, camera 30A is assigned to photograph sections 1 to 4 and sections 7 to 8 out of sections 1 to 8, and camera 30B can be assigned to photograph sections 5 and 6. By such an assignment of sections, it is possible to achieve close-range photographing of racehorses throughout the course C and also suppress the occurrence of occlusion by facilities for the competition or spectators, such as large display boards.

[0087] <System> Regarding the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above document or drawings, they can be arbitrarily changed unless otherwise specified. For example, any one or more of the functional units such as the image acquisition unit 15A, object detection unit 15B, control amount calculation unit 15C, camera control unit 15E, and output control unit 15F may be configured by separate devices.

[0088] In addition, each component of each device shown in the drawings is a functional concept, and it is not necessarily physically configured as shown in the drawings. That is, the specific forms of distribution and integration of each device are not limited to those shown. In other words, all or part of it can be functionally or physically distributed and integrated in arbitrary units according to various loads and usage situations. Note that each configuration may be a physical configuration.

[0089] Furthermore, each processing function performed by each device can be realized in whole or in any part by a CPU (Central Processing Unit) and a program analyzed and executed by the CPU, or can be realized as hardware by wired logic.

[0090] <Hardware> Next, a hardware configuration example of the computer described in the above embodiment will be described. FIG. 11 is a diagram showing a hardware configuration example. As shown in FIG. 11, the information processing apparatus 10 includes a communication apparatus 10a, a storage apparatus 10b, a memory 10c, and a processor 10d. Note that each unit shown in FIG. 11 may be mutually connected by a bus or the like.

[0091] The communication apparatus 10a is a network interface card or the like. The storage apparatus 10b is a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive). For example, the storage apparatus 10b stores a program or a DB that realizes the functions shown in FIG. 4.

[0092] The processor 10d reads out a program that executes the same processing as the processing unit shown in FIG. 4 from the storage apparatus 10b or the like and expands it in the memory 10c, thereby operating a process that executes the functions described in FIG. 4.

[0093] Such a process realizes the same functions as the processing unit included in the information processing apparatus 10. For example, the processor 10d reads out a program having the same functions as an image acquisition unit 15A, an object detection unit 15B, a control amount calculation unit 15C, a camera control unit 15E, an output control unit 15F, and the like from the storage apparatus 10b or the like. Then, the processor 10d executes a process that executes the same processing as the image acquisition unit 15A, the object detection unit 15B, the control amount calculation unit 15C, the camera control unit 15E, the output control unit 15F, and the like.

[0094] In this way, the information processing apparatus 10 operates as an information processing apparatus that executes a calculation method by reading and executing a program. Further, the information processing apparatus 10 can also read the program from a recording medium by a medium reading device and realize the same functions as those of the above-described embodiments by executing the read program. Note that the program in this other embodiment is not limited to being executed by the information processing apparatus 10. For example, the present invention can be similarly applied when another computer or server executes the program, or when these cooperate to execute the program.

[0095] The above program can be distributed via a network such as the Internet. Further, the above program can be recorded on an arbitrary recording medium and executed by being read from the recording medium by a computer. For example, the recording medium can be realized by a hard disk, a flexible disk (FD), a CD-ROM, a MO (Magneto-Optical disk), a DVD (Digital Versatile Disc), or the like.

Explanation of Reference Numerals

[0096] 1 Camera control system 10 Information processing apparatus 11 Communication control unit 13 Storage unit 13A Interval setting data 15 Control unit 15A Image acquisition unit 15B Object detection unit 15C Control amount calculation unit 15E Camera control unit 15F Output control unit 30 Camera

Claims

1. Obtain a captured image of a camera, When controlling the pan of the camera according to the position of the object to be photographed in the captured image, if the coordinates of the pan of the camera are within a predetermined angle range, perform pan control to move at a predefined speed. A camera control program characterized by causing a computer to execute the process.

2. The process to be executed includes, among the setting data in which the angle range of the pan corresponding to a specific section among the sections into which the course on which the object to be photographed moves is divided and the speed of the pan of the camera are defined, when the coordinates of the pan of the camera correspond to the angle range of the pan corresponding to the specific section, performing a pan control to move at the speed associated with the specific section. The camera control program according to claim 1, characterized in that it includes the process.

3. The process to be executed includes, when controlling the tilt of the camera according to the position of the object to be photographed in the captured image, if the coordinates of the tilt of the camera are within a predetermined angle range, performing tilt control to move at a predefined speed. The camera control program according to claim 1, characterized in that it includes the process.

4. The process to be executed includes, when controlling the zoom of the camera according to the position of the object to be photographed in the captured image, if the coordinates of the zoom of the camera are within a predetermined angle range, performing zoom control to move at a predefined speed. The camera control program according to claim 1, characterized in that it includes the process.

5. Obtain a captured image of a camera, When controlling the pan of the camera according to the position of the object to be photographed in the captured image, if the coordinates of the pan of the camera are within a predetermined angle range, perform pan control to move at a predefined speed. A camera control method characterized in that a computer executes the process.

6. Obtain a captured image of a camera, When controlling the pan of the camera according to the position of the object to be photographed in the captured image, if the coordinates of the pan of the camera are within a predetermined angle range, perform pan control to move at a predefined speed. An information processing apparatus characterized by having a control unit that executes the process.

Images