Sphere detection device, sphere detection method, and program

Abstract

To accurately detect a sphere with a simple configuration.SOLUTION: A sphere detector comprises an input unit, an extract unit, an identification unit, and an output unit. The input unit inputs a frame image capturing the sphere. The extraction unit extracts an extraction area including the sphere from the frame image. The identification unit identifies the sphere contour by detecting degree of separation in the extraction area using a prescribed separability filter. The output unit outputs information related to the sphere position based on the identified contour.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a sphere detection device, a sphere detection method, and a program. [Background technology]

[0002] Conventionally, there are known techniques for extracting and tracking human movements and flying spheres from video. These techniques are expected to be put to practical use in various sports scenes for the purpose of analyzing movements and evaluating performance in sports (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6575609 Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional techniques have been complex, requiring the use of multiple cameras, and have not been able to accurately detect spheres.

[0005] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique that can accurately detect a sphere with a simple configuration. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, one aspect of the present invention is a sphere detection device that includes an input unit that inputs a frame image in which a sphere is captured, an extraction unit that extracts an extraction area including the sphere from the frame image, an identification unit that identifies the contour of the sphere by detecting the degree of separation in the extraction area using a predetermined separation filter, and an output unit that outputs information regarding the position of the sphere based on the identified contour.

[0007] In order to solve the above-mentioned problems, one aspect of the present invention is a sphere detection method in which a computer used in a sphere detection device performs processing including an input process for inputting a frame image in which a sphere is captured, an extraction process for extracting an extraction area including the sphere from the frame image, an identification process for identifying the contour of the sphere by detecting the degree of separation in the extraction area using a predetermined separation filter, and an output process for outputting information regarding the position of the sphere based on the identified contour.

[0008] Another aspect of the present invention is a program that causes a computer used in a sphere detection device to function as an input unit that inputs a frame image in which a sphere is captured, an extraction unit that extracts an extraction area including the sphere from the frame image, an identification unit that identifies the contour of the sphere by detecting the degree of separation in the extraction area using a predetermined separation filter, and an output unit that outputs information regarding the position of the sphere based on the identified contour. [Effects of the Invention]

[0009] According to the present invention, a sphere can be detected with high accuracy using a simple configuration. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 2 is an explanatory diagram showing an example of a hardware configuration of the mobile terminal device 100. [Figure 2] FIG. 2 is an explanatory diagram showing a functional configuration of the mobile terminal device 100 related to ball detection. [Figure 3] FIG. 10 is an explanatory diagram showing an example of the shape of a variable elliptical separability filter 204a. [Figure 4] FIG. 2 is an explanatory diagram showing an example of the shape of a variable elliptical separability filter 204a. [Figure 5] FIG. 10 is an explanatory diagram showing an example of the shape of a variable elliptical separability filter 204a. [Figure 6] 10 is a flowchart showing an example of a ball detection process performed by the mobile terminal device 100. DETAILED DESCRIPTION OF THE INVENTION

[0011] Next, a sphere detection device, a sphere detection method, and a program according to the present embodiment will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiments to which the present invention is applied are not limited to the following embodiments.

[0012] (Embodiment) 1 is an explanatory diagram showing an example of the hardware configuration of a mobile terminal device 100. The mobile terminal device 100 is an example of a sphere detection device. The mobile terminal device 100 is, for example, a smartphone. However, the mobile terminal device 100 is not limited to a smartphone, and may be other computer devices such as a notebook computer or a tablet terminal.

[0013] 1, mobile terminal device 100 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a memory 104, a communication interface (communication I / F) 105, an operation unit 106, a camera 107, a microphone 108, a display 109, a speaker 110, and a GPS (Global Positioning System) unit 111. Each unit is connected to each other via a bus 120.

[0014] CPU 101 controls the entire mobile terminal device 100. ROM 102 stores various programs. RAM 103 is used as a work area for CPU 101. That is, CPU 101 controls the entire mobile terminal device 100 by executing various programs stored in ROM 102 and memory 104 while using RAM 103 as a work area.

[0015] The memory 104 stores various data. For example, a flash memory is used as the memory. The memory 104 stores various programs such as a sphere detection application (application program). The communication interface 105 is connected to a network, and is connected via the network to other devices such as an external server, etc. Examples of communication networks that function as networks include the Internet and mobile phone networks.

[0016] The operation unit 106 is a touch panel that displays a plurality of touch keys for inputting characters, numbers, various instructions, etc., or hard keys. The camera 107 captures moving images. For the camera 107, for example, a CCD (Charge-Coupled Device) camera can be used. The microphone 108 inputs the operator's voice. The display 109 displays icons, cursors, menus, windows, characters, images, codes, and the like.

[0017] The speaker 110 outputs sounds including voice. The GPS unit 111 receives radio waves from GPS satellites and outputs information indicating the current position of the mobile terminal device 100. The mobile terminal device 100 also includes various sensors (e.g., a gyro sensor, an acceleration sensor, etc.) not shown, and has the function of detecting various states of the mobile terminal device 100 (e.g., the tilt of the mobile terminal device 100).

[0018] (About ball detection) In this embodiment, the mobile terminal device 100 can estimate the position and trajectory of a ball in a ball game such as soccer by launching a ball detection app. Ball detection will be described in detail below. Note that, although a soccer ball will be used as an example of a sphere in the following description, the sphere may also be a volleyball, basketball, handball, tennis ball, or the like. Furthermore, the sphere is not necessarily limited to a spherical one, and may also be an oval sphere such as a rugby ball.

[0019] 2 is an explanatory diagram showing the functional configuration of mobile terminal device 100 related to ball detection. As shown in FIG. 2, mobile terminal device 100 includes input unit 201, YOLO (You Only Look Once) 202, Kalman filter 203, separability filter 204, and output unit 205. Each of units 201 to 205 is realized by CPU 101. That is, CPU 101 executes a sphere detection program (sphere detection application) stored in memory 104, thereby realizing the functions of each of units 201 to 205.

[0020] The processing according to this embodiment can be performed not only by CPU 201 executing a program, but also by using hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or by using software and hardware in cooperation with each other to perform the processing according to this embodiment.

[0021] The input unit 201 receives input of each frame of a video captured by the camera 107. The video may be, for example, a video of a soccer player kicking a ball. The input unit 201 outputs the input video to YOLO 202 frame by frame.

[0022] (About YOLO202) YOLO is an example of an identification unit. YOLO202 detects the area containing a ball from one frame of an image output from the input unit 201. YOLO202 can output a bounding box simply by inputting the image into a single trained neural network. Specifically, YOLO202 analyzes the input image using a trained neural network and outputs the area of ​​the ball contained in the image (bounding box). The bounding box is a rectangle that circumscribes the ball. The bounding box is an example of an identification area.

[0023] YOLO 202 also outputs a confidence value (reliability score). The confidence value indicates the accuracy of whether the bounding box accurately surrounds the area of ​​the target object (ball). YOLO 202 outputs the bounding box detected from one frame of image and the confidence value to the Kalman filter 203.

[0024] (About the trained model) Here, we will provide further information on generating trained models. Trained models are generated by a learning device such as a personal computer. The learning device uses a previously prepared dataset to train parameters of a machine learning model such as a neural network. At this time, the learning device updates each parameter using a gradient descent method or the like to minimize a loss function using the calculation results of the machine learning model. For example, the loss function represents the cross-entropy error between the output value of the classification model and the output sample of the dataset. The learning device terminates the learning process when the evaluation value of the loss function falls below a predetermined threshold or when the learning process has been repeated a predetermined number of times. A trained model is generated when the learning device completes the learning process.

[0025] (About Kalman Filter 203) The Kalman filter 203 is an example of a region estimation unit, a state estimation unit, and an extraction unit. The Kalman filter 203 estimates the state of a dynamic system using a given state space model and observation values ​​with errors. In this embodiment, a simple motion model (uniformly accelerated motion model) that can be expressed by a quadratic polynomial is used. The acceleration due to gravity is 9.8 m / s 2 The state vector of the Kalman filter 203 is defined as equation (1).

[0026]

number

[0027] x t , y t , r t indicate the x-coordinate, y-coordinate, and radius of the ball at time t, respectively. Also, dot notation indicates time derivatives (velocity and acceleration). The Kalman filter 203 has two steps: state prediction and state update. First, the Kalman filter 203 uses a given state space model, i.e., a state transition matrix, to calculate the previous state x t-1 to current state x t In other words, the Kalman filter 203 estimates the position, velocity, and acceleration of the sphere. In the next step, the Kalman filter 203 fuses (filters) the measurement vector (YOLO output) with the model estimate. Typically, the filtered estimate is used to update the current state. In this embodiment, the Kalman filter 203 refines the output of the YOLO 202 using the separability filter 204.

[0028] The Kalman filter 203 extracts a region of interest (ROI) around the ball from the frame image and outputs it to the separability filter 204. The ROI is an example of an extracted region. If the confidence value detected by YOLO 202 is equal to or greater than a predefined threshold, the filtered estimate is used to extract the ROI. On the other hand, if the confidence value is less than the threshold or if the ball is not detected, the region identified from the position and radius of the sphere predicted by the Kalman filter 203 is used instead.

[0029] (Regarding the separability filter 204) Next, we will explain the separability filter 204. The separability filter 204 is an example of a specifier. First, we will explain an elliptical separability filter as an example of the separability filter 204. The elliptical separability filter (ESF) is an elliptical extension of the circular separability filter (CSF). The elliptical separability filter has been studied and proposed for the purpose of accurately detecting the pupil contour. The separability filter 204 is a filter that focuses on local image features for accurate detection. This is in contrast to YOLO, which is designed to take global context into account, so using them simultaneously can take advantage of the advantages of both. The separability η is a statistical quantity that indicates the degree to which image features extracted from two regions are separated, and has been proven to be robust against the influence of several noises. The separability can be calculated based on Equations (1) to (3).

[0030]

number

[0031]

number

[0032]

number

[0033] Here, N is the total number of pixels in all regions, n1 is the number of pixels in the first region, n2 is the number of pixels in the second region, "P1 bar" is the average value of the feature values ​​in the first region, "P2 bar" is the average value of the feature values ​​in the second region, "P m The "η" is the average value of the features across all regions. The range of separability is 0≦η≦1. The elliptical separability filter defines two elliptical regions and calculates a separability map. A two-dimensional peak search algorithm is applied to this separability map to calculate the estimated position of the ball.

[0034] Applying an elliptical filter to the entire image with an appropriate filter setting increases the degree of separation around the ball. While it is difficult to set an appropriate filter without prior information such as the radius (pixel value) of the object, this embodiment makes it possible to limit the size of the filter by using the output of YOLO 202 and the calculated region of interest. Furthermore, this embodiment uses a variable elliptical separability filter (Adaptive-ESF) 204a, an extension of the elliptical separability filter, as a refinement step to improve robustness when tracking a fast-moving circular object with a complex background. The variable elliptical separability filter 204a adds a mechanism to the elliptical separability filter that deforms the major and minor axes of the elliptical filter based on the ball velocity estimated by the Kalman filter 203. The semimajor axis α of the inner elliptical filter (first region) at time t is calculated using the following equation: t , semi-minor axis β t , angle θ t , define

[0035]

number

[0036]

number

[0037]

number

[0038] Here, we introduced a hyperparameter λ to control the impact of the ball's velocity on the filter shape. The elliptical filter (second region) outside the ball is defined as γ times larger than the first region while maintaining the same aspect ratio. Examples of filters with various shapes are shown in Figures 3 to 5.

[0039] (An example of the shape of the variable elliptical isolation filter 204a) 3 to 5 are explanatory diagrams showing an example of the shape of variable ellipse separability filter 204a. In Fig. 3 to Fig. 5, variable ellipse separability filter 204a includes first region 301 (301a, 301b, 301c) and second region 302 (302a, 302b, 302c). As shown in the figures, in variable ellipse separability filter 204a, the major and minor axes of the ellipse filter are deformed based on the ball velocity estimated by Kalman filter 203.

[0040] In this way, the variable ellipse separability filter 204a accurately identifies the position and radius of the ball by detecting peak values ​​that are characteristic of the image showing the ball from the region of interest. Since the peak values ​​are high at the contour of the ball, the variable ellipse separability filter 204a can calculate the position and radius of the ball by fitting the contour of the ball to the set of peak values. Once the ball is identified by the variable ellipse separability filter 204a, the Kalman filter 203 calculates the next state x t+1 To predict this, a state update is performed.

[0041] Furthermore, when the ball is identified by the variable ellipse separability filter 204a, the output unit 205 outputs information about the identified position, including the position and radius of the ball. The output unit may display the information about the position directly on the display 109, or may output the information about the position to a processing unit that performs processing based on the information about the position. The processing includes, for example, processing to derive the ball speed, ball movement, etc. The processing also includes processing to generate an image showing the ball's trajectory based on the derived information. The display 109 may display the information after processing.

[0042] It is also possible that multiple balls (balls that are not the target of tracking) are detected from the frame image. In this case, the ball with the highest confidence value in YOLO 202 may be determined as the ball to be tracked. Alternatively, the ball corresponding to the position predicted by Kalman filter 203 may be determined as the ball to be tracked.

[0043] (An example of ball detection processing performed by the mobile terminal device 100) 6 is a flowchart showing an example of ball detection processing performed by the mobile terminal device 100. In FIG. 6, the mobile terminal device 100 starts a ball detection app and determines whether ball detection has started by accepting a predetermined operation (step S601). If ball detection has not started (step S601: NO), the mobile terminal device 100 ends the processing. If ball detection has started (step S601: YES), the input unit 201 inputs frame images of a video captured by the camera 107 to the YOLO 202 (step S602).

[0044] Then, YOLO 202 detects the ball (step S603) and outputs a bounding box and a confidence value to the Kalman filter 203. Next, the Kalman filter 203 determines whether the confidence value output from YOLO 202 is equal to or greater than a threshold (step S604). If the confidence value is equal to or greater than the threshold (step S604; YES), the mobile terminal device 100 cuts out a region of interest (ROI) using the estimated value filtered by the Kalman filter 203 (step S605), and proceeds to step S607.

[0045] On the other hand, if the confidence value is less than the threshold (step S604; NO) or if the ball is not detected by YOLO 202, the mobile terminal device 100 extracts a region of interest using the predicted value by the Kalman filter 203 (step S606). Next, the variable elliptical separability filter 204a determines the shape of the elliptical filter based on the ball velocity estimated by the Kalman filter 203 (step S607).

[0046] Then, the variable elliptical separability filter 204a detects a peak value that is a feature of the image showing the ball from the region of interest (step S608). Next, the output unit 205 outputs the position and radius of the ball according to the peak value (step S609). Then, the Kalman filter 203 calculates the next state x t+1 To predict the ball speed, a state update is performed using the ball position and radius determined in step S609 (step S610).

[0047] Then, the mobile terminal device 100 determines whether or not there is a next frame (step S611). If there is a next frame (step S611: YES), the mobile terminal device 100 returns to step S602 and repeats the processes of steps S602 to S611. On the other hand, if there is no next frame (step S611: NO), the mobile terminal device 100 ends the series of processes.

[0048] (Experimental results) To quantitatively evaluate the performance of the sphere detection method according to this embodiment, tracking was performed on video of a soccer kick. Since the use case envisioned was that the mobile terminal device 100 would be used at a practice field, the evaluation was performed on video of the practice field. The hyperparameter λ and the noise matrix of the Kalman filter 203 were determined using grid search. The algorithms for each method are as follows:

[0049] Conventional mode: "YOLO202 + Kalman filter 203". This is a combination of object detection using YOLO202 and Kalman filter 203, which compensates for missing values. Aspects of this embodiment: "YOLO 202 + Kalman filter 203 + variable elliptical separability filter 204a."

[0050] As experimental results, we show the root mean square error (RMSE). Conventional size: 44.32cm This embodiment: 2.19 cm

[0051] From these results, it can be seen that the embodiment of the present invention can significantly reduce the root mean square error (RMSE) compared to the conventional embodiment. In the conventional embodiment, the RMSE was high because YOLO 202 was unable to detect the ball in all frames, and the Kalman filter 203 gradually deviated from the correct trajectory. Furthermore, when the detection results of the conventional embodiment were examined, it was found that, although the conventional embodiment was able to perform stable detection in situations where the background of the ball was nearly monochromatic (when the ball was close to the ground or in the air), accurate ball tracking was difficult when the background of the ball was complex, such as trees or fences. On the other hand, the embodiment of the present invention can accurately track the ball by calculating a separability map, even when the background of the ball was complex, such as trees or fences.

[0052] As described above, the mobile terminal device 100 according to this embodiment uses a predetermined separability filter to detect the degree of separation in an extracted region including a ball extracted from a frame image, thereby identifying the contour of the sphere and outputting information about the position of the sphere based on the contour. This allows the ball to be detected easily and accurately using an app installed on the mobile terminal device 100.

[0053] Furthermore, in this embodiment, the Kalman filter 203 extracts a region of interest from the frame image based on the bounding box output by the YOLO 202. This makes it possible to instantly and accurately detect the ball in the frame image.

[0054] In this embodiment, the Kalman filter 203 estimates an estimated region that is likely to include the ball from the frame image and extracts a region of interest from the frame image, thereby improving the accuracy of ball detection.

[0055] In this embodiment, the Kalman filter 203 extracts a region of interest based on a bounding box when the confidence value is equal to or greater than a threshold, and extracts a region of interest based on an estimated region when the confidence value is less than the threshold. This makes it possible to extract a predicted region of interest even when the ball detection accuracy by YOLO 202 is low due to the ball's speed or trajectory, thereby enabling the ball position to be detected with high accuracy.

[0056] In this embodiment, the separability filter 204 identifies the contour of the ball using a variable ellipse separability filter 204a that determines the major and minor axes of an ellipse based on the ball velocity estimated by the Kalman filter 203. This allows the separability filter 204 to identify the ball more quickly and accurately, thereby enabling more efficient ball detection.

[0057] In this way, the mobile terminal device 100 according to this embodiment can improve the detection performance of the ball, and therefore can more accurately determine the trajectory characteristics of the ball. Furthermore, by utilizing the accurately determined trajectory characteristics, training efficiency can be improved. Specifically, by capturing an image of an athlete's kick, quantitative data on the kick can be obtained. Therefore, this data can be used for coaching or can be checked by the athlete himself, thereby helping to acquire and improve the athletic ability of the athlete.

[0058] <Modifications of the embodiment> Next, a modified example of the embodiment will be described. In the above-described embodiment, a case where a variable elliptical separability filter 204a (Adaptive-ESF) is used as the separability filter 204 is described. In the modified example, a case where an elliptical separability filter (ESF) is used as the separability filter 204 is described.

[0059] The modified embodiment is as follows. Aspect of this embodiment: "YOLO 202 + Kalman filter 203 + elliptical separability filter." The elliptical separability filter is a filter whose shape is invariant and whose size depends on the region of interest. When the lengths of the major and minor axes of the elliptical separability filter are equal, the elliptical separability filter is equivalent to a circular separability filter (CSF). In other words, the elliptical separability filter includes a circular separability filter.

[0060] As experimental results, we show the root mean square error (RMSE). Modified embodiment: 2.19 cm (conventional embodiment: 44.32 cm)

[0061] From this result, it can be seen that the root mean square error (RMSE) can be significantly reduced in the modified embodiment as well, compared to the conventional embodiment. According to the modified embodiment, even if the ball is in a complex background such as trees and fences, it is possible to accurately track the ball by calculating the separability map.

[0062] The mobile terminal device 100 according to the above-described modification uses an elliptical separability filter 204 with a size according to the region of interest. This allows attention to be paid to local image features, making it possible to accurately identify the ball. Therefore, the separability filter 204 can quickly and accurately identify the ball. This allows for efficient ball detection.

[0063] In another embodiment, the mobile terminal device 100 may extract an ROI in the region output by the YOLO 202 without using the Kalman filter 203. In another embodiment, the mobile terminal device 100 may always extract an ROI using the predicted value of the Kalman filter 203 without using the YOLO 202. However, as in the above-described embodiment, by using the YOLO 202 and the Kalman filter 203 to compensate for each other's shortcomings, it is possible to extract an ROI more accurately.

[0064] In another embodiment, the mobile terminal device 100 may execute another detection process using a machine learning model instead of YOLO 202. For example, the mobile terminal device 100 may use another model such as R-CNN or SSD.

[0065] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention described in the claims and their equivalents.

[0066] The program for implementing the mobile terminal device 100 described above may be recorded on a computer-readable recording medium and loaded into a computer system for execution. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" also refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into a computer system. The term "computer-readable recording medium" also includes devices that retain a program for a certain period of time, such as volatile memory (RAM) within a computer system that serves as a server or client when the program is transmitted via a network such as the Internet or a communication line such as a telephone line. The program may also be transmitted from a computer system storing the program in a storage device to another computer system via a transmission medium or by transmission waves within the transmission medium. The term "transmission medium" used to transmit the program refers to a medium capable of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may also be used to implement some of the functions described above. Furthermore, the above-mentioned functions may be realized in combination with a program already recorded in the computer system, that is, a so-called differential file (differential program). [Explanation of symbols]

[0067] 100... Mobile terminal device, 102... ROM, 103... RAM, 104... Memory, 109... Display, 201... Input unit, 202... YOLO, 203... Kalman filter, 204... Separability filter, 204a... Variable elliptic separability filter, 205... Output unit

Claims

1. an input unit for inputting frame images of a sphere; an identification unit that inputs an image to a trained model that has been trained to output a region that is likely to contain a sphere, and outputs an identified region by inputting the frame image, and outputs a confidence score that indicates the accuracy of the identification region containing a sphere; a region estimation unit that estimates an estimated region that is highly likely to include the sphere from the frame image based on the position of the sphere in the past frame image; an extraction unit that extracts an extraction region including the sphere from the frame image based on the estimated region when the reliability score is less than a threshold, and extracts the extraction region based on the identified region when the reliability score is equal to or greater than a threshold; an identifying unit that identifies the contour of the sphere by detecting the degree of separation in the extracted region using a predetermined separability filter; an output unit that outputs information about the position of the sphere based on the identified contour; A sphere detection device comprising:

2. a state estimation unit that estimates the velocity of the sphere based on the position of the sphere in a past frame image; the predetermined separability filter is a variable elliptical separability filter that changes the major axis and minor axis of an elliptical shape, The identifying unit identifies the contour using the predetermined separability filter, the lengths of which are determined based on the velocity. The spherical object detection device according to claim 1 .

3. the predetermined separability filter is an elliptical separability filter whose shape is invariant and whose size corresponds to the extraction region; The spherical object detection device according to claim 1 .

4. An input unit for inputting a frame image in which a sphere is captured; an extraction unit that extracts an extraction region including the sphere from the frame image; a state estimation unit that estimates the velocity of the sphere based on the position of the sphere in a past frame image; an identifying unit that identifies the contour of the sphere by detecting the degree of separation in the extracted region using a predetermined separability filter that determines the lengths of a major axis and a minor axis based on the velocity; an output unit that outputs information about the position of the sphere based on the identified contour; Equipped with the predetermined separability filter is a variable elliptical separability filter that changes the major axis and minor axis of an elliptical shape; Sphere detection device.

5. A computer used in the sphere detection device an input step of inputting a frame image in which a sphere is captured; an identification process in which an image is input, the frame image is input to a trained model that has been trained to output an area that is likely to contain a sphere, and an identification region is output by inputting the frame image, and a confidence score indicating the accuracy of the inclusion of a sphere in the identification region is output; a region estimation step of estimating an estimated region that is highly likely to include the sphere from the frame image based on the position of the sphere in the past frame image; an extraction step of extracting an extraction region including the sphere from the frame image based on the estimated region when the reliability score is less than a threshold, and extracting the extraction region based on the identified region when the reliability score is equal to or greater than a threshold; a step of detecting the degree of separation in the extracted region using a predetermined separability filter to identify the contour of the sphere; an output step of outputting information relating to the position of the sphere based on the identified contour; A sphere detection method that performs a process including:

6. A computer used in the sphere detection device, an input unit for inputting frame images of the sphere; an identification unit that inputs an image into a trained model that has been trained to output a region that is likely to contain a sphere, and outputs the frame image to output an identified region, and outputs a confidence score that indicates the accuracy of the identification region containing a sphere; a region estimation for estimating an estimated region that is highly likely to include the sphere from the frame image based on the position of the sphere in the past frame image; an extraction unit that extracts an extraction region including the sphere from the frame image based on the estimated region when the reliability score is less than a threshold, and extracts the extraction region based on the identified region when the reliability score is equal to or greater than a threshold; an identifying unit that identifies the contour of the sphere by detecting the degree of separation in the extracted region using a predetermined separability filter; an output unit that outputs information regarding the position of the sphere based on the identified contour; A program that functions as a

7. A computer used in a sphere detection device, an input step of inputting a frame image in which a sphere is captured; an extraction step of extracting an extraction region including the sphere from the frame image; a state estimation step of estimating the velocity of the sphere based on the position of the sphere in a past frame image; a specifying step of specifying the contour of the sphere by detecting the degree of separation in the extracted region using a predetermined separability filter whose major and minor axis lengths are determined based on the velocity; an output step of outputting information relating to the position of the sphere based on the identified contour; Perform a process including the predetermined separability filter is a variable elliptical separability filter that changes the major axis and minor axis of an elliptical shape; Sphere detection method.

8. A computer used in a sphere detection device, an input unit for inputting frame images of the sphere; an extraction unit that extracts an extraction region including the sphere from the frame image; a state estimation unit that estimates the velocity of the sphere based on the position of the sphere in a past frame image; an identification unit that identifies the contour of the sphere by detecting the degree of separation in the extracted region using a predetermined separability filter that determines the lengths of a major axis and a minor axis based on the velocity; an output unit that outputs information regarding the position of the sphere based on the identified contour; It functions as the predetermined separability filter is a variable elliptical separability filter that changes the major axis and minor axis of an elliptical shape; program.

Images