Image automatic extraction and replay system and method, and storage medium

Abstract

The invention discloses an automatic image extracting and replaying system and method and a storage medium. The method is applied to the image automatic extracting and replaying system to automatically extract and replay images of a ball hitting action, and a signal sensing device of the system is arranged on a handheld ball tool, senses the ball hitting action of a sporter holding the handheld ball tool by hand and outputs a sensing signal. The method comprises the following steps of: extracting an image of a ball hitting action of a sporter by holding a handheld ball tool through an image extraction device; identifying a hitting ball type conforming to the specific ball type from the sensing signal through the electronic device; automatically extracting a ball hitting image corresponding to the ball hitting type from the ball hitting action images extracted by the image extraction device through the electronic device; and playing the ball hitting image.

Description

Technical Field

[0001] The present invention relates to a system, a method and a storage medium, and in particular to an automatic image extraction and replaying system, an automatic image extraction and replaying method and a storage medium. Background Art

[0002] Ball sports, such as badminton, baseball, table tennis, tennis or golf, have always been popular sports. When watching ball sports, if there are exciting hitting actions, such as the smash action in badminton or the swing action in baseball, in order to allow viewers to watch or recall the exciting actions again, it is often necessary to replay the exciting action clips using real-time recorded images.

[0003] However, in the conventional replay of the images of the wonderful actions, the operator has to watch the recorded images and manually find and extract the image segments of the wonderful actions before playing them out, which is quite inconvenient. Summary of the invention

[0004] The purpose of the present invention is to provide a system, method and storage medium for automatically extracting and replaying images. Unlike the traditional manual operation of replaying wonderful actions, the present invention can automatically identify the athlete's hitting action, and automatically extract and play the image of the hitting action, so that viewers can watch or recall the wonderful action again.

[0005] The present invention can also allow athletes and / or coaches to confirm the correctness of exercise posture in real time to avoid injuries, and can also allow athletes and / or coaches to obtain exercise status and conduct exercise evaluation in real time.

[0006] To achieve the above-mentioned purpose, an automatic image extraction and replay system according to the present invention is used to automatically extract and replay images of an athlete holding a handheld ball club and hitting the ball. The system includes a signal sensing device, an image extraction device and an electronic device. The signal sensing device is set on the handheld ball club, and senses the athlete's hitting action of holding the handheld ball club and outputs a sensing signal. The image extraction device extracts the image of the athlete holding the handheld ball club and hitting the ball. The electronic device is coupled to the signal sensing device and the image extraction device respectively, and the electronic device includes one or more processing units and a storage unit, and the one or more processing units are coupled to the storage unit, and the storage unit stores one or more program instructions, wherein when the one or more program instructions are executed by the one or more processing units, the one or more processing units perform the following processing procedures: Processing procedure 1: identifying the type of ball that matches a specific ball type from the sensing signal; Processing procedure 2: automatically extracting the hitting image corresponding to the hitting ball type from the hitting action image extracted by the image extraction device; and Processing procedure 3: playing the hitting image.

[0007] To achieve the above-mentioned purpose, an automatic image extraction and replay method according to the present invention is applied to an automatic image extraction and replay system to automatically extract and replay images of an athlete holding a handheld ball club and hitting the ball. The automatic image extraction and replay system includes a signal sensing device, an image extraction device and an electronic device. The signal sensing device is arranged on the handheld ball club, and senses the athlete's hitting action of holding the handheld ball club and outputs a sensing signal. The electronic device is coupled to the signal sensing device and the image extraction device respectively. The method includes: extracting the image of the athlete's hitting action of holding the handheld ball club by the image extraction device; identifying the hitting ball type that matches the specific ball type from the sensing signal by the electronic device; automatically extracting the hitting image corresponding to the hitting ball type from the hitting action image extracted by the image extraction device by the electronic device; and playing the hitting image.

[0008] In one embodiment, the one or more processing units identify the type of ball being hit from the sensing signal through a hitting ball type identification algorithm, and the hitting ball type identification algorithm includes a motion signal segmentation step, a signal normalization step, a convolutional neural network classification step and a ball type identification step.

[0009] In one embodiment, the shot type is a forehand backcourt serve, a backhand frontcourt serve, a forehand backcourt high and far ball, a backhand backcourt high and far ball, a forehand push and pick backcourt ball, a backhand push and pick backcourt ball, a frontcourt forehand short ball, a frontcourt backhand short ball, a midfield forehand flat drive, a midfield backhand flat drive, a midfield forehand catch and kill to block the ball in front of the net, a midfield backhand catch and kill to block the ball in front of the net, a backcourt forehand cut, a backcourt forehand high and far ball, a backcourt forehand smash or a midfield forehand raid.

[0010] In one embodiment, the system further includes a playback device coupled to the electronic device, and the electronic device plays the ball hitting image through the playback device.

[0011] In one embodiment, the one or more processing units further perform: Processing procedure four: calculating the parameters of the athlete when performing the ball hitting from the sensing signal, and displaying the parameters.

[0012] In one embodiment, the electronic device identifies the type of ball being hit from the sensing signal using a hitting ball type identification algorithm, and the hitting ball type identification algorithm includes an action signal segmentation step, a signal normalization step, a convolutional neural network classification step, and a ball type identification step.

[0013] In one embodiment, the method further comprises: calculating parameters of the player's performance of the ball hitting technique from the sensing signal by an electronic device, and displaying the parameters.

[0014] To achieve the above object, a storage medium having application software stored therein according to the present invention can complete the aforementioned automatic image extraction and replay method when the application software is loaded and executed by a device.

[0015] As described above, in the image automatic extraction and replay system, method and storage medium of the present invention, the image extraction device can extract the image of the striker hitting the ball with a handheld sports equipment, and the electronic device can identify the ball type of the striker's hitting action that conforms to a specific ball type from the sensing signals, and can automatically extract the hitting image corresponding to the ball type from the hitting action images extracted by the image extraction device. After that, the hitting image corresponding to the ball type is played. Thus, different from the traditional operation mode of manually searching for and playing exciting image segments for exciting action replay, the image automatic extraction and replay system, method and storage medium of the present invention can automatically identify the ball type of the hit, and automatically extract and play the hitting image corresponding to the ball type that conforms to a specific ball type, so that viewers can watch or relive the exciting hitting action again. In an application example, the present invention can also enable the striker and / or coach to confirm the correctness of the movement posture during hitting in real time to avoid injury, and at the same time enable the striker and / or coach to obtain the movement status and conduct sports evaluation in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1A FIG. is a functional block diagram of an image automatic extraction and replay system according to an embodiment of the present invention.

[0017] Figure 1B is Figure 1A a functional block diagram of the electronic device in the image automatic extraction and replay system of

[0018] Figure 2A and Figure 2B are respectively signal schematic diagrams of acceleration and angular velocity obtained by the signal sensing device when the striker hits the ball with a handheld sports equipment.

[0019] Figure 3 FIG. is a display schematic diagram of the playback device of an image automatic extraction and replay system according to an embodiment of the present invention.

[0020] Figure 4 FIG. is a schematic diagram of the flow steps of an image automatic extraction and replay method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] Hereinafter, with reference to the related drawings, an image automatic extraction and replay system, an image automatic extraction and replay method and a storage medium according to an embodiment of the present invention will be described, wherein the same elements will be described with the same reference numerals.

[0022] Figure 1A FIG. is a functional block diagram of an image automatic extraction and replay system according to an embodiment of the present invention, Figure 1B is Figure 1A a functional block diagram of the electronic device in the image automatic extraction and replay system of Figure 2A and Figure 2B They are schematic diagrams of acceleration and angular velocity signals obtained by the signal sensing device when an athlete hits the ball with a handheld ball club, and Figure 3 FIG. 1 is a display diagram of a playback device of an automatic image extraction and playback system according to an embodiment of the present invention. First, Figure 2A and Figure 2B Schematic diagrams of waveforms of the sensing signal SS of a hitting action of the handheld golf club are respectively shown.

[0023] The automatic image extraction and replay system 1 of the present invention can be applied to automatically extract and replay images of athletes holding handheld ball clubs to hit the ball. Here, the hitting sport can be, for example, but not limited to sports such as badminton, baseball, table tennis, tennis or golf, or other sports that use handheld ball clubs to swing. The above-mentioned handheld ball clubs can be badminton rackets, bats, tennis rackets, table tennis rackets, or golf clubs, or other ball clubs for hitting the ball. The handheld ball clubs in the following embodiments are taken as an example of badminton rackets. Therefore, the "hitting" action that appears below is the action of "swinging a badminton racket", or the "swinging" action. Of course, if applied to baseball or golf, the hitting action is the action of swinging a bat or a golf club, and so on. In addition, the "athlete" that appears below refers to a person who uses a handheld ball club to perform a hitting action.

[0024] Please refer to Figure 1A and Figure 1B The automatic image extraction and replaying system 1 includes a signal sensing device 11, an image extraction device 12 and an electronic device 13. In addition, the automatic image extraction and replaying system 1 of this embodiment may further include a playback device 14.

[0025] The signal sensing device 11 is arranged in a handheld ball tool. The signal sensing device 11 can sense the hitting action (such as a swing action such as a smash or a slice) of the player holding the handheld ball tool (such as a badminton racket) and output a sensing signal SS. The signal sensing device 11 is, for example but not limited to, arranged in the handle or club head (such as a golf club) of the handheld ball tool (such as a badminton racket, a bat, a tennis racket or a table tennis racket). Taking a badminton racket as an example, the signal sensing device 11 can be, for example, arranged in the handle of the badminton racket, or in the back cover of the handle, but it is not limited thereto. In different embodiments, the signal sensing device 11 can also be installed in other parts of the handheld ball tool, and it is not limited thereto. The following embodiments take the signal sensing device 11 being arranged in the handle of the badminton racket as an example. Therefore, when the player holds the badminton racket and performs a swing action, the signal sensing device 11 can sense the player's hitting (swinging) action and output a sensing signal SS corresponding to the hitting (swinging) action. Of course, when there are multiple hitting (swinging) actions within a period of time, the output signal is also called the sensing signal SS.

[0026] The signal sensing device 11 of this embodiment may include an inertial sensor, such as but not limited to a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer, so as to obtain an accurate hitting (swinging) action. Therefore, the sensing signal SS is an inertial sensing signal, which may include an acceleration signal (e.g. Figure 2A as shown), angular velocity signals (e.g. Figure 2B In one embodiment, a six-axis sensor (such as ICM-20649) and a three-axis magnetometer (such as LIS2MDL) including an accelerometer and a gyroscope can be used as a nine-axis inertial sensor. The accelerometer is used to sense the earth's gravity and the acceleration generated by the motion; the gyroscope is used to sense the angular velocity generated by the motion; and the magnetometer is used to sense the earth's magnetic field vector, and the azimuth information can be obtained after calculation.

[0027] In one embodiment, the signal sensing device 11 may further include a microcontroller unit and a power supply unit. The microcontroller unit may extract and collect the sensing signal SS generated by the inertial sensor (speedometer, gyroscope and magnetometer) due to the hitting action and process it (e.g., temporarily store and encode it). The processed sensing signal SS may be wirelessly transmitted to the electronic device 13 in batches through, for example, a Wi-Fi module or a Bluetooth module, so as to analyze the hitting action and the type of ball being hit; and the power supply unit may be, for example, a lithium battery, which may provide the power required for the signal sensing device 11 to operate.

[0028] The image capturing device 12 can capture images of an athlete holding a handheld club and hitting the ball. Here, when an athlete holds a handheld club and hits the ball, because each action is captured by the image capturing device 12, each hitting action can correspond to a hitting image and a signal waveform. In one embodiment, at least one image capturing device 12 (such as a high-speed camera) can be set up on the sports field to capture images of the athlete's hitting action. In one embodiment, multiple image capturing devices 12 can be set up in different directions to capture hitting actions at different directions or angles. The present invention does not limit the number of image capturing devices 12 and their installation directions, as long as the image of the athlete's hitting action can be clearly captured.

[0029] The electronic device 13 is coupled to the signal sensing device 11 and the image capturing device 12, respectively. The electronic device 13 may be a computer, a server, or a cloud device (e.g., a remote server). In one embodiment, the electronic device 13 and the signal sensing device 11 may be electrically coupled by wireless coupling, thereby receiving, storing, and processing the sensing signal SS output by the signal sensing device 11. In one embodiment, the electronic device 13 and the image capturing device 12 may be electrically coupled by wire or wireless coupling, thereby processing (e.g., capturing) the image of the hitting action. The wireless coupling method may be wirelessly coupled, for example, via a Wi-Fi module or a Bluetooth module.

[0030] The electronic device 13 of this embodiment may include one or more processing units 131 and a storage unit 132. The one or more processing units 131 are coupled to the storage unit 132. Figure 1B As shown, it takes a processing unit 131 and a storage unit 132 as an example. Among them, the processing unit 131 can access the data stored in the storage unit 132, and can include the core control components of the electronic device 13, for example, it can include at least one central processing unit (CPU) and a memory, or include other control hardware, software or firmware. When the signal sensing device 11 senses the player's hitting action and outputs the sensing signal SS and transmits it to the electronic device 13, it can be stored in the storage unit 132 for processing and analysis by the processing unit 131. In addition, the storage unit 132 can be a non-transitory computer readable storage medium, for example, it can include at least one memory, a memory card, an optical disc, a video tape, a computer tape, or any combination thereof. In one embodiment, the aforementioned memory can include a read-only memory (ROM), a flash memory, a field-programmable gate array (FPGA), or a solid state disk (SSD), or other forms of memory, or a combination thereof.

[0031] In addition, the storage unit 132 may also store at least one application software, and the application software may include one or more program instructions 1321. When the one or more program instructions 1321 of the application software are executed by the one or more processing units 131, the one or more processing units 131 at least perform processing procedures 1 to 3. Among them, processing procedure 1 is: identifying the type of ball that matches the specific type of ball from the sensing signal SS. Processing procedure 2 is: automatically extracting the hitting image corresponding to the hitting ball type from the hitting action image extracted by the image extraction device 12. And processing procedure 3 is: playing the hitting image. The technical contents of processing procedures 1 to 3 are described in detail below.

[0032] However, before the first processing procedure is performed, in order to make the subsequent analysis, processing and result of the sensing signal SS more accurate, the inertial signal output by the signal sensing device 11 needs to be subjected to a signal pre-processing procedure, and the signal pre-processing procedure may include a signal correction step and a signal filtering step. Here, the signal correction step may correct the inertial signal, and the signal filtering step may filter out the noise in the inertial signal, so that the ball type identification result obtained based on the sensing signal SS is more accurate.

[0033] In the first processing procedure, since the signal sensing device 11 is disposed in the handheld golf club, when the athlete holds the handheld golf club to perform a hitting (swinging) action, the signal sensing device 11 can output a sensing signal SS corresponding to the hitting (swinging) action (e.g. Figure 2A and Figure 2B ), therefore, the electronic device 13 (processing unit 131) can identify the batting action of the athlete that meets the batting ball type of the specific ball type according to the user's (viewer's) demand for the specific ball type from the sensing signal SS, and then automatically extract the batting image corresponding to the batting ball type from the batting action image extracted by the image extraction device 12 (processing procedure two), and then play the batting image (processing procedure three).

[0034] Specifically, the sensing signal SS generated during a series of hitting actions by the athlete on the court may include many different types of hitting balls (e.g., including smash, slice, lob, etc.), and each type of hitting ball (hitting action) corresponds to a type of sensing signal SS waveform. Of course, each type of hitting ball will also have its corresponding hitting image (segment) extracted and stored by the image extraction device 12, and the processing unit 131 can identify the different types of signal waveforms appearing in the sensing signal SS to obtain the type of hitting ball represented by the signal waveform. For example, if the user (viewer) wants to watch, for example, a "backcourt forehand smash", the processing unit 131 can find the signal waveform of the "backcourt forehand smash" from the sensing signal SS, and automatically extract the hitting image segment corresponding to the signal waveform of the "backcourt forehand smash", and then play the segment of the hitting image.

[0035] To achieve this goal, the electronic device 13 (processing unit 131) must first identify the type of ball type of each hitting action of the athlete according to the waveform of the sensing signal SS, so as to find the hitting ball type that matches the specific ball type from these hitting ball types. Here, the one or more processing units 131 identify the hitting ball type that matches the specific ball type from the sensing signal SS through a hitting ball type recognition algorithm. The hitting ball type recognition algorithm may include an action signal segmentation step, a signal normalization step, a convolutional neural network classification step, and a ball type recognition step.

[0036] The steps of motion signal segmentation are as follows: when the swing action is in progress, there will be a static motion interval before and after the action begins. At this time, because the signal sensing device 11 is stationary, the three-axis combined force values ​​of the accelerometer and the gyroscope are all 0. Therefore, the swing action interval can be detected by setting a dynamic threshold value. For example, the standard score (z-score) is calculated based on the sensing signal of the first 200 sampling points, and the standard score is used as the dynamic threshold value, thereby correctly segmenting the signal of the swing action.

[0037] The signal normalization step is: after the inertial sensing signal output by the hitting (swinging) action has gone through the above-mentioned signal correction step, signal filtering step and action signal segmentation step, the signal normalization step can be performed to normalize the sensing signal SS.

[0038] The convolution neural network classification step and the ball type identification step are as follows: in the sensing signal SS after the signal normalization step, the three-axis angular velocity signal in each batting swing action signal can be used as the input of the convolution neural network (CNN) classifier, and then the batting ball type is identified by the sensing signal SS. In this embodiment, the batting ball type can be, for example, a forehand backcourt serve, a backhand frontcourt serve, a forehand backcourt high and far ball, a backhand backcourt high and far ball, a forehand push backcourt ball, a backhand push backcourt ball, a frontcourt forehand short ball, a frontcourt backhand short ball, a midfield forehand flat shot, a midfield backhand flat shot, a midfield forehand receiving and killing the ball in front of the net, a midfield backhand receiving and killing the ball in front of the net, a backcourt forehand cut, a backcourt forehand high and far ball, a backcourt forehand kill or a midfield forehand surprise ball, a total of sixteen batting ball types. Here, the architecture of the convolutional neural network classifier may include two convolutional layers, two pooling layers, a fully connected layer, and an output layer, as detailed below:

[0039] Convolution layer: Each convolution layer contains multiple convolution kernels. By setting the convolution kernel size and using the convolution principle to gradually slide the window and weightedly calculate the value in each area, the output of the convolution layer is obtained through activation function calculation, thereby extracting important information from the input signal. Among them, 128 convolution kernels (convolutional kernel / filter) of size 1×5 are set in each convolution layer to extract image features.

[0040]

[0041] in, is the input vector composed of the three-axis angular velocity signals; i is the index of the data point in each step window; N is the number of data points in each step window; l is the index of the layer; M is the convolution kernel size (kernel / filter size); is the bias weight of the kth feature map of the lth layer; For input The connection weight with the kth feature image of the lth layer; ReLU is the linear rectification activation function.

[0042] Pooling layer: It mainly uses the output of the convolutional layer as its input and performs downsampling. Here, the maximum pooling operation is used to reduce the feature image dimension (network training parameters) and only retain the important features in the input image. The pooling size is 1×2 and the stride is 2. Among them, R is the pooling size; T is the span of the pooling

[0043]

[0044] Fully connected layer: flattens the features obtained after multiple convolutional layers and pooling layers into a feature vector p l =[p 1 ,p 2 ,…,p g ] as the input of this layer, where g is the number of neurons in the last pooling layer, and the following operations are performed:

[0045]

[0046] in, is the connection weight between the gth neuron in the l-1th layer and the hth neuron in the lth layer in the fully connected layer; is the bias value of the hth neuron in the lth layer of the fully connected layer; ReLU is the linear rectification activation function. Finally, That is the deep feature obtained through convolutional neural network operation.

[0047] Output layer: usually a classifier is used. Here, the Softmax classifier is used. The Softmax classifier is based on the log-sigmoid function. The X-axis range is positive infinity to negative infinity, and the Y-axis range is 0 to 1. By mapping the output of the fully connected layer to the [0,1] interval, the obtained value is converted into the corresponding probability, and the maximum value is taken as the classification result.

[0048] Since the batting ball type of each batting action has been identified by using the batting ball type identification algorithm according to the sensing signal SS, the electronic device 13 (processing unit 131) can find a batting ball type that matches a specific ball type that the user (viewer, such as an athlete, coach or other person) wants to watch among these batting ball types, and automatically extract the batting image corresponding to the batting ball type from the batting action image extracted by the image extraction device 12 (processing procedure 2). For example, if the viewer wants to watch the wonderful action of the athlete when smashing the ball, the electronic device 13 (processing unit 131) can find the batting image (segment) corresponding to the smash action among these batting ball types swung by the athlete, and broadcast it in processing procedure 3.

[0049] In addition, please refer to Figure 1A , the playback device 14 is coupled to the electronic device 13, and the electronic device 13 can play the batting image through the playback device 14. Here, the electrical coupling method between the playback device 14 and the electronic device 13 can be wired or wireless. In one embodiment, the playback device 14 can be, for example, an image playback computer or a video player, but is not limited. Therefore, the batting image clip that matches the specific ball type can be played through the playback device 14 (processing procedure three) so that the user (viewer) can watch it. In one embodiment, if the athlete has more than one image clip of the batting ball type that matches the specific ball type, these image clips of the batting ball type can be played in sequence.

[0050] In actual application, in order to avoid missing wonderful scenes, the batting video (segment) extracted by the electronic device 13 can be added with the video before and after the batting action, for example, 0.5 seconds. For example, if the batting video (segment) extracted starts at 3 minutes 25 seconds and ends at 3 minutes 26.5 seconds, the batting video actually played can start at 3 minutes 24.5 seconds and end at 3 minutes 27 seconds.

[0051] In addition to the above-mentioned processing procedures 1 to 3, the one or more processing units 131 can further perform processing procedure 4: calculating the parameters of the player's hitting action from the sensing signal SS and displaying the parameters. Figure 3 As shown, in addition to playing the video clip of the athlete 1 hitting the ball in accordance with the specific ball type, the electronic device 1 can also display some important parameters of the hitting ball type through the playback device 14, such as the "swing speed" and "swing force" (and the hitting ball type) of the hitting ball type. Of course, the parameters displayed by the playback device 14 are not limited to these.

[0052] Figure 4 The figure is a schematic diagram of the process steps of an automatic image extraction and replaying method according to an embodiment of the present invention.

[0053] The present invention also provides an automatic image extraction and replay method, which can be applied to the automatic image extraction and replay system 1 to automatically extract and replay the image of the athlete holding the handheld ball club and hitting the ball. The components and functions of the automatic image extraction and replay system 1 have been described in detail above and will not be further described here. Figure 4 As shown, the automatic image extraction and replaying method of this embodiment may include steps S01 to S05. Steps S01 to S05 may be implemented in software program form, or in hardware or firmware form, which is not limited by the present invention.

[0054] Step S01 is: using the image capture device to capture the image of the athlete holding the handheld ball tool and hitting the ball. Step S02 is: using the electronic device to identify the type of ball that matches the specific type of ball from the sensing signal. Step S03 is: using the electronic device to automatically extract the hitting image corresponding to the type of ball from the hitting action image captured by the image capture device. And step S04 is: playing the hitting image. Finally, step S05 is: using the electronic device to calculate the parameters of the athlete performing the hitting of the ball from the sensing signal, and displaying the parameters. In actual applications, steps S01 and S02 can be performed separately or simultaneously.

[0055] Here, the technical contents of step S01 to step S05 of the automatic image extraction and replaying method have been described in detail in the automatic image extraction and replaying system 1 mentioned above, and will not be repeated here.

[0056] In addition, the present invention also provides a storage medium having application software stored therein, and when the application software is loaded and executed by a device, the aforementioned method for automatic image extraction and replay can be completed. Here, the device can be an electronic device of any type or kind, such as a computer, a server, or a mobile electronic device. In one embodiment, the storage medium can be a non-transient computer-readable storage medium, such as at least one memory, a memory card, an optical disk, a videotape, a computer tape, or any combination thereof. Among them, the memory can include a read-only memory, a flash memory, a field programmable gate array or a solid-state hard disk, or other forms of memory, or a combination thereof. In one embodiment, the storage medium can be an internal memory of a computer or a server; or, the storage medium can also be a cloud storage located in a cloud device, so the application software can also be stored in the cloud device, and then loaded and executed by the cloud device.

[0057] As mentioned above, it can be seen from the above disclosure that, unlike the traditional operation mode of manually searching for wonderful video clips and playing them for replaying wonderful actions, the automatic image extraction and replaying system, automatic image extraction and replaying method and storage medium of the present invention can automatically identify the type of ball that the athlete hits from the sensing signal, and automatically extract and play the hitting image corresponding to the specific ball type according to the specific ball type, so that the viewer can watch or recall the wonderful hitting action again. In one application example, the athlete and / or coach can also confirm the correctness of the sports posture in real time to avoid injury, and at the same time, the athlete and / or coach can obtain the sports status and conduct sports evaluation in real time.

[0058] In summary, in the automatic image extraction and replay system, method and storage medium of the present invention, the image extraction device can extract the image of the athlete holding the handheld ball club to hit the ball, and the electronic device can identify the type of ball that matches the specific ball type in the athlete's hitting action from the sensing signal, and can automatically extract the hitting image corresponding to the hitting ball type from the hitting action image extracted by the image extraction device, and then play the hitting image corresponding to the hitting ball type. Therefore, unlike the traditional operation method of manually searching for wonderful image clips and playing the replay of wonderful actions, the automatic image extraction and replay system, method and storage medium of the present invention can automatically identify the type of ball that hits the ball, and automatically extract and play the hitting image corresponding to the hitting ball type that matches the specific ball type, so that the viewer can watch or recall the wonderful hitting action again. In an application example, the present invention can also allow the athlete and / or coach to confirm the correctness of the sports posture when hitting the ball in real time to avoid injury, and at the same time, it can also allow the athlete and / or coach to obtain the sports status and conduct sports evaluation in real time.

[0059] The above description is for illustrative purposes only and is not intended to be limiting. Any equivalent modifications or changes made to the present invention without departing from the spirit and scope of the present invention should be included in the scope of the appended claims.

Claims

1. An automatic image extraction and replay system, used for automatically extracting and replaying images of a player holding a handheld ball hitting action, the system include: A signal sensing device is provided on the handheld golf club, and senses the hitting action of the player holding the handheld golf club and outputs a sensing signal; An image capturing device for capturing an image of the player holding the handheld ball club and hitting the ball; and An electronic device is coupled to the signal sensing device and the image capturing device, respectively, and includes one or more processing units and a storage unit. The one or more processing units are coupled to the storage unit, and the storage unit stores one or more program instructions. When the one or more program instructions are executed by the one or more processing units, the one or more processing units perform the following processing procedures: Processing procedure 1: identifying the type of ball that matches the specific type of ball from the sensing signal; Processing procedure 2: automatically extracting the hitting image corresponding to the hitting ball type from the hitting action image extracted by the image extraction device; and Processing procedure three: Play the ball hitting image.

2. The system according to claim 1, in, The one or more processing units identify the type of ball being hit from the sensing signal through a hitting ball type identification algorithm, wherein the hitting ball type identification algorithm includes an action signal segmentation step, a signal normalization step, a convolutional neural network classification step, and a ball type identification step.

3. The system according to claim 1, wherein the types of shots are forehand backcourt serve, backhand frontcourt serve, forehand backcourt high and far ball, backhand backcourt high and far ball, forehand push backcourt ball, backhand push backcourt ball, frontcourt forehand short ball, frontcourt backhand short ball, midfield forehand flat drive, midfield backhand flat drive, midfield forehand catch and kill to block the ball in front of the net, midfield backhand catch and kill to block the ball in front of the net, backcourt forehand cut, backcourt forehand high and far ball, backcourt forehand smash or midfield forehand surprise ball.

4. The system according to claim 1, further comprising: include: a playback device, coupled to the electronic device, Wherein, the electronic device plays the ball hitting image through the playing device.

5. The system according to claim 1, in, The one or more processing units further perform: Processing procedure 4: Calculate the parameters of the player when performing the hitting action from the sensing signal, and display the parameters.

6. An automatic image extraction and replay method, applied to an automatic image extraction and replay system to automatically extract and replay an image of an athlete holding a handheld ball club and hitting a ball. The automatic image extraction and replay system comprises a signal sensing device, an image capturing device and an electronic device. The signal sensing device is disposed on the handheld ball club and senses the athlete holding the handheld ball club and hitting a ball and outputs a sensing signal. The electronic device is coupled to the signal sensing device and the image capturing device respectively. include: The image capturing device captures an image of the player holding the handheld ball tool and hitting the ball; Identifying the type of ball that matches the specific type of ball from the sensing signal by the electronic device; Automatically extracting, by the electronic device, a hitting image corresponding to the hitting ball type from the hitting action image captured by the image capturing device; and Play the ball hitting video.

7. The method according to claim 6, wherein the electronic device identifies the type of ball being hit from the sensing signal through a ball type identification algorithm, and the ball type identification algorithm includes a motion signal segmentation step, a signal normalization step, a convolutional neural network classification step and a ball type identification step.

8. The method according to claim 6, wherein the shot type is a forehand backcourt serve, a backhand frontcourt serve, a forehand backcourt high and far ball, a backhand backcourt high and far ball, a forehand push and pick backcourt ball, a backhand push and pick backcourt ball, a frontcourt forehand short ball, a frontcourt backhand short ball, a midfield forehand flat drive, a midfield backhand flat drive, a midfield forehand catch and kill to block the ball in front of the net, a midfield backhand catch and kill to block the ball in front of the net, a backcourt forehand cut, a backcourt forehand high and far ball, a backcourt forehand smash or a midfield forehand raid ball.

9. The method according to claim 6, wherein the automatic image extraction and replay system further comprises a playback device, the playback device is coupled to the electronic device, and the playback device plays the shot image.

10. The method according to claim 6, further comprising: include: The electronic device calculates the parameters of the player's hitting action from the sensing signal and displays the parameters.

11. A storage medium having application software stored therein, which, when loaded and executed by a device, can implement the automatic image extraction and replay method according to any one of claims 6 to 10.

Images