Method, system and device for determining highlights, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a wonderful fragment determination method, system and device, electronic equipment and a storage medium, and relates to the technical field of video processing, and the method comprises the steps: determining a round starting moment in a game video based on a ball hitting action detection result of a video frame in the game video of a billiard ball; motion tracking is conducted on balls on the billiard table top in the video frames from the turn starting moment, and the turn ending moment corresponding to the turn starting moment is determined based on the video frames where all the balls are tracked to be in the static state; performing video content analysis on the first round fragment to obtain a first analysis result describing whether the first ball hitting object in the first round goals or not; and if the first round clip is not the first round clip of the game video, determining a wonderful clip in the game video based on the first analysis result and the second analysis result. In this way, the efficiency of determining the wonderful clips in the billiard game video can be improved.

Description

Technical Field

[0001] This application relates to the field of video processing technologies, and in particular, to a method, system, device, electronic device, and storage medium for determining highlight segments. Background Art

[0002] Billiards is a popular sport among people, and billiards competitions often draw much attention. To enable people to better appreciate the highlight collections of billiards, sports media workers, video creators, etc. need to watch billiards videos, then determine the highlight segments in the billiards videos, and then complete manual editing based on the highlight segments.

[0003] However, the above process of determining highlight segments requires sports media workers, video creators, etc. to watch the videos personally and manually determine the highlight segments, which is time-consuming and inefficient. Summary of the Invention

[0004] The purpose of the embodiments of this application is to provide a method, system, device, electronic device, and storage medium for determining highlight segments, so as to improve the efficiency of determining highlight segments in billiards game videos. The specific technical solutions are as follows:

[0005] In the first aspect of the embodiments of this application, a method for determining highlight segments is provided. The method includes:

[0006] Based on the detection result of the hitting action of video frames in the billiards game video, determine the starting moment of a round in the game video;

[0007] Starting from the starting moment of the round, perform motion tracking on the balls on the billiards table in the video frames, and based on the video frames when all the tracked balls are in a stationary state, determine the ending moment of the round corresponding to the starting moment of the round;

[0008] Perform video content analysis on the first-round segment to obtain a first analysis result describing whether the first hitting object scores a goal in the first round, where the first-round segment is: the video segment from the starting moment of the round to the ending moment of the round;

[0009] If the first-round segment is not the first-round segment of the game video, then based on the first analysis result and the second analysis result, determine the highlight segments in the game video, where the second analysis result is: the result of performing video content analysis on the second-round segment before the first-round segment.

[0010] In a possible implementation manner, the determining the highlight segments in the game video based on the first analysis result and the second analysis result includes:

[0011] If the first analysis result indicates that the first hitting object fails to score a goal, then based on the second analysis result, starting from the third round segment, search forward for the second round segment in which the first hitting object scores consecutive goals, where the third round segment is: the second round segment before and adjacent to the first round segment;

[0012] If the number of found second round segments is greater than the first number, determine the found second round segments as the wonderful segments in the game video.

[0013] In a possible implementation manner, the determining the wonderful segments in the game video based on the first analysis result and the second analysis result includes:

[0014] If the first analysis result indicates that the first hitting object scores a goal, then based on the first analysis result, determine whether the ball hit by the first hitting object is the last hittable ball on the billiard table;

[0015] If so, based on the second analysis result, starting from the first round segment, search forward for the second round segment in which the first hitting object scores consecutive goals;

[0016] If the number of found second round segments is greater than the first number, determine the found second round segments as the wonderful segments in the game video.

[0017] In a possible implementation manner, the determining the starting moment of a round in the game video based on the hitting action detection result of video frames in the billiard game video includes:

[0018] Perform hitting action recognition on the video frames in the billiard video to obtain the hitting segments and hitting types corresponding to the recognized hitting actions;

[0019] Determine the hitting moment based on the hitting segment corresponding to the first hitting action, where the hitting type of the first hitting action is the opening shot;

[0020] For each hitting moment, if there is another hitting moment after this hitting moment, determine the video segment between this hitting moment and the adjacent hitting moment after this hitting moment as the game video. If there is no other hitting moment after this hitting moment, determine the video segment between this hitting moment and the end of the billiard video as the game video;

[0021] Based on the hitting segments corresponding to the hitting actions in the game video, determine the starting moment of a round in the game video.

[0022] In a possible implementation manner, before determining the hitting moment based on the hitting segment corresponding to the first hitting action, it further includes:

[0023] Find the hitting actions whose frame distances between the hitting segments corresponding to the recognized hitting actions are less than the first frame distance;

[0024] Merge the found hitting actions, and determine the hitting segments and hitting types corresponding to the merged hitting actions based on the hitting segments and hitting types corresponding to the found hitting actions.

[0025] In a possible implementation manner, the recognition of the hitting actions for the video frames in the billiards video to obtain the hitting segments and hitting types corresponding to the recognized hitting actions includes:

[0026] Extract features from the video frames in the billiards video;

[0027] Based on the extracted features, determine the video frames with hitting actions in the billiards video;

[0028] According to the arrangement order of the determined video frames in the billiards video, determine the hitting segments corresponding to the hitting actions in the billiards video;

[0029] According to the distribution information of the balls on the billiards table in the video frames within the hitting segments corresponding to the hitting actions, obtain the hitting types corresponding to the hitting actions.

[0030] In a possible implementation manner, when the first analysis result indicates that the first hitting object scores a goal, the first analysis result further includes: description information of the pocketed ball;

[0031] The description information of the pocketed ball includes at least one of the following information:

[0032] The color of the pocketed ball, the number of the pocketed ball, the pocket number of the pocketed ball, the goal time of the pocketed ball.

[0033] In a possible implementation manner, after determining the wonderful segment, it further includes:

[0034] Based on at least one of the following information, determine the wonderful label of the wonderful segment:

[0035] The description information of the pocketed ball included in the analysis result of the video content of the round segment in the wonderful segment;

[0036] Whether the wonderful segment includes the first round segment in the game video;

[0037] The goal complexity of the pocketed ball within the round segment in the wonderful segment;

[0038] The number of goals in the wonderful segment.

[0039] In a possible implementation, the video content analysis of the first-round segment to obtain the first analysis result describing whether the first hitting object scores a goal in the first round includes:

[0040] Identify the hitting object in the hitting segment of the first-round segment to obtain the description information of the first hitting object;

[0041] Based on the movement path of the ball in the first-round segment, determine whether a goal is scored, where the movement path of the ball is obtained by tracking the movement of the ball on the billiard table;

[0042] Generate the first analysis result based on the obtained description information and the goal judgment result.

[0043] In the second aspect of the embodiments of the present application, a wonderful segment determination system is provided, and the system includes:

[0044] An image acquisition device, which is set in the billiard game venue and is used to acquire the game video of the billiards. If the image acquisition device does not have the image intelligent analysis ability, the game video will be sent to the backend device. If the image acquisition device has the image intelligent analysis ability, the wonderful segment determination method described in any one of the first aspects above will be executed;

[0045] The backend device is a service device set locally in the billiard game venue or a remote server, and is used to receive and store the game video acquired by the image acquisition device, and execute the wonderful segment determination method described in any one of the first aspects above.

[0046] In the third aspect of the embodiments of the present application, a wonderful segment determination device is provided, and the device includes:

[0047] The first determination module is used to determine the starting moment of the round in the game video based on the hitting action detection result of the video frame in the billiards game video;

[0048] The second determination module is used to start from the starting moment of the round, track the movement of the balls on the billiard table in the video frame, and determine the ending moment of the round corresponding to the starting moment of the round based on the video frame when all the tracked balls are in a static state;

[0049] The analysis module is used to perform video content analysis on the first-round segment to obtain the first analysis result describing whether the first hitting object scores a goal in the first round, where the first-round segment is the video segment from the starting moment of the round to the ending moment of the round;

[0050] A third determination module, configured to, if the first round segment is not the first round segment of the game video, determine an exciting segment in the game video based on the first analysis result and the second analysis result, where the second analysis result is the result of video content analysis of a second round segment before the first round segment.

[0051] In a possible implementation manner, the third determination module is specifically configured to: if the first analysis result indicates that the first hitting object does not score a goal, based on the second analysis result, search forward from the third round segment to find a second round segment in which the first hitting object continuously scores a goal, where the third round segment is the second round segment adjacent to and before the first round segment; if the number of found second round segments is greater than a first number, determine the found second round segments as the exciting segments in the game video.

[0052] In a possible implementation manner, the third determination module is specifically configured to: if the first analysis result indicates that the first hitting object scores a goal, determine whether the ball hit by the first hitting object is the last hittable ball on the billiard table based on the first analysis result; if so, based on the second analysis result, search forward from the first round segment to find a second round segment in which the first hitting object continuously scores a goal; if the number of found second round segments is greater than a first number, determine the found second round segments as the exciting segments in the game video.

[0053] In a possible implementation manner, the first determination module is specifically configured to: identify a hitting action for video frames in a billiards video to obtain a hitting segment and a hitting type corresponding to the identified hitting action; determine a hitting moment based on the hitting segment corresponding to the first hitting action, where the hitting type of the first hitting action is a break shot; for each hitting moment, if there is another hitting moment after this hitting moment, determine the video segment between this hitting moment and the adjacent hitting moment after this hitting moment as the game video, and if there is no other hitting moment after this hitting moment, determine the video segment between this hitting moment and the end of the billiards video as the game video; determine the starting moment of a round in the game video based on the hitting segments corresponding to the hitting actions in the game video.

[0054] In a possible implementation manner, the device further includes a merging module, configured to, before determining the hitting moment based on the hitting segment corresponding to the first hitting action, search for hitting actions with an inter-frame distance less than a first inter-frame distance between the hitting segments corresponding to the identified hitting actions; merge the found hitting actions, and determine the hitting segment and the hitting type corresponding to the merged hitting action based on the hitting segments and the hitting types corresponding to the found hitting actions.

[0055] In a possible implementation manner, the first determination module is specifically configured to: extract features from video frames in the billiards video; determine video frames with a hitting action in the billiards video based on the extracted features; determine a hitting segment corresponding to the hitting action in the billiards video according to the arrangement order of the determined video frames in the billiards video; and obtain a hitting type corresponding to the hitting action according to the distribution information of the balls on the billiards table in the video frames within the hitting segment corresponding to the hitting action.

[0056] In a possible implementation manner, when the first analysis result indicates that the first hitting object scores a goal, the first analysis result further includes: description information of the pocketed ball; the description information of the pocketed ball includes at least one of the following information: the color of the pocketed ball, the number of the pocketed ball, the pocket number of the pocketed ball, and the goal-scoring moment of the pocketed ball.

[0057] In a possible implementation manner, the device further includes a fourth determination module, configured to, after determining the highlight segment, determine a highlight label of the highlight segment based on at least one of the following information: the description information of the pocketed ball included in the video content analysis result of the round segment in the highlight segment; whether the highlight segment includes the first round segment in the game video; the complexity of the goal scored by the pocketed ball within the round segment in the highlight segment; and the number of goals scored in the highlight segment.

[0058] In a possible implementation manner, the analysis module is specifically configured to: identify a hitting object in the hitting segment of the first round segment to obtain description information of the first hitting object; determine whether a goal is scored based on the movement path of the balls in the first round segment, where the movement path of the balls is obtained by tracking the movement of the balls on the billiards table; and generate a first analysis result based on the obtained description information and the goal determination result.

[0059] In the fourth aspect of the embodiments of the present application, an electronic device is provided, including:

[0060] A memory for storing a computer program;

[0061] A processor, configured to implement the highlight segment determination method according to any one of the first aspects when executing the program stored in the memory.

[0062] In the fifth aspect of the embodiments of the present application, a computer-readable storage medium is provided, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the highlight segment determination method according to any one of the first aspects is implemented.

[0063] In a sixth aspect of the embodiments of the present application, there is provided a computer program product containing instructions, which when running on a computer, causes the computer to execute the highlight segment determination method described in any one of the above first aspects.

[0064] Advantages of the embodiments of the present application:

[0065] In the technical solution provided by the embodiments of the present application, the electronic device first determines the starting moment of a round based on the hitting action detection result of video frames in the billiards game video, and then tracks the balls on the billiards table to determine the ending moment of the round. Based on the above starting moment and ending moment of the round, each round segment can be determined. Furthermore, the electronic device can perform video content analysis on each round segment and combine the analysis results in each round segment to determine the highlight segments in the game video. In this way, there is no need to manually determine the highlight segments, which improves the efficiency of determining the highlight segments in the billiards game video.

[0066] Of course, when implementing any product or method of the present application, it is not necessarily required to achieve all the above-mentioned advantages at the same time. Description of the Drawings

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application, and those of ordinary skill in the art can also obtain other embodiments based on these drawings.

[0068] Figure 1 It is a schematic flowchart of a method for determining highlight segments provided by the embodiments of the present application;

[0069] Figure 2 It is a schematic diagram of the starting moment and ending moment of a round provided by the embodiments of the present application;

[0070] Figure 3 It is a first refined flowchart of step S104 provided by the embodiments of the present application;

[0071] Figure 4 It is a second refined flowchart of step S104 provided by the embodiments of the present application;

[0072] Figure 5 It is a first refined flowchart of step S101 provided by the embodiments of the present application;

[0073] Figure 6 It is a second refined flowchart of step S101 provided by the embodiments of the present application;

[0074] Figure 7A schematic diagram of a hitting segment provided by an embodiment of the present application;

[0075] Figure 8 A refined process schematic diagram of step S501 provided by an embodiment of the present application;

[0076] Figure 9 A refined process schematic diagram of step S103 provided by an embodiment of the present application;

[0077] Figure 10 A process schematic diagram for determining exciting segments in an example provided by an embodiment of the present application;

[0078] Figure 11 A structural schematic diagram of an exciting segment determination system provided by an embodiment of the present application;

[0079] Figure 12 A structural schematic diagram of an exciting segment determination device provided by an embodiment of the present application;

[0080] Figure 13 A structural schematic diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0081] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art based on the present application belong to the scope of protection of the present application.

[0082] Billiards is a popular sport, and billiards events are often highly concerned by people. In order to enable people to better appreciate the highlights of billiards, sports media workers, video creators, etc. need to watch billiards videos, then determine the exciting segments in the billiards videos, and then complete manual editing based on the exciting segments. The exciting segments can include continuous goal-scoring segments, clearing the table in one shot segments, etc. Among them, continuous goal-scoring means that the same hitting object hits multiple hittable balls in multiple consecutive own turns. Clearing the table in one shot means that the same hitting object hits all the hittable balls in the current game in multiple consecutive own turns and wins.

[0083] However, the above process of determining the exciting segments requires sports media workers, video creators, etc. to watch the videos and manually determine the exciting segments, which is time-consuming and inefficient.

[0084] To improve the efficiency of determining the exciting segments in billiards game videos, an embodiment of the present application provides an exciting segment determination method, as Figure 1 shown, the method includes the following steps S101 - S104.

[0085] For ease of description, in the following, an electronic device is used as the execution subject for illustration. The electronic device may be a computer, a server, or the like.

[0086] Step S101: Based on the hitting action detection result of the video frames in the billiards game video, determine the starting moment of a round in the game video.

[0087] The game video is a video recording containing a billiards game. The video screen includes the hitting object, the billiards table, and the billiards on the table. A billiards game is a process from the start of the opening shot to the moment when one of the two players in the game wins. A round in billiards is a time period from the moment when the hitting object holds the cue stick to hit the ball until all the billiards on the table are in a stationary state. There is only one hitting action at the starting moment of a round. The starting moment of a round is the moment when the hitting object holds the cue stick to hit the ball.

[0088] The electronic device can obtain the recorded game video from the image acquisition device that records the billiards game. The image acquisition device can be a camera, a mobile phone, or any other device with image acquisition function, and there is no limitation on this.

[0089] The hitting action detection result can be the detection result obtained by using an action recognition algorithm to recognize the hitting action in the video frames of the game video. This detection result reflects whether there is a hitting action in the video frame. The electronic device can obtain the above detection result, that is, the hitting action detection result of the video frames in the game video.

[0090] The above action recognition algorithm can be: OpenPose (Real-time Multi-Person 2D Pose Estimation Based on Part Affinity Fields), DeepLabCut (Markerless Pose Estimation Based on Deep Learning), or AlphaPose (Regional Multi-Person Pose Estimation). This is only an example and does not play a limiting role.

[0091] The electronic device can also use an action recognition model pre-trained based on the above action recognition algorithm to recognize the hitting action in the video frames of the game video to obtain the hitting action detection result.

[0092] When recognizing the hitting action in the video frames of the game video, each video frame in the game video can be recognized, or some of the video frames can be selected for recognition. When selecting video frames, the preset number of video frames selected per second can be ensured according to the actual frame rate of the game video.

[0093] After obtaining the hitting action detection result of the video frame, the electronic device may, based on the obtained hitting action detection result of the video frame, determine the video frame in which the hitting object makes a hitting action, and then use the moment corresponding to the determined video frame as the starting moment of the round.

[0094] Step S102: Starting from the starting moment of the round, track the movement of the balls on the billiard table in the video frame, and determine the ending moment of the round corresponding to the starting moment of the round based on the video frame in which all the tracked balls are in a stationary state.

[0095] The ending moment of the round is: the first moment when all the billiard balls on the table are in a stationary state after the hitting object hits the ball with the cue.

[0096] The electronic device may use a target tracking algorithm to track the movement of all the billiard balls on the billiard table in the video frame starting from the starting moment of the round, and track the movement state of each billiard ball on the table. When the first video frame in which all the billiard balls on the table are in a stationary state is tracked, the electronic device may determine the moment corresponding to the video frame as the ending moment of the round.

[0097] The above target tracking algorithm may be: a method based on a Kalman filter, a method based on data association, a SORT (Simple Online and Realtime Tracking) algorithm, or a DeepSort (DeepSimple Online and Realtime Tracking) algorithm. This is only for example and does not play a limiting role.

[0098] The electronic device may also use a target tracking model pre-trained based on the above target tracking algorithm to track the movement of the billiard balls to determine the ending moment of the round.

[0099] Step S103: Perform video content analysis on the first round segment to obtain a first analysis result describing whether the first hitting object scores a goal in the first round.

[0100] Wherein, the first round segment is: a video segment from the starting moment of the round to the ending moment of the round.

[0101] In a possible implementation manner, when the first analysis result indicates that the first hitting object scores a goal, the first analysis result may further include: description information of the pocketed ball. Wherein, the description information of the pocketed ball may include at least one of the following information: the color of the pocketed ball, the number of the pocketed ball, the pocket number of the pocketed ball, the scoring moment of the pocketed ball.

[0102] In this way, the specific content in the highlight clip can be described based on the description information about the balls potted in the first analysis result later, so that relevant sports media workers and video creators can obtain the information in the highlight clip more quickly.

[0103] The electronic device can use a pre-trained multimodal large language model to recognize the video content in the first-round clip. Specifically, the electronic device can input the first-round clip into the multimodal large language model and input a prompt word into the multimodal large language model. For example, the prompt word can be: "Video frame title", "Please describe this video", etc. The multimodal large language model can perform video content analysis on the first-round clip based on the input first-round clip and prompt word, and output an analysis result. After the electronic device obtains the analysis result output by the multimodal large language model, it can determine the first analysis result that describes whether the first hitting object scored a goal in the first round from it.

[0104] Step S104, if the first-round clip is not the first-round clip of the game video, determine the highlight clip in the game video based on the first analysis result and the second analysis result.

[0105] The second analysis result is the result of video content analysis of the second-round clip before the first-round clip.

[0106] The electronic device can determine whether there are other round clips before the first-round clip according to the time sequence corresponding to each round clip and the first-round clip.

[0107] If not, it means that the first-round clip is the first-round clip of the game video, that is, the first hitting round in a billiards game included in the game video. In this billiards game, there will be no other goals before the first hitting round corresponding to the first-round clip, so it is impossible to determine the highlight clip including consecutive goals or a clean sweep from the first-round clip and the round clips before the first-round clip.

[0108] If so, it means that the first-round clip is not the first-round clip of the game video. That is to say, in a billiards game included in the game video, there may be goals in the hitting rounds before the hitting round corresponding to the first-round clip, and there may be highlight clips such as consecutive goals or a clean sweep. The electronic device can determine the highlight clip in the game video based on the first analysis result corresponding to the first-round clip and the second analysis result corresponding to the second-round clip before the first-round clip.

[0109] How to specifically determine the highlight clip based on the first analysis result and the second analysis result will be described in detail later and will not be elaborated here.

[0110] The above steps S101 - S104 are the analysis process of a round segment. The electronic device can execute the above steps S101 - S104 multiple times based on the time sequence. For each round segment, determine the start time and end time of the round segment, perform video content analysis on the round segment to obtain an analysis result, and thus determine the highlight segment based on the analysis result.

[0111] In the technical solution provided by the embodiment of the present application, the electronic device first determines the start time of the round based on the detection result of the hitting action of the video frame in the billiards game video, and then tracks the balls on the billiards table to determine the end time of the round. Based on the above start time and end time of the round, each round segment can be determined. Furthermore, the electronic device can perform video content analysis on each round segment and combine the analysis results in each round segment to determine the highlight segment in the game video. In this way, there is no need to manually determine the highlight segment, which improves the efficiency of determining the highlight segment in the billiards game video.

[0112] In a possible implementation manner, the electronic device can also first determine all the start times of the rounds at one time, and then determine the end time of the round corresponding to each start time of the round. As Figure 2 shown, the electronic device determines multiple start times of the rounds according to step S101 as the Figure 2 times shown on the upper time axis in Figure 2 the figure. After that, the electronic device then determines the end times of the rounds corresponding to the above multiple start times of the rounds according to step S102, such as the

[0113] times other than the start time of the round shown on the lower time axis in Figure 3 the figure. After determining all the start times and end times of the rounds, the electronic device then executes steps S103 and S104 for each round segment based on the time sequence to determine the highlight segment.

[0114] Step S301, if the first analysis result indicates that the first hitting object does not score a goal, then based on the second analysis result, search backward from the third round segment to find the second round segment in which the first hitting object scores consecutive goals.

[0115] Among them, the third-round segment is: the second-round segment that is before and adjacent to the first-round segment. The first analysis result indicates that the first hitting object did not score a goal, indicating that it is the turn of the opponent of the first hitting object to hit the ball after the current round. The electronic device can trace back whether the first hitting object scored consecutive goals in multiple previous second-round segments. Specifically, the electronic device can sequentially search for the round segments before the first-round segment based on the chronological order. If the first hitting object scored a goal in the third-round segment that is before and adjacent to the first-round segment, the electronic device can continue to search forward for the round segment that is before and adjacent to the third-round segment until it finds the round segment corresponding to the hitting round that does not belong to the first hitting object, or until it finds the first-round segment of the game video.

[0116] Step S302, if the number of the found second-round segments is greater than the first number, determine that the found second-round segments are the wonderful segments in the game video.

[0117] The first number can be set to 1, or can be set to any value between 1 and 2 (excluding 2), and there is no limitation on this. If the electronic device finds 2 or more second-round segments, it means that the first hitting object scored 2 or more consecutive goals, that is, the found second-round segments are the wonderful segments of consecutive goals, and the electronic device can determine that the found second-round segments are the wonderful segments in the game video.

[0118] In this way, the wonderful segments including consecutive goals in the game video can be determined more accurately.

[0119] In a possible implementation manner, the electronic device can also determine the wonderful segments in the following way, as Figure 4 shown, the above step S104 may include the following steps S401 - S403.

[0120] Step S401, if the first analysis result indicates that the first hitting object scored a goal, then based on the first analysis result, determine whether the ball hit by the first hitting object is the last ball that can be hit on the billiard table.

[0121] If the first analysis result indicates that the first hitting object scored a goal, that is, the first hitting object scored a goal within the first-round segment, there are two possibilities. One is that the first hitting object can hit the next ball in the next-round segment, and the other is that the first hitting object has hit the last ball that can be hit on the billiard table and won.

[0122] If the first hitting object can hit the next ball in the next-round segment, it means that there is still a possibility for the first hitting object to hit the next ball in the future, and the electronic device can determine the wonderful segments after analyzing the next-round segment.

[0123] If the first hitting object has pocketed the last hittable ball on the billiards table and won, the electronic device executes subsequent steps S402 - S403, and searches backward from the first round segment to find the second round segment to determine the highlight segment.

[0124] The electronic device can determine whether the ball hit by the first hitting object is the last hittable ball on the billiards table based on the first analysis result. Specifically, the electronic device can combine the rules adopted in the billiards game corresponding to the game video, and the color of the ball pocketed by the first hitting object in the first round segment to determine whether the ball hit by the first hitting object is the last hittable ball on the billiards table. For example, in eight-ball billiards, the last hittable ball is the black eight ball, and the electronic device can determine whether it is the last hittable ball by judging whether the ball hit by the first hitting object is black based on the first analysis result.

[0125] Step S402, if so, based on the second analysis result, search backward from the first round segment to find the second round segment in which the first hitting object continuously pockets balls.

[0126] If the electronic device determines whether the ball hit by the first hitting object is the last hittable ball on the billiards table, it means that the first hitting object has pocketed the last hittable ball on the billiards table and won. The electronic device can trace back whether the first hitting object has continuously pocketed balls in multiple previous second round segments.

[0127] Specifically, the electronic device can sequentially search backward from the first round segment based on the time sequence. If the first hitting object pockets a ball in the round segment adjacent to and before the first round segment, the electronic device can continue to search backward for the previous adjacent round segment until it finds the round segment corresponding to the hitting round that does not belong to the first hitting object, or the first round segment of the game video.

[0128] Step S403, if the number of found second round segments is greater than the first number, determine the found second round segments as the highlight segments in the game video.

[0129] The first number can be set to 1, or any value between 1 and 2 (excluding 2), and there is no limitation on this. Here, the second round segment includes the first round segment. If the electronic device finds 2 or more second round segments, it means that the first hitting object has continuously pocketed 2 or more balls, that is, the second round segments found by the electronic device are highlight segments of continuous ball pocketing. The electronic device can determine the found second round segments as the highlight segments in the game video.

[0130] In this way, the highlight segments including continuous ball pocketing and a clean sweep in the game video can be determined more accurately.

[0131] In a possible implementation, the billiards video obtained by the electronic device may include multiple billiards games. Before determining the starting moment of a round from the game video, the electronic device may first determine the game video corresponding to each billiards game from the billiards video. Then, based on the determined game video, the electronic device determines the starting moment of the round in the game video. Based on this, the embodiments of the present application provide the following method for determining the starting moment of a round, as Figure 5 shown, the above step S101 may include the following steps S501 - S504.

[0132] Step S501, perform a hitting action recognition on the video frames in the billiards video to obtain the hitting segment and hitting type corresponding to the recognized hitting action.

[0133] The action of the hitting object hitting the ball usually lasts for a period of time. When the electronic device performs a hitting action recognition on the video frames, it will recognize that the hitting object makes a hitting action in a continuous plurality of video frames. The electronic device may use these continuous plurality of video frames as the hitting segment corresponding to the hitting action.

[0134] The electronic device may use an action recognition algorithm to perform a hitting action recognition on the video frames in the billiards video, and recognize a continuous plurality of video frames with hitting actions, that is, the hitting segment composed of these plurality of video frames. Specifically, the electronic device may use the action recognition algorithm or action recognition model mentioned in step S101 to perform a hitting action recognition, which will not be elaborated here.

[0135] The hitting type includes a break shot and a normal shot. A normal shot may also be referred to as a shot during the game process. The electronic device may use an image recognition algorithm to recognize whether the arrangement position of the billiard balls in the video frame corresponding to the hitting action presents a break shot arrangement, and then determine whether the hitting type of the recognized hitting action is a break shot according to the recognized arrangement position. For example, in billiards, the break shot arrangement is usually a triangular arrangement. The electronic device may recognize whether the billiard balls in the video frame corresponding to the hitting action present a triangular arrangement. If it presents a triangular arrangement, the electronic device may determine that the hitting type of the hitting action is a break shot. If it does not present a triangular arrangement, the electronic device may determine that the hitting type of the hitting action is a normal shot.

[0136] The electronic device may also combine the above action recognition algorithm and image recognition algorithm to train a classification model that can recognize the hitting segment and hitting type corresponding to the hitting action. The classification model may classify the video frames into a hitting state and a non - hitting state based on the action recognition algorithm, and then further classify the video frames into a break shot state and a normal shot state based on the image recognition algorithm. Among them, the video frames in the hitting state are: video frames with hitting actions; the video frames in the non - hitting state are: video frames without hitting actions.

[0137] Step S502: Determine the hitting moment based on the hitting segment corresponding to the first hitting action.

[0138] Among them, the hitting type of the first hitting action is the opening shot. The electronic device can determine the start moment, end moment, or any moment in the middle of the hitting segment as the hitting moment, and there is no limitation on this.

[0139] Step S503: For each hitting moment, if there is another hitting moment after this hitting moment, determine the video segment between this hitting moment and the adjacent hitting moment after this hitting moment as the game video. If there is no other hitting moment after this hitting moment, determine the video segment between this hitting moment and the end of the billiards video as the game video.

[0140] The hitting moment can be used as the start moment of each billiards game. The electronic device can sequentially determine whether there is another hitting moment after each hitting moment based on the time sequence. If there is, the electronic device can determine the video segment between this hitting moment and the adjacent hitting moment after this hitting moment as the game video. If not, it means that the billiards game corresponding to this hitting moment is the last game in this billiards video, and the electronic device can determine the video segment between this hitting moment and the end of the billiards video as the game video.

[0141] Step S504: Determine the starting moment of each round in the game video based on the hitting segment corresponding to the hitting action in the game video.

[0142] After determining each game video in the billiards video, for each game video, the electronic device can determine the starting moment of each round in the game video based on the hitting segment corresponding to the hitting action. Specifically, the electronic device can determine the start moment, end moment, or any moment in the middle of each hitting segment as the starting moment of the round, and there is no limitation on this.

[0143] In this way, it is possible to first determine each game video corresponding to each game included in the billiards video, avoid interfering with the subsequent determination of exciting segments, and improve the accuracy of determining exciting segments.

[0144] In a possible implementation manner, before determining the hitting moment in step S502, the electronic device can also perform a merging process on the recognized hitting actions. Based on this, the embodiments of the present application also provide a way to determine the starting moment of each round, as Figure 6 shown, the above step S101 may include the following steps S601 - S606.

[0145] Step S601: Identify the hitting actions in the billiards video frames, obtain the hitting segments and hitting types corresponding to the identified hitting actions, which is the same as step S501 above.

[0146] Step S602: Search for the hitting actions whose frame distances between the hitting segments corresponding to the identified hitting actions are less than the first frame distance.

[0147] The frame distance between hitting segments: It is the distance between the end frame of the previous hitting segment and the start frame of the next hitting segment among two adjacent hitting segments. The electronic device can sequentially identify whether the frame distance between the hitting segments corresponding to two adjacent hitting actions is less than the first frame distance based on the time sequence, so as to determine the hitting actions whose corresponding frame distances between the hitting segments are less than the first frame distance.

[0148] As Figure 7 shown, the electronic device has identified the hitting segments corresponding to each hitting action in the billiards video. The electronic device can sequentially identify the frame distance d1 between hitting segment 1 and hitting segment 2, the frame distance d2 between hitting segment 2 and hitting segment 3, and the frame distance d3 between hitting segment 3 and hitting segment 4 starting from hitting segment 1, and so on for the subsequent ones. The electronic device can determine that d1 and d2 are greater than the first frame distance, and d3 is less than the first frame distance, and thus can determine that the hitting actions corresponding to hitting segment 3 and hitting segment 4 are the hitting actions whose corresponding frame distances between the hitting segments are less than the first frame distance.

[0149] Step S603: Merge the found hitting actions, and determine the hitting segment and hitting type corresponding to the merged hitting action based on the hitting segment and hitting type corresponding to the found hitting action.

[0150] After finding the hitting actions, the electronic device can merge the found hitting actions into one hitting action, and merge the hitting segments corresponding to the hitting actions into one hitting segment. Specifically, the electronic device takes the start moment of the previous hitting segment as the start moment of the new hitting segment, and the end moment of the next hitting segment as the end moment of the new hitting segment.

[0151] The electronic device can also determine the hitting type corresponding to the merged hitting action based on the hitting type corresponding to the hitting action. Specifically, if the hitting type corresponding to any of the found hitting actions is the opening shot hitting, the electronic device can determine that the hitting type corresponding to the merged hitting action is the opening shot hitting; if the hitting type corresponding to any of the found hitting actions is the ordinary hitting, the electronic device can determine that the hitting type corresponding to the merged hitting action is the ordinary hitting.

[0152] Still taking Figure 7For example, the hitting actions found by the electronic device are the hitting actions corresponding to hitting segment 3 and hitting segment 4, where the hitting types corresponding to hitting segment 3 and hitting segment 4 are both ordinary hits. The electronic device can merge the hitting actions corresponding to hitting segment 3 and hitting segment 4 into 1 hitting action, merge hitting segment 3 and hitting segment 4 into hitting segment 5, use hitting segment 5 as the hitting segment corresponding to the merged hitting action, and use ordinary hit as the hitting type corresponding to the merged hitting action.

[0153] Step S604, determining the hitting moment based on the hitting segment corresponding to the first hitting action is the same as step S502 above.

[0154] Step S605, for each hitting moment, if there is another hitting moment after this hitting moment, then determine the video segment between this hitting moment and the adjacent hitting moment after this hitting moment as the game video. If there is no other hitting moment after this hitting moment, then determine the video segment between this hitting moment and the end of the billiards video as the game video, which is the same as step S503 above.

[0155] Step S606, determining the starting moment of a round in the game video based on the hitting segment corresponding to the hitting action in the game video is the same as step S504 above.

[0156] In this way, merging the hitting actions with relatively close frame distances of the corresponding hitting segments among the determined hitting actions can avoid inaccurate hitting segments caused by misidentification. It improves the determination of the hitting moment based on this hitting segment subsequently, and further improves the accuracy of determining the game video, and thus further improves the accuracy of determining the wonderful segments based on the game video.

[0157] In a possible implementation manner, the electronic device can also determine the hitting segment and hitting type corresponding to the hitting action in the following way. As Figure 8 shown, the above step S501 can include the following steps S801 - 804.

[0158] Step S801, extracting features from the video frames in the billiards video.

[0159] The electronic device can use an image feature extraction algorithm to extract the action features of the video frames in the billiards video. The above image feature extraction algorithm can be: edge detection algorithm, corner detection algorithm, deep learning feature extraction algorithm, texture feature extraction algorithm, etc. This is only for example and does not play a limiting role.

[0160] Step S802, determining the video frames with hitting actions in the billiards video based on the extracted features.

[0161] The electronic device can perform feature matching between the extracted features and the hitting action features, and determine the video frame corresponding to the features that match the hitting action features as the hitting action video frame.

[0162] Step S803: Determine the hitting segment corresponding to the hitting action in the billiards video according to the arrangement order of the determined video frames in the billiards video.

[0163] The electronic device can determine consecutive multiple hitting action video frames as the hitting segment corresponding to the hitting action based on the time sequence. For the case where there is a single non-hitting action video frame among consecutive multiple hitting action video frames, the electronic device can also use this non-hitting action video frame as part of the hitting segment.

[0164] Step S804: Obtain the hitting type corresponding to the hitting action according to the distribution information of the balls on the billiards table in the video frames within the hitting segment corresponding to the hitting action.

[0165] The electronic device can use an image recognition algorithm to recognize the distribution information of the balls on the billiards table in the video frames within the hitting segment corresponding to the hitting action, and determine whether the distribution presents the opening ball arrangement according to the distribution information to determine the hitting type corresponding to the hitting action.

[0166] In this way, the electronic device can more accurately determine the hitting segment and the hitting type corresponding to the hitting action, so as to more accurately determine the wonderful segments in the game video subsequently.

[0167] In a possible implementation manner, after determining the wonderful segment, the electronic device can also determine a wonderful label for the wonderful segment. Based on this, the embodiment of the present application also provides a method for determining a wonderful label: Determine the wonderful label of the wonderful segment based on at least one of the following information:

[0168] The description information of the pocketed balls included in the analysis result of the video content of the round segment in the wonderful segment;

[0169] Whether the wonderful segment includes the first round segment in the game video;

[0170] The complexity of the pocketed balls' goals within the round segment in the wonderful segment;

[0171] The number of goals in the wonderful segment.

[0172] The description information of the pocketed balls can include at least one of the following information: the color of the pocketed ball, the number of the pocketed ball, the pocket number of the pocketed ball, the goal time of the pocketed ball. The electronic device can determine the complexity of the pocketed balls' goals based on the movement path of the pocketed balls before they enter the pocket.

[0173] The electronic device can determine whether the pocketed ball is the last hittable ball of the hitting object according to the description information of the pocketed ball. If so, the electronic device can further determine whether the number of goals in the exciting clip is the total number of all hittable balls of the first hitting object in this game. If so, the electronic device can determine that the exciting label of the exciting clip is a clean sweep.

[0174] The electronic device can also determine whether there is a goal with a set high complexity according to the goal complexity of the pocketed ball in the round clip of the exciting clip. If so, the electronic device can determine that the exciting label of the exciting clip includes a high-difficulty goal. In addition, the electronic device can also determine exciting labels such as consecutive goals, invalid goal moments, black eight goal moments, white ball goal moments, etc. based on the above information, which will not be elaborated here.

[0175] In this way, relevant sports media workers and video creators can obtain the information in the exciting clip more quickly based on the above exciting labels.

[0176] In a possible implementation manner, the electronic device can also obtain the first analysis result through the following method. As Figure 9 shown, the above step S103 may include the following steps S901 - S903.

[0177] Step S901, perform hitting object recognition on the hitting clip in the first round clip to obtain the description information of the first hitting object.

[0178] The electronic device can use an object recognition algorithm to perform object recognition on the hitting clip in the first round clip to obtain the description information of the first hitting object. The description information of the first hitting object may include information such as the clothing and height of the first hitting object.

[0179] Step S902, determine whether there is a goal based on the movement path of the ball in the first round clip.

[0180] Among them, the movement path of the ball is obtained by tracking the movement of the balls on the billiard table.

[0181] The electronic device can use a target tracking algorithm to track all the balls on the table in the first round clip. If the electronic device tracks a ball into the pocket, the electronic device can determine that there is a goal.

[0182] Step S903, generate the first analysis result based on the obtained description information and the goal judgment result.

[0183] Based on the above-described description information and the goal determination result, the electronic device can generate a first analysis result indicating whether the first hitting object described by the description information scores a goal. The first analysis result can be presented in text form or in non-text form. In one example, the first analysis result is presented in text form. For example, if the description information of the first hitting object is an object wearing a red top and black pants, and there is a goal in the first-round clip, the electronic device can generate a first analysis result of "the object wearing a red top and black pants in the video scores a goal".

[0184] In this way, the generated first analysis result meets the requirements for subsequent determination of exciting clips, thereby improving the accuracy of subsequent determination of exciting-round clips based on this first analysis result.

[0185] To more clearly illustrate the above-described exciting-clip determination method, an embodiment of the present application also provides a Figure 10 flow diagram for determining exciting clips in an example as shown.

[0186] Figure 10 The flow shown is set according to the rules of eight-ball billiards. The electronic device starts from the opening moment (i.e., the hitting moment) of a billiards game and sequentially analyzes each round clip decomposed.

[0187] The electronic device first determines whether a goal is scored in a round clip based on the first analysis result of the first-round clip.

[0188] If no goal is scored, the electronic device can determine whether there is a goal-scoring round, that is, look forward to find whether there is a second-round clip with consecutive goals, and when the number of the found second-round clips is greater than the first number, determine the second-round clip as an exciting clip, as in the above steps S301 - S302. The electronic device can also determine an exciting label for the determined exciting clip as consecutive goals.

[0189] If a goal is scored, the electronic device can determine whether it is a black-eight goal, that is, determine whether the ball hit by the first hitting object is the last hittable ball on the billiards table. If it is a black-eight goal, the electronic device can look forward to find the second-round clip of the first hitting object's consecutive goals. The electronic device can determine whether the number of the found second-round clips is greater than the first number (not shown in the figure). If it is greater, the electronic device can determine the found second-round clip as an exciting clip in the game video, as in the above steps S401 - S403.

[0190] Further, the electronic device can also determine a wonderful label for the determined wonderful clip. The electronic device can determine whether the number of the above-mentioned second-round clips is a second number, that is, determine whether the first hitting object continuously scores the second number of goals. The second number is the total number of balls that can be hit by the first hitting object in the entire billiards game. If so, it means that the first hitting object scores all the balls that can be hit in this game within multiple consecutive own rounds and wins. The electronic device can determine the wonderful label for the determined wonderful clip as a clean sweep. If not, it means that the first hitting object only scores multiple balls that can be hit within multiple consecutive own rounds. The electronic device can determine the wonderful label for the determined wonderful clip as continuous scoring.

[0191] If the electronic device determines that it is not a black-eight goal, the electronic device can continue to determine whether there are other round clips after the current round clip. If there are, continue to analyze the next round clip. If not, end the determination of the wonderful clips in this game video.

[0192] Applying the technical solution provided by the embodiments of the present application, the electronic device can quickly determine wonderful clips based on the acquired billiards video, without sports media workers, video creators, etc. having to watch the video in person and manually determine the wonderful clips, greatly saving the working time and improving the work efficiency of sports media workers and video creators.

[0193] The above wonderful clip determination method can be applied to the unmanned billiards scenario. Currently, in many unmanned billiards halls, in order to make it more convenient for users to use the billiards table, there are usually no referees or other staff. Users can directly scan the code to open the table on the billiards table and use the billiards table for billiards games by themselves. For the convenience of management, image acquisition devices are usually installed in such unmanned billiards halls to record the process of the billiards game with the user's knowledge.

[0194] To facilitate users to view the wonderful operations during the game process, the above wonderful clip determination method can be used to provide the determined wonderful clips to users. Specifically, the electronic device can acquire the above-recorded billiards video, determine the wonderful clips in the billiards video based on the above wonderful clip determination method, and then the unmanned billiards hall can send them to the users through other clients.

[0195] The embodiments of the present application also provide a wonderful clip determination system, as Figure 11 shown. The system includes:

[0196] An image acquisition device 1101, which is set in the billiards game venue and is used to acquire the game video of the billiards. If the image acquisition device does not have the ability of image intelligent analysis, it sends the game video to the backend device. If the image acquisition device has the ability of image intelligent analysis, it executes any of the above wonderful clip determination methods;

[0197] The back-end device 1102 is a service device located locally in the billiards game venue or a remote server, which is used to receive and store the game videos collected by the image acquisition device and execute any of the above-mentioned highlight determination methods.

[0198] As can be seen from the above content, if the image acquisition device does not have the image intelligent analysis ability, the image acquisition device can send the game video to the back-end device, and the back-end device processes the game video to determine the highlights in the game video.

[0199] If the image acquisition device has the image intelligent analysis ability, the image acquisition device can process the game video to determine the highlights in the game video. In a possible implementation, after the image acquisition device determines the highlights in the game video, it can send the determined highlights to the back-end device, and the back-end device can store the highlights determined by the image acquisition device. The image acquisition device can also send the information of the determined highlights and the game video to the back-end device, and the back-end device stores the received information of the highlights and the game video. The information of the highlights can include the start time, end time of the highlights or the highlight tags of the highlights, etc.

[0200] In a possible implementation, if the image acquisition device has the image intelligent analysis ability, the image acquisition device can also send the game video to the back-end device, and the back-end device can process the received game video to determine the highlights in the game video.

[0201] In the technical solution provided by the embodiments of the present application, the electronic device first determines the start time of the round based on the hitting action detection result of the video frames in the billiards game video, and then tracks the balls on the billiards table to determine the end time of the round. Based on the above start time and end time of the round, each round segment can be determined. Furthermore, the electronic device can perform video content analysis on each round segment and combine the analysis results in each round segment to determine the highlights in the game video. In this way, there is no need to manually determine the highlights, which improves the efficiency of determining the highlights in the billiards game video.

[0202] Corresponding to the above-mentioned highlight determination method, the embodiments of the present application also provide a highlight determination device, as Figure 12 shown, the device includes:

[0203] The first determination module 1201 is configured to determine the start time of the round in the game video based on the hitting action detection result of the video frames in the billiards game video;

[0204] A second determination module 1202, configured to perform motion tracking on the balls on the billiard table in the video frames starting from the beginning moment of a round, and determine the end moment of the round corresponding to the beginning moment of the round based on the video frames in which all the tracked balls are in a stationary state;

[0205] An analysis module 1203, configured to perform video content analysis on the first round segment to obtain a first analysis result describing whether the first hitting object scores a goal in the first round, where the first round segment is: a video segment from the beginning moment of the round to the end moment of the round;

[0206] A third determination module 1204, configured to, if the first round segment is not the first round segment of the game video, determine the exciting segment in the game video based on the first analysis result and the second analysis result, where the second analysis result is: the result of performing video content analysis on the second round segment before the first round segment.

[0207] In the technical solution provided by the embodiment of the present application, the electronic device first determines the beginning moment of the round based on the hitting action detection result of the video frames in the billiard game video, then tracks the balls on the billiard table to determine the end moment of the round, and can determine each round segment based on the above-mentioned beginning moment and end moment of the round. Furthermore, the electronic device can perform video content analysis on each round segment and determine the exciting segment in the game video in combination with the analysis results in each round segment. In this way, there is no need to manually determine the exciting segment, which improves the efficiency of determining the exciting segment in the billiard game video.

[0208] In a possible implementation manner, the third determination module 1204 may specifically be configured to: if the first analysis result indicates that the first hitting object does not score a goal, based on the second analysis result, search forward from the third round segment to find the second round segment in which the first hitting object continuously scores a goal, where the third round segment is: the second round segment before and adjacent to the first round segment; if the number of the found second round segments is greater than the first number, determine the found second round segments as the exciting segment in the game video.

[0209] In a possible implementation manner, the third determination module 1204 may specifically be configured to: if the first analysis result indicates that the first hitting object scores a goal, determine whether the ball hit by the first hitting object is the last ball that can be hit on the billiard table based on the first analysis result; if so, based on the second analysis result, search forward from the first round segment to find the second round segment in which the first hitting object continuously scores a goal; if the number of the found second round segments is greater than the first number, determine the found second round segments as the exciting segment in the game video.

[0210] In a possible implementation manner, the first determination module 1201 may specifically be configured to: identify a hitting action in video frames of a billiards video to obtain a hitting segment and a hitting type corresponding to the identified hitting action; determine a hitting moment based on the hitting segment corresponding to the first hitting action, where the hitting type of the first hitting action is a break shot; for each hitting moment, if there is another hitting moment after this hitting moment, determine the video segment between this hitting moment and the adjacent hitting moment after this hitting moment as a game video, and if there is no other hitting moment after this hitting moment, determine the video segment between this hitting moment and the end of the billiards video as a game video; determine a round start moment in the game video based on the hitting segment corresponding to the hitting action in the game video.

[0211] In a possible implementation manner, the above-mentioned highlight segment determination device may further include a merging module, configured to, before determining the hitting moment based on the hitting segment corresponding to the first hitting action, find hitting actions with a frame distance less than a first frame distance between the hitting segments corresponding to the identified hitting actions; merge the found hitting actions, and determine the hitting segment and the hitting type corresponding to the merged hitting action based on the hitting segments and the hitting types corresponding to the found hitting actions.

[0212] In a possible implementation manner, the first determination module 1201 may specifically be configured to: extract features from video frames in a billiards video; determine video frames with a hitting action in the billiards video based on the extracted features; determine the hitting segment corresponding to the hitting action in the billiards video according to the arrangement order of the determined video frames in the billiards video; obtain the hitting type corresponding to the hitting action according to the distribution information of the balls on the billiards table in the video frames within the hitting segment corresponding to the hitting action.

[0213] In a possible implementation manner, when the first analysis result indicates that the first hitting object scores a goal, the first analysis result may further include: description information of the pocketed ball; the description information of the pocketed ball may include at least one of the following information: the color of the pocketed ball, the number of the pocketed ball, the pocket number of the pocketed ball, the goal moment of the pocketed ball.

[0214] In a possible implementation manner, the above-mentioned highlight segment determination device may further include a fourth determination module, configured to, after determining the highlight segment, determine a highlight label of the highlight segment based on at least one of the following information: the description information of the pocketed ball included in the video content analysis result of the round segment in the highlight segment; whether the highlight segment includes the first round segment in the game video; the complexity of the pocketed ball goal in the round segment of the highlight segment; the number of goals in the highlight segment.

[0215] In a possible implementation manner, the above analysis module 1203 can be specifically configured to: identify a hitting object in the hitting segment of the first-round segment to obtain the description information of the first hitting object; determine whether a goal is scored based on the movement path of the ball in the first-round segment, where the movement path of the ball is obtained by tracking the movement of the balls on the billiard table; generate a first analysis result based on the obtained description information and the goal determination result.

[0216] The embodiments of the present application also provide an electronic device, as Figure 13 shown, including:

[0217] A memory 1301 for storing a computer program;

[0218] A processor 1302, when executing the program stored on the memory 1301, implements any of the above wonderful segment determination methods.

[0219] And the above electronic device may further include a communication bus and / or a communication interface, and the processor 1302, the communication interface, and the memory 1301 complete communication with each other through the communication bus.

[0220] In the solution provided by the embodiments of the present application, the collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information and other processes are all executed under the premise of the user's knowledge and authorization, and comply with the provisions of relevant laws and regulations, and do not violate public order and good customs.

[0221] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0222] The communication interface is used for communication between the above electronic device and other devices.

[0223] The memory may include a Random Access Memory (RAM), and may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0224] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0225] In another embodiment provided by the present application, there is also provided a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned wonderful segment determination methods are implemented.

[0226] In another embodiment provided by the present application, there is also provided a computer program product containing instructions. When it runs on a computer, it causes the computer to execute any of the wonderful segment determination methods in the above embodiments.

[0227] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a Solid State Disk (SSD), etc.

[0228] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.

[0229] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other, and the key point of each embodiment is to illustrate the differences from other embodiments. In particular, for the embodiments of the system, device, electronic device, computer-readable storage medium, and computer program product, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments.

[0230] The above are only the preferred embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application are all included in the protection scope of the present application.

Claims

1. A method for determining a highlight segment, characterized in that: The method comprises: Determine the start time of a round in the billiards game video based on a ball hitting action detection result of a video frame in the billiards game video; Starting from the round start time, the motion of the balls on the billiard table in the video frames is tracked, and based on the video frames in which all the balls are tracked to be in a stationary state, the round end time corresponding to the round start time is determined; Performing video content analysis on the first round segment to obtain a first analysis result describing whether the first hitting object in the first round scores a goal, wherein the first round segment is: a video segment from the start time of the round to the end time of the round; If the first round segment is not the first round segment of the game video, determine the highlight segments in the game video based on the first analysis result and the second analysis result, wherein the second analysis result is: the result of video content analysis of the second round segment before the first round segment.

2. The method according to claim 1, characterized in that: The determining of the highlight segments in the game video based on the first analysis result and the second analysis result includes: If the first analysis result indicates that the first hitting object has not scored a goal, based on the second analysis result, searching for a second round segment in which the first hitting object has scored consecutive goals starting from the third round segment, wherein the third round segment is: a second round segment before and adjacent to the first round segment; If the number of the found second-round segments is greater than the first number, it is determined that the found second-round segments are the highlight segments in the game video.

3. The method according to claim 1, characterized in that The determining of the highlight segments in the game video based on the first analysis result and the second analysis result includes: If the first analysis result indicates that the first ball-hitting object scores a goal, determining whether the ball hit by the first ball-hitting object is the last hittable ball on the billiard table based on the first analysis result; If yes, based on the second analysis result, searching forward from the first round segment for a second round segment in which the first ball-hitting object scores consecutive goals; If the number of the found second-round segments is greater than the first number, it is determined that the found second-round segments are the highlight segments in the game video.

4. The method according to claim 1, characterized in that: The step of determining the start time of a round in the billiards game video based on the ball hitting action detection result of the video frame in the billiards game video comprises: Recognize the hitting action of the video frames in the billiard video to obtain the hitting segment and hitting type corresponding to the identified hitting action; Determining a hitting moment based on a hitting segment corresponding to a first hitting action, wherein the hitting type of the first hitting action is a tee shot; For each hitting moment, if there are other hitting moments after the hitting moment, the video clip from the hitting moment to the adjacent hitting moment after the hitting moment is determined as the game video; if there are no other hitting moments after the hitting moment, the video clip from the hitting moment to the end of the billiards video is determined as the game video; Based on the batting segment corresponding to the batting action in the game video, the round start time in the game video is determined.

5. The method according to claim 4, characterized in that Before determining the hitting moment based on the hitting segment corresponding to the first hitting action, the method further includes: Searching for a batting action whose frame distance between batting segments corresponding to the identified batting action is less than the first frame distance; The found hitting actions are merged, and based on the hitting segments and hitting types corresponding to the found hitting actions, the hitting segments and hitting types corresponding to the merged hitting actions are determined.

6. The method according to claim 4, characterized in that The step of performing hitting action recognition on video frames in the billiards video to obtain a hitting segment and a hitting type corresponding to the recognized hitting action includes: Extract features from video frames in billiards videos; Determine, based on the extracted features, a video frame in the billiards video where a hitting action exists; Determining a hitting segment corresponding to a hitting action in the billiards video according to an arrangement order of the determined video frames in the billiards video; According to the distribution information of the balls on the billiard table in the video frame in the hitting segment corresponding to the hitting action, the hitting type corresponding to the hitting action is obtained.

7. The method according to any one of claims 1 to 6, characterized in that In the case where the first analysis result indicates that the first ball-hitting object scores a goal, the first analysis result further includes: description information of the ball that has been pocketed; The description information of the pocketed ball includes at least one of the following information: The color of the ball that pockets, the number of the ball that pockets, the pocket number of the ball that pockets, and the time when the ball pockets.

8. The method according to any one of claims 1 to 6, characterized in that After determining the wonderful clips, the method further includes: Determine a highlight tag of the highlight segment based on at least one of the following information: Description information of pocketed balls included in the video content analysis results of the round segments in the highlight segments; Whether the highlight clip includes the first round clip in the game video; The complexity of the goal scored in the round segment in the highlight segment; The number of goals scored in the highlight clip.

9. The method according to any one of claims 1 to 6, characterized in that: The performing of video content analysis on the first round segment to obtain a first analysis result describing whether the first hitting object scores a goal in the first round includes: Performing ball hitting object recognition on the ball hitting segment in the first round segment to obtain description information of the first ball hitting object; Determining whether there is a goal based on a movement path of the ball in the first round segment, wherein the movement path of the ball is obtained based on motion tracking of the ball on the billiard table; Based on the obtained description information and the goal judgment result, a first analysis result is generated.

10. A system for determining a highlight segment, characterized in that: The system comprises: An image acquisition device is provided at a billiards competition venue and is used to acquire a video of a billiards game. If the image acquisition device does not have an image intelligent analysis capability, the video of the game is sent to a back-end device. If the image acquisition device has an image intelligent analysis capability, the method according to any one of claims 1 to 9 is executed. The backend device is a service device located locally at the billiards competition venue or a remote server, and is used to receive and store the game video captured by the image acquisition device, and execute the method as described in any one of claims 1-9.

11. A device for determining a highlight segment, characterized in that: The device comprises: A first determination module is used to determine the start time of a round in the billiards game video based on the ball hitting action detection result of the video frame in the billiards game video; A second determination module is used to track the motion of the balls on the billiard table in the video frame starting from the round start time, and determine the round end time corresponding to the round start time based on the video frame in which all the balls are tracked to be in a stationary state; An analysis module is used to perform video content analysis on a first round segment to obtain a first analysis result describing whether a first hitting object scores a goal in the first round, wherein the first round segment is a video segment from a start time of the round to an end time of the round; A third determination module is used to determine the wonderful segments in the game video based on the first analysis result and the second analysis result if the first round segment is not the first round segment of the game video, wherein the second analysis result is: the result of video content analysis of the second round segment before the first round segment.

12. The device according to claim 11, characterized in that The third determination module is specifically used for: if the first analysis result indicates that the first hitting object has not scored a goal, then based on the second analysis result, searching forward from the third round segment for a second round segment in which the first hitting object has scored consecutive goals, wherein the third round segment is: a second round segment before and adjacent to the first round segment; if the number of the found second round segments is greater than the first number, determining that the found second round segment is a highlight segment in the game video; and / or The third determination module is specifically used to: if the first analysis result indicates that the first hitting object has scored a goal, determine based on the first analysis result whether the ball hit by the first hitting object is the last hittable ball on the billiard table; if yes, search forward from the first round segment for a second round segment in which the first hitting object has scored consecutive goals based on the second analysis result; if the number of the second round segments found is greater than the first number, determine that the second round segments found are the highlights in the game video; and / or The first determination module is specifically used to: identify the hitting action of the video frames in the billiards video to obtain the hitting segment and the hitting type corresponding to the identified hitting action; determine the hitting moment based on the hitting segment corresponding to the first hitting action, wherein the hitting type of the first hitting action is the opening shot; for each hitting moment, if there are other hitting moments after the hitting moment, determine the video segment from the hitting moment to the adjacent hitting moment after the hitting moment as the game video, and if there are no other hitting moments after the hitting moment, determine the video segment from the hitting moment to the end of the billiards video as the game video; determine the start time of the round in the game video based on the hitting segment corresponding to the hitting action in the game video; and / or The device further includes a merging module, which is used to search for a hitting action whose frame distance between hitting segments corresponding to the identified hitting action is less than a first frame distance before determining the hitting moment based on the hitting segment corresponding to the first hitting action; merge the found hitting actions, and determine the hitting segment and hitting type corresponding to the merged hitting action based on the found hitting segments and hitting types corresponding to the hitting actions; and / or The first determination module is specifically used to: extract features from video frames in a billiard video; determine video frames in the billiard video containing a hitting action based on the extracted features; determine a hitting segment corresponding to the hitting action in the billiard video according to an arrangement order of the determined video frames in the billiard video; obtain a hitting type corresponding to the hitting action according to distribution information of balls on the billiard table in the video frames in the hitting segment corresponding to the hitting action; and / or In the case where the first analysis result indicates that the first hitting object scores a goal, the first analysis result further includes: description information of the pocketed ball; the description information of the pocketed ball includes at least one of the following information: the color of the pocketed ball, the number of the pocketed ball, the pocket number of the pocketed ball, and the goal time of the pocketed ball; and / or The device further includes a fourth determination module, which is used to determine the highlight tag of the highlight segment based on at least one of the following information after determining the highlight segment: description information of the pocketed ball included in the video content analysis result of the round segment in the highlight segment; whether the highlight segment includes the first round segment in the game video; complexity of the pocketed ball goal in the round segment in the highlight segment; number of goals in the highlight segment; and / or The analysis module is specifically used to: identify the hitting object in the hitting segment in the first round segment to obtain description information of the first hitting object; based on the movement path of the ball in the first round segment, determine whether there is a goal, wherein the movement path of the ball is obtained based on motion tracking of the ball on the billiard table; and generate a first analysis result based on the obtained description information and the goal judgment result.

13. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing the method according to any one of claims 1 to 9 when executing a program stored in a memory.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images