Video marking method and device, computing equipment and computer readable storage medium

Abstract

The embodiment of the invention provides a video marking method and device, computing equipment and a computer readable storage medium. The method comprises the following steps: for each frame of image in multiple frames of images of a motion video, identifying a target text area in the image; performing comparative analysis on the position information of each target text region in the multiple frames of images, and determining candidate display positions of the action information; determining an action segmentation moment of the motion video based on change information of text content in a target text area corresponding to the candidate display position and a target change rule corresponding to action information; aiming at each action segment obtained by dividing the motion video according to the action segmentation moment, determining the action name of each action segment; and marking the motion video based on the motion segmentation time and the motion name of each motion segment. The method can improve the video marking efficiency.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of computer technology, and more particularly to a video tagging method and apparatus, a computing device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the development of computer technology, there are more and more videos of various types. It is crucial to enable users to quickly obtain the content of interest in the video.

[0003] Usually, the creator of a video or the staff of a video playback platform can divide a long video into segments and mark each segment with description information of its content. In this way, when watching the video, the user can quickly understand the overall content distribution of the video based on the description information marked for each segment, and can also choose to directly watch the content of interest according to needs.

[0004] However, the marking process for the video consumes much time and effort, and the video marking efficiency is low. Summary of the invention

[0005] The embodiments of this specification provide a video marking method, which can improve the efficiency of video marking. One or more embodiments of this specification also relate to a video marking device, a computing device, a computer-readable storage medium and a computer program product.

[0006] According to one aspect of an embodiment of this specification, a video marking method is provided, the method comprising: For each frame of the multiple frames of the motion video, identifying a target text area in the image; Comparing and analyzing the position information of each target text area in the multiple frames of images to determine candidate display positions of the action information; Determine the action segmentation time of the motion video based on the change information of the text content in the target text area corresponding to the candidate display position and the target change rule corresponding to the action information; Determine the action name of each action segment obtained by dividing the motion video according to the action segmentation time; The motion video is marked based on the action segmentation time and the action name of each action segment.

[0007] According to another aspect of an embodiment of this specification, a video marking device is provided, including: A recognition module, for identifying a target text area in each frame of multiple frames of a motion video; An analysis module, used for comparing and analyzing the position information of each target text area in the multiple frames of images to determine a candidate display position of the action information; A first determination module is used to determine the action segmentation time of the motion video based on the change information of the text content in the target text area corresponding to the candidate display position and the target change law corresponding to the action information; A second determination module is used to determine the action name of each action segment obtained by dividing the motion video according to the action segmentation time; The marking module is used to mark the motion video based on the action segmentation time and the action name of each action segment.

[0008] According to another aspect of the embodiments of this specification, there is provided a computing device, including: a memory and a processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the above method are implemented.

[0009] According to another aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores a computer program / instruction, and the steps of the above method are implemented when the computer program / instruction is executed by a processor.

[0010] According to another aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instruction, and when the computer program / instruction in the computer program product is executed in a processor, the steps of the above method are implemented.

[0011] In one embodiment of the present specification, for each frame image in a motion video, a target text area can be identified therein, and a candidate display position of the action information in the video can be determined based on the target text area. Accordingly, based on the change in the text content in the candidate display position, the action segmentation moment of the motion video and the action name of each action segment obtained by segmentation can be determined to achieve the marking of the motion video. In this way, automatic marking can be achieved by analyzing the motion video, without the need for manual marking of the motion video, which can improve the efficiency of video marking and reduce labor costs. For scenes with a large number of motion videos to be marked, the efficiency improvement is more significant. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a flow chart of a video marking method provided by an embodiment of this specification; Figure 2 This is a simplified flow chart of a video marking method provided in an embodiment of this specification; Figure 3It is a structural schematic diagram of a video marking device provided in an embodiment of this specification; Figure 4 It is a structural block diagram of a computing device provided in one embodiment of this specification. DETAILED DESCRIPTION

[0013] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0014] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms of "one", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items. The term "at least one" in one or more embodiments of this specification refers to "one or more", and "multiple" refers to "two or more". The term "including" is an open description and should be understood as "including but not limited to", and may include other content on the basis of the content already described.

[0015] It should be understood that although the terms "first", "second", etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, "first" may also be referred to as "second", and similarly, "second" may also be referred to as "first". Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0016] In addition, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant standards and requirements, and corresponding operation entrances shall be provided for users to choose to authorize or refuse.

[0017] With the development of computer technology, the creation and dissemination of various videos are becoming more and more widespread, and the requirements for video display effects are getting higher and higher. When the video is long, it may be difficult for users to accurately obtain the content of interest in the video. At present, by adding dot information on the video progress bar, the content of each period in the video can be simply described, so that users can quickly locate the position they need to watch.

[0018] In the sports video scene, more and more sports bloggers or individuals will make their own sports videos, such as sports teaching videos or follow-up training videos. In order to facilitate others to use their sports videos, video creators will add video effects and action information to the video, for example: adding the current action name, action countdown / action count, action explanation, rest section, and the next action preview and other information to the video. In order to further facilitate users to use videos, creators can manually parse and segment the video and add action names, add dot information to the video progress bar to mark the video, so that users can know at a glance the action training interval, action name, action rest interval, and action explanation interval in the video. Users can quickly locate the specified action according to their needs, or quickly skip the rest section and explanation section. However, this method is time-consuming and labor-intensive, with high labor costs and low video marking efficiency. Especially when there are many videos to be marked, the impact of this method on video marking efficiency will be more prominent.

[0019] The embodiments of this specification provide a video marking method that can automatically mark motion videos and improve video marking efficiency. This specification also relates to a video marking device, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail one by one in the following embodiments. The video marking method provided in this specification can be applied to a video marking device and the computing device. The video marking device and the computing device can be a server device, and the terminal device can send the video to the server device for the server device to execute the video marking method provided in the embodiments of this specification. In some embodiments, the video marking device and the computing device can be a terminal device with computing capabilities such as a smart phone, a laptop computer, a tablet computer, a virtual reality display device, or an augmented reality display device.

[0020] Figure 1 1 is a flowchart of a video marking method provided in an embodiment of this specification. The method can be applied to a video marking device. In this embodiment of the specification, the video marking device is taken as a server device as an example. Figure 1 As shown, the video marking method includes the following steps: Step 102: for each frame of the multiple frames of the motion video, identify a target text area in the image.

[0021] For example, the video marking device can be a background server device of the target application, which is used to support the normal operation of the target application. For example, the target application can be a sports fitness program, or it can be other video playback or publishing programs. The target application can be installed in the terminal, and the user uploads the sports video created by the user to the video marking device through the target application installed on the terminal, and the video marking device can receive the sports video accordingly. Then, the video marking device can automatically trigger the video marking function to execute the video marking method provided in the embodiment of this specification. In some embodiments, the user can also choose whether to mark the video when uploading the video. When the user chooses to mark the video, the video marking device will execute the video marking method provided in the embodiment of this specification.

[0022] The video marking device can determine multiple frames of images to be analyzed in the motion video. For example, the multiple frames of images can be all the frames of images in the motion video, or some images can be extracted from the motion video for analysis to reduce computing resource consumption and improve analysis efficiency.

[0023] In one embodiment, frame extraction can be performed on the motion video to determine the multiple frames of images, so as to reduce the amount of calculation. Accordingly, step 102 may include performing frame extraction processing on the motion video to determine the multiple frames of images to be analyzed in the motion video; for each frame of the multiple frames, identifying the target text area in the image. For example, the video marking device can determine the number of image frames to be extracted based on the frame rate of the motion video. If the frame rate of the motion video is between 25 frames and 30 frames per second, the video marking device can extract 5 frames or 10 frames per second to obtain the multiple frames of images to be analyzed. The video marking device can evenly extract image frames in the motion video, such as the number of image frames spaced between adjacent extracted image frames can be roughly equal.

[0024] For each frame of image to be analyzed in the motion video, the video marking device can perform text recognition on it to determine the text information in the image, and then infer the content of the image based on the text information. For example, optical character recognition (OCR) technology can be used to implement the text recognition, and the text information obtained by text recognition can include information about the text content and the location of the text content, and the location can be reflected by a detection frame. In the embodiment of this specification, a target text area can be obtained by performing text recognition on the image to be analyzed in the motion video. The target text area is the area where the text information in the image is located, such as a detection frame of the text information obtained after corresponding text recognition.

[0025] According to the regular statistics of sports training videos, the text indicating the name of the current action and some other descriptive information for the action are generally displayed in the sports training videos. If the descriptive information is information indicating the status of the action, the information on the status of the action may include the timing or count information of the action. The text indicating the name of the current action may be text information, and the information indicating the status of the action may be digital information. When the sports action in the video changes, the displayed text information will also change accordingly. For marking sports videos, it is usually necessary to mark the corresponding stages based on the sports actions in each stage of the video. Therefore, in the embodiments of this specification, the actual content of each position in the video can be determined by identifying the text information in each frame image in the video, and then subsequent video marking can be performed.

[0026] Step 104 : performing a comparison analysis on the position information of each target text area in the multiple frames of images to determine a candidate display position of the action information.

[0027] According to the regular statistics of sports videos, the area where the text information indicating the video content is located in a video is relatively fixed, while the area where the text information indicating the video content is located in different videos may be different. For example, in order to ensure the consistency of video special effects, the action name in the video can always be placed in the upper left corner, and the timing area can always be placed in the upper right corner, but the action name area and timing area of ​​different videos will be in different positions.

[0028] According to this rule, the video marking device can compare the positions of the text areas identified by each frame image in the motion video, and believe that the text information displayed at the position where the position of the target text area in each frame image changes less is more likely to indicate the video content. Accordingly, the position information of each target text area in the multiple frames to be analyzed can be compared and analyzed to determine whether the position of each target text area is roughly unchanged in each frame image, and the positions where the target text areas in each frame image do not overlap or overlap very little are eliminated to determine the candidate display positions of the action information. The action information may include action name information, and may also include information on the status of the action. After this comparison and analysis, the video marking device can determine one or more candidate display positions, and these candidate display positions can be further analyzed later to determine the specific motion information indicating the video content.

[0029] Step 106: Determine the action segmentation time of the motion video based on the change information of the text content in the target text area corresponding to the candidate display position and the target change rule corresponding to the action information.

[0030] A motion video usually includes multiple motion actions, and the motion information displayed in the image frames of different time periods in the motion video will change to indicate different motion actions. Accordingly, the video marking device can determine the intervals corresponding to different motion actions in the motion video by analyzing the differences in the motion information in each image frame to perform motion segmentation of the motion video. For example, based on these intervals, the motion segmentation moment of the motion video can be determined. The motion segmentation moment is the endpoint moment of the interval, corresponding to the first and last image frames in the interval. The motion information in the video can change according to certain specific change rules (such as target change rules), and different motion information change rules can be different, so the video marking device can determine the intervals corresponding to different motion actions in combination with the set motion change rules.

[0031] If the motion information is an action name, the action name can remain unchanged during the training period of a motion action; when the motion action in the video changes, the action name also changes accordingly. The target change rule corresponding to the motion information can be that the motion information remains unchanged for a period of time and then changes, and then remains unchanged again for a period of time. For example, a squat action changes to a leg press action after 30 seconds, and the leg press action changes to the next action after 30 seconds.

[0032] If the motion information is timing information or count information for an action, the timing information or count information can change continuously and stably (such as increasing or decreasing numbers) during the training period of a motion action. When the motion action in the video changes, it is necessary to re-time or count the next motion action. At this time, the timing or count information needs to be reset to the initial value, and the information will mutate. The target change rule corresponding to the motion information can be that the motion information mutates after continuously changing according to a certain amount of change, and then mutates out of the amount of change. For example, after a 30-second squat action, it becomes a 30-second leg press action, and a countdown is performed for each action. During the squat action, the action information is displayed as 30, 29, 28, ..., 3, 2, 1 in sequence, and the motion information changes continuously for multiple times according to the fixed amount of change "1"; then it is changed to a leg press action, and the action information directly changes to 30, and the motion information mutates out of the fixed amount of change, and then 29, 28, ..., 3, 2, 1 are displayed in sequence.

[0033] In the embodiment of the present specification, the video marking device determines the candidate display position of the motion information in the aforementioned step, and then can determine the accurate motion information indicating the motion action based on the text content in the candidate display position and the target change law corresponding to the motion information, so as to determine the intervals corresponding to different motion actions in the motion video based on the actual changes of the motion information.

[0034] Each candidate display position determined by the video marking device may correspond to multiple target text areas, which are areas in different frame images. The video marking device may determine the change information of the text content according to the text content in the target text area corresponding to each candidate display position in each frame image. According to whether the change information of the text content corresponding to each candidate display position satisfies the target change law, the candidate display position corresponding to the change information that satisfies the target change law is determined as the actual display position of the action information. Furthermore, based on the change of the action information at the actual display position (that is, the text content in each target text area corresponding to the position), the intervals corresponding to different motion actions in the motion video are determined to obtain the motion segmentation moments of the motion video.

[0035] Step 108: for each action segment obtained by dividing the motion video according to the action segmentation time, determine the action name of each action segment.

[0036] After determining the moments of each motion segment of the motion video, the video marking device can segment the motion video to obtain multiple motion segments, each of which can be a training interval for a motion action. Accordingly, for each motion segment, the action name of each action segment can be determined based on the content of each frame image in the action segment (such as motion information in each frame image).

[0037] Step 110: Mark the motion video based on the action segmentation time and the action name of each action segment.

[0038] The action segment moments and action names of each action segment determined by the video marking device for the motion video may belong to the marking information of the motion video, and the motion video may be marked thereafter. For example, marking points are added to the progress bar of the motion video based on the action segment moments, and an action segment is formed between adjacent marking points. The video marking device may display the action name of the action segment at the portion of the progress bar corresponding to each action segment, or display the action name of the action segment at the marking point corresponding to the starting position of an action segment.

[0039] After the above process, the video marking device can analyze the text area and text content in each image frame in the sports video, determine the motion information indicating the video content in the sports video, and then mark the relevant sports actions for the sports video based on the motion information and its changes. After marking the sports video, the video marking device can publish the sports video so that the user can watch the marked sports video and quickly obtain the content of the sports video based on the mark.

[0040] In summary, in the video marking method provided in the embodiments of this specification, for each frame image in a motion video, a target text area can be identified therein, and a candidate display position of the action information in the video can be determined based on the target text area. Correspondingly, based on the change in the text content in the candidate display position, the action segmentation moment of the motion video and the action name of each action segment obtained by segmentation are determined to achieve the marking of the motion video. In this way, automatic marking can be achieved by analyzing the motion video, and there is no need for manual marking of the motion video, which can improve the efficiency of video marking and reduce labor costs. For scenes with a large number of motion videos to be marked, the efficiency improvement is more significant.

[0041] In the embodiments of the present specification, the text area and the digital area in each frame image of the motion video can be determined respectively, so as to determine the motion information in the video based on the text information in the text area (that is, text information of the text type) and the digital information in the digital area (that is, text information of the digital type). Accordingly, in the above step 102, for each frame image in the multiple frames of the motion video, the target text area in the image is identified, including: performing text recognition on each frame image in the multiple frames of the motion video to obtain the text area where each text information in the image is located; in the text area where each text information is located, determining the first target text area where the text information of the text type is located, and / or the second target text area where the text information of the digital type is located. The first target text area is the above-mentioned text area, and the second target text area is the above-mentioned digital area.

[0042] The video marking device can use OCR technology to extract text information from each frame of the image to be analyzed in the motion video, and obtain the text content of each text information and its detection frame (that is, the text area where the text information is located). For each extracted text information, it can be split based on the type of text information, and each text information can be split into text information and digital information. Accordingly, the text area where each text information is located (such as the first target text area) and the text area where the digital information is located (such as the second target text area) can be obtained. After that, the action name can be determined based on the text information in the first target text area, and the segmented time of each action can be determined based on the digital information in the second target text area.

[0043] Optionally, performing text recognition on an image may only obtain text information or may only obtain digital information. In the case where only one type of text information is obtained, subsequent analysis and processing may be performed only on the text information of that type and the text area where it is located, and the analysis and processing process for the other type of text information and the area where it is located may not be performed.

[0044] In some embodiments, after obtaining the text area of ​​each text information, these text areas can be preliminarily filtered to eliminate the text area that is unlikely to contain motion information. In the embodiment of this specification, the text area remaining after the preliminary filtering can be referred to as the target text area. For example, a text filtering condition can be set for text information, and a digital filtering condition can be set for digital information. The above-mentioned determination of the first target text area where the text information of the text type is located, and / or the second target text area where the text information of the digital type is located in the text area where each text information is located can include: in the text area where each text information is located, the text area where the text information of the text type that does not meet the text filtering condition is located is determined as the first target text area; and / or, the text area where the text information of the digital type that meets the digital filtering condition is located is determined as the second target text area. Among them, the text filtering condition includes: the text information is located in the reference area of ​​the image, the character length is less than the length threshold, and the text information includes at least one of the reference characters, and the digital filtering condition includes that the text information is composed of only numbers.

[0045] The video marking device can filter out the text information that does not include the action name and the text area where it is located based on the text filtering conditions, and determine the text area where the remaining text information is located as the first target text area. For example, a reference area can be set in advance for the image, and the reference area is an area that displays certain specific information and does not display the action name. For example, the reference area is a letter area, which is located at the lower position of the image and the height does not exceed a certain height threshold. For example, the length threshold of the character length that the action name needs to meet can also be pre-set, such as the character length of the action name is greater than 1. The video marking device can filter out the text area with a character length less than in each determined text area. For example, reference characters that cannot appear in the action name can also be pre-set, such as special characters such as "?", "¥" and "#". The video marking device can compare the characters in each text information with the reference character for similarity, and filter out the text area corresponding to the text information when it is determined that the reference character appears in a certain text information.

[0046] The video marking device can determine that the state of the action is usually represented by pure numbers without any other characters. Accordingly, the digital screening condition set for the digital information can include: the text information is composed of only numbers. The video marking device determines whether the digital information in the text area where each digital information is determined to be located only includes numbers, and then only retains the text area including numbers as the second target text area.

[0047] When the video marking device divides the identified text regions into first target text regions and second target text regions according to text information and digital information in step 102, the first target text regions and the second target text regions may be analyzed separately in subsequent steps.

[0048] Accordingly, in the above step 104, comparing and analyzing the position information of each target text area in the multiple-frame images to determine the candidate display position of the action information may include: comparing and analyzing the position information of each first target text area in the multiple-frame images to determine the first candidate display position of the action information including the action name; and / or comparing and analyzing the position information of each second target text area in the multiple-frame images to determine the second candidate display position of the action information including the action status.

[0049] In the embodiment of the present specification, when each target text area in the image is divided into a first target text area and a second target text area according to a text area and a digital area, since the action name is represented by text information and the action entry state is represented by digital information, the candidate display position of the action name can be determined based on each first target text area, and the candidate display position of the action progress state can be determined based on each second target text area. Since the position change degree of the action name and the action progress state displayed in the video in each frame image is small or unchanged, the candidate display position that meets the display requirements of the action name and the action progress state can be determined by comparing and analyzing the position information of each target text area. The determination of the first candidate display position and the second candidate display position here can refer to the relevant introduction about the candidate display position in the above step 104, which will not be repeated here.

[0050] In some embodiments, the video marking device can determine the candidate display position of the action information through the overlapping information of each target text area. In the embodiment of this specification, the display position of the text information is represented by the first target text area. If the first target text area overlaps in multiple frames of images, it can be indicated that the position of the area has text information displayed for a long time, which is closer to the display characteristics of the action name. Accordingly, the position has a higher probability of being the display position of the action name.

[0051] By way of example, the above-mentioned comparison and analysis of the position information of each first target text area in the multiple-frame images to determine the first candidate display position of the action information containing the action name may include: determining the overlapping information of the positions of each first target text area in the multiple-frame images of the motion video; and determining the position of the first target text area whose overlapping information is greater than a first target threshold and whose image accounts for a higher than a first target ratio in the multiple-frame images as the first candidate display position of the action information containing the action name.

[0052] The video marking device can calculate the intersection of union (IOU) value between each first target text area to characterize the overlapping information of the position of each first target text area. The IOU value can be the IOU value of the detection box of each text information. When the IOU value between two first target text areas is greater than the first target threshold, it can be considered that the two first target text areas are located at the same position and the two first target text areas overlap. If the first target threshold is 50%, optionally, the first target threshold can also be 60% or other values.

[0053] After obtaining the IOU value between each first target text area in each frame image and each first target text area in other frame images, the video marking device can determine the position in the entire motion video where the first target text area appears more frequently (such as higher than the first target ratio), and determine the position as the first candidate display position for the action name, assuming that the position has a higher probability of being used to display the action name. For example, the first target ratio can be 0.7, or it can also be 0.6 or other values. The video marking device can indicate the frequency of occurrence of the position where the first target text area is located by determining the proportion of the number of frames in the multiple frames to be analyzed to which the first target text area that meets the first target threshold belongs.

[0054] That is, the video marking device determines the IOU value of each first target text area in each frame image with the first target text areas in other frames, and then determines the reference images of each frame corresponding to the first target text area whose IOU value is greater than the target threshold. After that, the proportion of these reference images in the total number of images is determined. When the proportion is greater than the first target ratio, the location of these first target text areas is determined as the first candidate display position of the action name. If text information appears at a certain position in all n frames of images to be analyzed, and there are a total of m frames of images to be analyzed in the motion video, the frequency of appearance of the position is n / m.

[0055] In some embodiments, the video marking device can determine the candidate display position of the action in progress state through the overlapping information of each target text area. In the embodiment of this specification, the display position of the digital information is represented by the second target text area. If the second target text area overlaps in multiple frames of images, it can be indicated that the position of the area has digital information displayed for a long time, which is closer to the display characteristics of the action in progress state. Accordingly, the position has a higher probability of being the display position of the action in progress state.

[0056] By way of example, the above-mentioned comparison and analysis of the position information of each second target text area in multiple frames of images to determine the second candidate display position of action information containing the action progress status includes: determining the overlapping information of the positions of each second target text area in multiple frames of images in multiple frames of motion video; determining the position of the second target text area whose overlapping information is greater than the second target threshold and whose image accounts for a higher proportion of the number of frames in the multiple frames of images than the second target ratio as the second candidate display position of the action information containing the action progress status.

[0057] The method for determining the second candidate display position is basically the same as the method for determining the first candidate display position, and reference may be made to the related introduction about the first candidate display position, which will not be described here. The second target threshold may be equal to the first target threshold, or may be different from the first target threshold; the second target ratio may be equal to the first target ratio, or may be different from the first target ratio.

[0058] In the above step 106, based on the change information of the text content in the target text area corresponding to the candidate display position and the target change law corresponding to the action information, determining the action segmentation moment of the motion video may include: determining the action segmentation moment of the motion video based on the change information of the text content in the first target text area corresponding to the first candidate display position and the target change law corresponding to the action name; or determining the action segmentation moment of the motion video based on the change information of the text content in the second target text area corresponding to the second candidate display position and the target change law corresponding to the action progress status. For the target change law corresponding to the action name and the target change law corresponding to the action progress status, please refer to the above-mentioned introduction to step 106, which will not be elaborated here.

[0059] For example, the video marking device can determine the action segmentation moment of the motion video based on the change of the digital information at the second candidate display position. If the second candidate display position is not determined before step 106, such as the motion video does not display the action status information (such as the timing or count information for the action), the video marking device determines the action segmentation moment of the motion video based on the change of the text information at the first candidate display position. Since the timing or count information is usually displayed continuously during the action, and the action name may only be displayed during a part of the action, the action segmentation moment of the motion video can be determined more accurately and conveniently based on the change of the digital information at the second candidate display position.

[0060] In some embodiments, the above-mentioned determination of the action segmentation moment of the motion video based on the change information of the text content in the second target text area corresponding to the second candidate display position and the target change law corresponding to the action status may include the following steps s2 and s4.

[0061] Step s2: Based on the change information of the text content in the second target text area corresponding to the second candidate display position, the second candidate display position whose change information satisfies the target change rule corresponding to the action progress state is determined as the target display position of the action progress state. The target change rule corresponding to the action progress state includes: after the number representing the action progress state changes continuously for multiple times according to the reference change amount, it deviates from the rule of sudden change.

[0062] The video marking device can obtain the target change law corresponding to the action progress state, and determine the change information of the text content (that is, digital information) in each second target text area corresponding to each second candidate display position, and the change information represents the sequential change of the digital information in each frame image. If the change of the digital information corresponding to a second candidate display position satisfies the target change law, the second candidate display position is determined as the target display position of the action progress state.

[0063] For example, action 1 is timed for 60 seconds, action 2 is timed for 30 seconds, and then there is a 5-second break. Then the numbers in the display position of the action status change in the order of 60, 59, 58...3, 2, 1, 0. After that, action 1 ends, and the numbers change in the order of 30, 29...2, 1, 0. After that, action 2 ends, and the numbers change in the order of 5, 4, 3, 2, 1. The reference change amount corresponding to this change method is 1, and the change from 0 to 30 and the change from 0 to 5 are both mutations that deviate from the reference change amount. If the numbers in a second candidate display position change in the manner of the above example, it can be determined that the digital change information corresponding to the second candidate display position meets the target change law corresponding to the action status, and then the second candidate display position is determined as the target display position of the action status. For example, after this process, the second candidate display position where digital information such as year, date and video progress is located can be filtered out.

[0064] In the embodiments of this specification, only timing is used as an example for description. The counting method is similar to timing, such as counting in the positive direction, such as counting in the order of 0, 1, 2...n. The change from n to 0 during counting can be a sudden change that deviates from the reference change amount.

[0065] Step s4, determining the action segmentation moment of the motion video based on the image belonging to the second target text area where the number suddenly deviates from the reference change amount in the second target text area corresponding to the target display position.

[0066] After determining the target display position, the video marking device can determine the action segmentation moment of the motion video based on the changes in the digital information displayed at the target display position, such as based on the moment when the number suddenly changes. The moment when the number suddenly changes corresponds to the two frames of images before and after the sudden change, and the moment when any of the two frames of images is located can be used as an action segmentation moment of the motion video. If there are other images between the two frames of images in the motion video, the moment when any of the other images is located can also be used as the action segmentation moment of the motion video. Continuing with the example in the above step s2, the video marking device can use each time the number is 0 as the action segmentation moment of the motion video.

[0067] The motion video marking device can also determine the period between two moments of sudden change as an action segment of the motion video.

[0068] In some embodiments, the above-mentioned determination of the action segmentation moments of the motion video based on the change information of the text content in the first target text area corresponding to the first candidate display position and the target change rule corresponding to the action name includes the following steps s6 to s12.

[0069] Step s6: Based on the change information of the text content in the first target text area corresponding to the first candidate display position, determine the first candidate display position whose text content change frequency is within the target frequency range as the auxiliary display position.

[0070] The change information of the text content in the first target text area corresponding to the first candidate display position is similar to the change information corresponding to the second candidate display position in the aforementioned step s2. Please refer to the aforementioned related introduction and will not be elaborated here.

[0071] After determining the change of the text information in each first candidate display position, the video marking device can obtain the change frequency of the text information, and then determine whether the change frequency is within the target frequency range. If it is within the target frequency range, it is considered that the first candidate display position is more likely to be the display position of the action name, and then it can be determined as an auxiliary display position to facilitate subsequent analysis.

[0072] Regarding the display position of the action name, there may be interference areas in the sports video. For example, the position that appears very frequently may not necessarily display the action name, but may also display text describing the action or background music, etc. These texts may also appear frequently at a fixed position in the video. In the embodiment of this specification, for the first candidate display position (that is, the candidate display position for text information), based on the target frequency range, the positions where the text content has remained unchanged and has been changing are filtered out to retain the auxiliary display position where the text is changing but not frequently. The auxiliary display position is closer to the display position of the action name. The auxiliary display position can then be further analyzed.

[0073] Step s8: performing named entity recognition on the text content in each first target text area corresponding to the auxiliary display position.

[0074] The video marking device can perform named entity recognition (NER) on the text information displayed in each obtained auxiliary display position to determine whether each piece of text information represents an action name.

[0075] Step s10: Based on the named entity recognition result, when it is determined that the probability of each first target text area corresponding to the auxiliary display position containing the action name meets the probability condition, the auxiliary display position is determined as the target display position of the action name.

[0076] The probability condition may include a maximum probability that is greater than a probability threshold. For example, the probability threshold is 0.6. The video marking device may also determine the target occurrence probability of the text information containing the action name at each auxiliary display position based on the named entity recognition results for each text information displayed at each auxiliary display position, that is, the probability that the action name is contained in each first target text area corresponding to the auxiliary display position. Then, the target occurrence probabilities corresponding to each auxiliary display position may be compared to determine the maximum target occurrence probability that is greater than the probability threshold, and the auxiliary display position corresponding to the target occurrence probability may be determined as the target display position of the action name. If there is no auxiliary display position that meets the probability condition, the subsequent steps may not be performed, and other methods may be used to determine the action segmentation moments of the motion video.

[0077] Step s12: determining the action segmentation moments of the motion video based on the target change rule corresponding to the action name and the change information of the action name in the first target text area corresponding to the target display position.

[0078] For step s12, reference may be made to the related introduction of the aforementioned step s4 and step 106, and no additional elaboration is given here.

[0079] Optionally, the target change rule corresponding to the action name may also include displaying the action name for a period of time, stopping displaying the action name for a period of time, and then changing the action name. If the action segmentation time is determined based on the change of the text information, then for this target change rule, the video marking device can use the start display time of different action names as an action segmentation time.

[0080] In the embodiment of the present specification, after determining the action segmentation moment of the motion video, the motion video can be segmented based on the action segmentation moment to obtain multiple action segments. The frames of the motion video can be directly associated with each other based on the action segmentation moment to determine the image frames belonging to the same action follow-up section or the image frames of the same rest section, and each section is an action segment. The first frame and the last frame belonging to the same section are the positions where this action segment starts and ends, and these two positions are the dot positions required for video marking. The video marking device can judge whether the digital information in the second candidate display position (such as the timing or counting area) of the adjacent frame image is continuous. If it is not continuous and a sudden change occurs, for example, 0 changes to 60 or 60 becomes 0, the task is to switch the action, and the information in the current section is merged and ended; if it is continuous, the two frames of information are merged into the same section. If there is no second candidate display position, that is, the timing or counting area is missing, it is determined whether the text information in the first candidate display position corresponding to the action name in the adjacent frame is the same, if the action names between the two frames are the same, they are merged, if they are different, the current section information is merged and ended.

[0081] Determining the action name of each action segment in step 108 is equivalent to naming each action segment. The video marking device can use the action name that appears most frequently at the target display position of the action name in the divided action segments as the name of the action segment. Optionally, rest can also be regarded as an exercise action. If the image of an action segment does not include an area displaying the exercise action name, or there is a target field, such as a "rest" field, the action segment can be classified as a rest action segment.

[0082] In some embodiments, the above-mentioned step 108 determines the action name of each action segment for each action segment obtained by dividing the motion video according to the action segmentation moment, and may include: when the first candidate display position is determined, for each action segment obtained by dividing the motion video according to the action segmentation moment, determining the action name of each action segment based on the action information containing the action name displayed at the first candidate display position in each action segment.

[0083] If the motion video displays an action name, the first candidate display position can be determined accordingly, and the target display position of the action name can be further determined. In this case, the video marking device can determine the action name displayed at the first candidate display position (or target display position) in each action segment, and use the action name with the most occurrences as the action name of the action segment, that is, the name of the action segment.

[0084] In some embodiments, the above-mentioned step 108 determines the action name of each action segment for each action segment obtained by dividing the motion video according to the action segmentation moment, including: when the first candidate display position has not been determined, performing action recognition on the image frames in each action segment obtained by dividing the motion video according to the action segmentation moment, and determining the action name of each action segment.

[0085] If there is no display position of the action name in the sports video, and accordingly the first candidate display position is not determined, the video marking device can perform action recognition on the image frames in each action segment to determine the action name of the action segment according to the recognized action information. Optionally, the recognition can be achieved by performing similarity comparison with images of multiple actions.

[0086] Figure 2 FIG. 1 is a simplified flow chart of a video marking method provided in an embodiment of this specification. Figure 2 As shown, the video marking device can first extract frames of the motion video, perform OCR processing on each frame of the image to be processed, obtain text information and its text area, and divide each text information into text information and digital information. Perform preliminary filtering on the text area of ​​the text information, and analyze the overlapping information and frequency of occurrence of the text area to determine the candidate display position of the action name. Perform named entity recognition on the candidate display position and compare it with the action name to obtain the target display position of the action name. Perform preliminary screening on the text area of ​​the digital information, and analyze the overlapping information and frequency of occurrence of the text area to determine the candidate display position of the timing or counting information. Filter the candidate display position using the detection of digital mutation to obtain the target display position of the timing or counting information. Then, based on the timing or counting information at the target display position of the timing or counting information, the frame information of the motion video can be merged to generate multiple action segments. Based on the action name displayed at the target display position of the action name in each action segment, each action segment is named. Then, dotting information is output, and the motion video is marked based on the dotting information. The dot information may include the time interval and action name of each action segment, and the action segment may include an action segment of a training action and an action segment of a rest action.

[0087] In the embodiment of the specification, the video marking device can automatically generate action segment information for a sports video having at least one of action name and timing or count information, such as the time interval and action name of the action segment. Based on the information of each action segment, automatic marking of the sports video can be achieved, and the structured marking of a long sports video can be completed more quickly and with less manpower.

[0088] In summary, in the video marking method provided in the embodiments of this specification, for each frame image in a motion video, a target text area can be identified therein, and a candidate display position of the action information in the video can be determined based on the target text area. Correspondingly, based on the change in the text content in the candidate display position, the action segmentation moment of the motion video and the action name of each action segment obtained by segmentation are determined to achieve the marking of the motion video. In this way, automatic marking can be achieved by analyzing the motion video, and there is no need for manual marking of the motion video, which can improve the efficiency of video marking and reduce labor costs. For scenes with a large number of motion videos to be marked, the efficiency improvement is more significant.

[0089] Corresponding to the above-mentioned video marking method embodiment, this specification also provides a video marking device embodiment. Figure 3 FIG. 1 is a schematic diagram of a video marking device provided in an embodiment of the present specification. Figure 3 As shown, the video marking device includes: The recognition module 302 is used to recognize a target text area in each frame of the multiple frames of the motion video; An analysis module 304 is used to compare and analyze the position information of each target text area in multiple frames of images to determine a candidate display position of the action information; A first determination module 306 is used to determine the action segmentation time of the motion video based on the change information of the text content in the target text area corresponding to the candidate display position and the target change law corresponding to the action information; The second determination module 308 is used to determine the action name of each action segment obtained by dividing the motion video according to the action segmentation time; The marking module 310 is used to mark the motion video based on the action segmentation time and the action name of each action segment.

[0090] Optionally, the identification module 302 is used to: Perform text recognition on each frame of multiple frames of the motion video to obtain the text area where each text information in the image is located; In the text area where each text information is located, a first target text area where the text information of the character type is located and / or a second target text area where the text information of the number type is located is determined.

[0091] Optionally, the analysis module 304 is used to: Comparing and analyzing the position information of each first target text area in the multiple frames of images to determine a first candidate display position of the action information including the action name; and / or, comparing and analyzing the position information of each second target text area in the multiple frames of images to determine a second candidate display position of the action information including the action progress status; The first determination module 306 is used for: The action segmentation moments of the motion video are determined based on the change information of the text content in the first target text area corresponding to the first candidate display position and the target change rule corresponding to the action name; or, the action segmentation moments of the motion video are determined based on the change information of the text content in the second target text area corresponding to the second candidate display position and the target change rule corresponding to the action status.

[0092] Optionally, the second determining module 308 is used to: When the first candidate display position is determined, for each action segment obtained by dividing the motion video according to the action segmentation time, the action name of each action segment is determined based on the action information containing the action name displayed at the first candidate display position in each action segment.

[0093] Optionally, the second determining module 308 is used to: When the first candidate display position is not determined, action recognition is performed on image frames in each action segment obtained by dividing the motion video according to the action segmentation time, and the action name of each action segment is determined.

[0094] Optionally, the identification module 302 is used to: In the text area where each text information is located, the text area where the text information of the text type that does not meet the text filtering condition is located is determined as the first target text area; and / or the text area where the text information of the number type that meets the number filtering condition is located is determined as the second target text area; Among them, the text filtering conditions include: the text information is located in the reference area of ​​the image, the character length is less than the length threshold, and the text information includes at least one of the reference characters, and the number filtering conditions include that the text information consists of only numbers.

[0095] Optionally, the analysis module 304 is used to: Determining overlapping information of positions of first target text regions in multiple frames of images; Determine the position of the first target text area where the overlapping information is greater than the first target threshold and the frame number ratio of the image to which it belongs in the multiple frames of images is greater than the first target ratio as the first candidate display position of the action information containing the action name; and / or, Determining overlapping information of positions of each second target text region in multiple frames of images; The position of the second target text area where the overlapping information is greater than the second target threshold and the frame number ratio of the image to which it belongs in the multiple frames of images is higher than the second target ratio is determined as the second candidate display position containing the action information of the action progress status.

[0096] Optionally, the first determining module 306 is used to: Based on the change information of the text content in the first target text area corresponding to the first candidate display position, a first candidate display position whose text content change frequency is within the target frequency range is determined as the auxiliary display position; Performing named entity recognition on text content in each first target text area corresponding to the auxiliary display position; Based on the named entity recognition result, if it is determined that the probability that each first target text area corresponding to the auxiliary display position contains the action name meets the probability condition, the auxiliary display position is determined as the target display position of the action name; Based on the target change rule corresponding to the action name and the change information of the action name in the first target text area corresponding to the target display position, the action segmentation moment of the sports video is determined.

[0097] Optionally, the analysis module 304 is used to: Based on the change information of the text content in the second target text area corresponding to the second candidate display position, the second candidate display position whose change information satisfies the target change rule corresponding to the action progress state is determined as the target display position of the action progress state; wherein the target change rule corresponding to the action progress state includes: the rule that the number representing the action progress state deviates from the reference change amount and undergoes a sudden change after undergoing multiple continuous changes according to the reference change amount; Based on the image belonging to the second target text area in which the number in the second target text area corresponding to the target display position suddenly deviates from the reference change amount, the action segmentation moment of the motion video is determined.

[0098] Optionally, the identification module 302 is used to: Perform frame extraction processing on the motion video to determine multiple frames of images to be analyzed in the motion video; For each frame of the multiple frames, a target text area in the image is identified.

[0099] In summary, in the video marking device provided in the embodiments of this specification, for each frame image in the motion video, the target text area therein can be identified, and the candidate display position of the action information in the video can be determined based on the target text area. Correspondingly, based on the change of the text content in the candidate display position, the action segmentation moment of the motion video and the action name of each action segment obtained by segmentation are determined to achieve the marking of the motion video. In this way, automatic marking can be achieved by analyzing the motion video, and there is no need for manual marking of the motion video, which can improve the efficiency of video marking and reduce labor costs. For scenes with a large number of motion videos to be marked, the efficiency improvement is more significant.

[0100] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. As for the display device of the target image, since it is basically similar to the embodiment of the display method of the target image, the description is relatively simple, and the relevant parts can be referred to the partial description of the embodiment of the display method of the target image.

[0101] Figure 4 4 is a block diagram of a computing device provided in an embodiment of the present specification. The components of the computing device 400 include but are not limited to a memory 410 and a processor 420. The processor 420 is connected to the memory 410 via a bus 430, and the database 450 is used to store data.

[0102] The computing device 400 also includes an access device 440 that enables the computing device 400 to communicate via one or more networks 460. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 440 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0103] In one embodiment of the present specification, the above components of the computing device 400 and Figure 4 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 4 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0104] The computing device 400 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 400 may also be a mobile or stationary server.

[0105] The processor 420 is used to execute computer programs / instructions, which, when executed by the processor, implement the above Figure 1 or Figure 2 The method shown.

[0106] As for the computing device embodiment, since it is basically similar to the above-mentioned method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0107] One embodiment of the present specification also provides a computer-readable storage medium, which stores computer instructions, and when the computer instructions are executed by the processor, the steps of the above-mentioned image processing method are implemented. The computer instructions include computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable storage medium may include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0108] An embodiment of the present specification also provides a computer program product, including a computer program / instruction, which implements the steps of the above method when the computer program / instruction is executed in a processor.

[0109] As for the computer-readable storage medium embodiment and the computer program product embodiment, since they are basically similar to the above-mentioned method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0110] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0111] It should be noted that the above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of the present specification.

[0112] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0113] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can understand and use this specification well.

Claims

1. A video marking method, characterized in that: The method comprises: For each frame of the multiple frames of the motion video, identifying a target text area in the image; Comparing and analyzing the position information of each target text area in the multiple frames of images to determine candidate display positions of the action information; Determine the action segmentation time of the motion video based on the change information of the text content in the target text area corresponding to the candidate display position and the target change rule corresponding to the action information; Determine the action name of each action segment obtained by dividing the motion video according to the action segmentation time; The motion video is marked based on the action segmentation time and the action name of each action segment.

2. The method according to claim 1, characterized in that The step of identifying a target text area in each frame of the multiple frames of the motion video includes: Performing text recognition on each frame of multiple frames of the motion video to obtain a text area where each text information in the image is located; In the text area where each text information is located, a first target text area where the text information of the character type is located and / or a second target text area where the text information of the number type is located is determined.

3. The method according to claim 2, characterized in that The comparing and analyzing the position information of each target text area in the multiple frames of images to determine the candidate display position of the action information includes: Comparing and analyzing the position information of each first target text area in the multiple frames of images to determine a first candidate display position of the action information including the action name; and / or, comparing and analyzing the position information of each second target text area in the multiple frames of images to determine a second candidate display position of the action information including the action progress status; The determining of the action segmentation time of the motion video based on the change information of the text content in the target text area corresponding to the candidate display position and the target change rule corresponding to the action information includes: Based on the change information of the text content in the first target text area corresponding to the first candidate display position and the target change rule corresponding to the action name, the action segmentation moment of the motion video is determined; or, based on the change information of the text content in the second target text area corresponding to the second candidate display position and the target change rule corresponding to the action status, the action segmentation moment of the motion video is determined.

4. The method according to claim 2, characterized in that: The step of dividing the motion video into action segments according to the action segmentation moments and determining the action names of the action segments comprises: When the first candidate display position is determined, for each action segment obtained by dividing the motion video according to the action segmentation moments, the action name of each action segment is determined based on the action information containing the action name displayed at the first candidate display position in each action segment.

5. The method according to claim 2, characterized in that: The step of dividing the motion video into action segments according to the action segmentation moments and determining the action names of the action segments comprises: In the case that the first candidate display position is not determined, action recognition is performed on image frames in each action segment obtained by dividing the motion video according to the action segmentation moments, and an action name of each action segment is determined.

6. The method according to any one of claims 2 to 5, characterized in that: The step of determining, in the text area where each text information is located, a first target text area where the text information of the character type is located, and / or a second target text area where the text information of the number type is located, comprises: In the text area where the text information is located, the text area where the text information of the text type that does not meet the text filtering condition is located is determined as the first target text area; and / or the text area where the text information of the number type that meets the number filtering condition is located is determined as the second target text area; The text filtering condition includes at least one of: the text information is located in a reference area in the image, the character length is less than a length threshold, and the text information includes reference characters; the number filtering condition includes that the text information consists only of numbers.

7. The method according to claim 3, characterized in that The comparing and analyzing the position information of each first target text area in the multiple frames of images to determine the first candidate display position of the action information containing the action name includes: Determining overlapping information of positions of each first target text area in the multiple frames of images; Determine the position of the first target text region where the overlapping information is greater than the first target threshold and the frame number ratio of the image to which the region belongs in the multiple frames of images is greater than the first target ratio as the first candidate display position of the action information containing the action name; And / or, comparing and analyzing the position information of each second target text area in the multiple frames of images to determine the second candidate display position of the action information containing the action progress status includes: Determining overlapping information of positions of each second target text region in the multiple frames of images; The position of the second target text area where the overlapping information is greater than the second target threshold and the frame number ratio of the image to which it belongs in the multiple frames of images is higher than the second target ratio is determined as the second candidate display position containing the action information of the action progress status.

8. The method according to claim 3, characterized in that The determining of the action segmentation time of the motion video based on the change information of the text content in the first target text area corresponding to the first candidate display position and the target change rule corresponding to the action name includes: Based on the change information of the text content in the first target text area corresponding to the first candidate display position, a first candidate display position whose text content change frequency is within a target frequency range is determined as an auxiliary display position; Performing named entity recognition on text content in each first target text area corresponding to the auxiliary display position; Based on the named entity recognition result, if it is determined that the probability that each first target text area corresponding to the auxiliary display position contains an action name satisfies a probability condition, the auxiliary display position is determined as a target display position for the action name; Based on the target change rule corresponding to the action name and the change information of the action name in the first target text area corresponding to the target display position, the action segmentation moment of the sports video is determined.

9. The method according to claim 3, characterized in that: The determining of the action segmentation time of the motion video based on the change information of the text content in the second target text area corresponding to the second candidate display position and the target change rule corresponding to the action progress state includes: Based on the change information of the text content in the second target text area corresponding to the second candidate display position, the second candidate display position whose change information satisfies the target change rule corresponding to the action progress state is determined as the target display position of the action progress state; wherein the target change rule corresponding to the action progress state includes: after the number representing the action progress state changes continuously for multiple times according to the reference change amount, it deviates from the reference change amount and undergoes a sudden change; Based on an image belonging to a second target text area in which a number suddenly deviates from the reference change amount in a second target text area corresponding to the target display position, the action segmentation moment of the motion video is determined.

10. The method according to any one of claims 1 to 5, characterized in that: The step of identifying a target text area in each frame of the multiple frames of the motion video includes: Performing frame extraction processing on the motion video to determine a plurality of frames of images to be analyzed in the motion video; For each frame of the multiple frames of images, a target text area in the image is identified.

11. A video marking device, characterized in that: The video marking device comprises: A recognition module, for identifying a target text area in each frame of multiple frames of a motion video; An analysis module, used for comparing and analyzing the position information of each target text area in the multiple frames of images to determine a candidate display position of the action information; A first determination module is used to determine the action segmentation time of the motion video based on the change information of the text content in the target text area corresponding to the candidate display position and the target change law corresponding to the action information; A second determination module is used to determine the action name of each action segment obtained by dividing the motion video according to the action segmentation time; The marking module is used to mark the motion video based on the action segmentation time and the action name of each action segment.

12. A computing device, characterized in that: include: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the method according to any one of claims 1 to 10 is implemented.

13. A computer-readable storage medium, characterized in that: A computer program / instruction is stored, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

14. A computer program product, characterized in that The method comprises a computer program / instruction, which implements the method according to any one of claims 1 to 10 when executed by a processor.

Images