A method, device, equipment and storage medium for identifying sports scores in video

Abstract

The present application discloses a method, device, equipment and storage medium for identifying scores of events in a video, the method comprising: decoding a video of a target event to obtain a video frame image; determining a text area in the video frame image; determining multiple candidate text areas from the text area according to a standard score character aspect ratio; performing digital character recognition on the images of multiple candidate text areas based on a digital character recognition model to obtain multiple digital characters; converting multiple digital characters into event scores according to the position information of the candidate text areas corresponding to the multiple digital characters in the video frame image; calculating the distance between the positions corresponding to two score values ​​in the event score; when the distance matches the standard score distance, determining the event score as the event score of the target event. The technical solution provided by the present application can improve the accuracy and processing speed of score recognition and meet the real-time requirements in actual business needs.

Description

Technical Field

[0001] The present application relates to the field of Internet communication technology, and in particular to a method, device, equipment and storage medium for identifying a game score in a video. Background Art

[0002] In recent years, with the research and progress of artificial intelligence (AI) technology, AI technology has been widely used in many fields. Among them, the application of AI in the field of image content recognition is an important branch of AI application.

[0003] At present, in order to report the game situation in a timely manner during the live broadcast of some sports events, it is often necessary to identify the game scores from the video. In the prior art, the game scores in the video are often directly identified based on the deep learning model. Specifically, the deep learning model is first used to detect the text area, and then the deep learning model is used to identify the score in the text area. However, the prior art directly uses the deep learning model to identify the score, which requires extracting many features and often cannot guarantee the recognition accuracy. It also brings about the complex network structure of the deep learning model, the slow processing speed, and the difficulty in meeting the real-time requirements in actual business needs. Therefore, it is necessary to provide a more reliable or more effective solution. Summary of the invention

[0004] The present application provides a method, device, equipment and storage medium for identifying game scores in a video, which can improve the accuracy and processing speed of score identification and meet the real-time requirements in actual business needs.

[0005] On the one hand, the present application provides a method for identifying a match score in a video, the method comprising:

[0006] Decode the video of the target event to obtain a video frame image;

[0007] Determine a text area in the video frame image;

[0008] Determine a plurality of candidate text regions from the text region according to a standard character aspect ratio;

[0009] Based on the digital character recognition model, digital character recognition is performed on the images of the multiple candidate text regions respectively to obtain multiple digital characters;

[0010] Convert the multiple numeric characters into a game score according to the position information of the candidate text regions corresponding to the multiple numeric characters in the video frame image;

[0011] Calculate the distance between the positions corresponding to two score values ​​in the match score;

[0012] When the interval matches the standard score interval, the event score is determined to be the event score of the target event.

[0013] In another aspect, a device for identifying a match score in a video is provided, the device comprising:

[0014] A video decoding module, used to decode the video of the target event to obtain a video frame image;

[0015] A text area determination module, used to determine the text area in the video frame image;

[0016] A candidate text region determination module, used for determining a plurality of candidate text regions from the text region according to a standard character aspect ratio;

[0017] A digital character recognition module, used to perform digital character recognition on the images of the plurality of candidate text regions respectively based on a digital character recognition model to obtain a plurality of digital characters;

[0018] A match score conversion module, configured to convert the plurality of numeric characters into match scores according to position information of candidate text regions corresponding to the plurality of numeric characters in the video frame image;

[0019] A spacing calculation module, used to calculate the spacing between positions corresponding to two score values ​​in the event score;

[0020] The event score determination module is used to determine that the event score is the event score of the target event when the interval matches the standard score interval.

[0021] On the other hand, a device for identifying event scores in a video is provided, the device comprising a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the method for identifying event scores in a video as described above.

[0022] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, a code set or an instruction set is stored in the storage medium, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the method for identifying event scores in a video as described above.

[0023] The method, device, equipment and storage medium for identifying the scores of events in a video provided by this application have the following technical effects:

[0024] This application can combine the digital character recognition model and the standard score character aspect ratio and standard score spacing of the event, which can improve the score recognition accuracy and processing speed and meet the real-time requirements in actual business needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 is a schematic diagram of an application environment provided by an embodiment of the present application;

[0027] Figure 2 It is a flowchart of a method for identifying a match score in a video provided by an embodiment of the present application;

[0028] Figure 3 It is a flow chart of a method for decoding a video of a target event to obtain a video frame image provided by an embodiment of the present application;

[0029] Figure 4 is a flow chart of a method for determining a text area in a video frame image provided by an embodiment of the present application;

[0030] Figure 5 is a flowchart of a method for determining a plurality of candidate text regions from the text region according to a standard character aspect ratio provided by an embodiment of the present application;

[0031] Figure 6 It is a structural diagram of a preset deep learning model provided in an embodiment of the present application;

[0032] Figure 7 It is a flowchart of a training and learning process of a digital character recognition model provided in an embodiment of the present application;

[0033] Figure 8 A schematic diagram of performing digital character recognition on images of the plurality of candidate text regions based on a digital character recognition model to obtain a plurality of digital characters is provided in an embodiment of the present application;

[0034] Fig. 9 It is a flow chart of converting the plurality of digital characters into a match score according to the position information of the candidate text regions corresponding to the plurality of digital characters in the video frame image provided by an embodiment of the present application;

[0035] Fig.10It is a structural schematic diagram of a device for identifying a match score in a video provided by an embodiment of the present application;

[0036] Fig.11 It is a structural diagram of a client provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0038] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0039] Artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning.

[0040] In recent years, with the research and progress of artificial intelligence technology, artificial intelligence technology has been widely used in many fields. The solution provided in the embodiment of the present application involves artificial intelligence machine learning / deep learning and other technologies, which are specifically described by the following embodiments:

[0041] See also Figure 1 , Figure 1 is a schematic diagram of an application environment provided by an embodiment of the present application, such as Figure 1 As shown, the application environment may include at least a terminal 01 , a server 02 and a terminal 03 .

[0042] In the embodiment of this specification, the terminal 01 may include physical devices such as smart phones, desktop computers, tablet computers, laptop computers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, smart wearable devices, etc., and may also include software running in physical devices, such as applications, etc. The operating system running on the terminal 01 in the embodiment of this application may include but is not limited to Android system, IOS system, Linux, Windows, etc. In the embodiment of this specification, the terminal 01 can be used to play the event video.

[0043] In the embodiment of this specification, the server 02 may include an independently operated server, or a distributed server, or a server cluster composed of multiple servers. The server 02 may include a network communication unit, a processor, a memory, etc. Specifically, the server 02 may be used to train a digital character recognition model.

[0044] In the embodiment of this specification, the terminal 03 may include physical devices such as smart phones, desktop computers, tablet computers, laptop computers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, smart wearable devices, etc., and may also include software running in physical devices, such as applications, etc. The operating system running on the terminal 03 in the embodiment of this application may include but is not limited to Android system, IOS system, Linux, Windows, etc. In the embodiment of this specification, the terminal 03 can obtain the event video from the terminal 01, and identify the event score in the event video by combining the digital character recognition model trained by the server 02 and the score reference information of the event (standard score character aspect ratio, standard score spacing).

[0045] In addition, it should be noted that Figure 1 What is shown is only an application environment for the match score recognition in the video. In actual applications, for example, the training of a digital character recognition model can also be processed on a device that provides the match score recognition service in the video, or the match video can also be played on a device that provides the match score recognition service in the video.

[0046] The following describes a method for identifying scores of events in a video of this application. Figure 2It is a flow chart of a method for identifying game scores in a video provided by an embodiment of the present application. This specification provides method operation steps as described in the embodiment or flow chart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in sequence or in parallel (for example, in a parallel processor or multi-threaded processing environment) according to the method shown in the embodiment or the accompanying drawings. Specifically, Figure 2 As shown, the method may include:

[0047] S201: Decode the video of the target event to obtain a video frame image.

[0048] In the embodiments of the present specification, the target event may include but is not limited to sports, entertainment and other competitions; accordingly, the video of the target event may be a recorded video of the game scene showing the event score of the target event. Specifically, the event video may include but is not limited to a live event video.

[0049] In practical applications, the video encapsulation format may include but is not limited to AVI (Audio Video Interleaved, audio video interleaved format), MP4 (Moving Picture Experts Group 4, dynamic picture experts group), FLV (FLASH VIDEO, streaming media format) and other encapsulation formats.

[0050] In the embodiments of this specification, Figure 3 As shown, the video decoding of the target event video to obtain the video frame image may include:

[0051] S2011: Obtain video segments of the target event.

[0052] S2013: Perform video decoding on the video slice to obtain the video frame image.

[0053] In practical applications, for live video streams, video fragments are obtained each time. Interdependently, the video fragments can be decoded to obtain video frame images. Specifically, in practical applications, the video fragments can generally be several seconds, such as 5 seconds. Generally, each second of video fragments will correspond to multiple frames of video images after decoding, such as 25 frames of video images. Interdependently, the video frame image can include multiple frames of video images after the video fragment is decoded.

[0054] In the embodiments of the present specification, a decoder may be used to decode the video slices. Specifically, the decoder may include but is not limited to a software decoder, a hardware decoder, and a wireless decoder.

[0055] S203: Determine a text area in the video frame image.

[0056] In practical applications, the display area of ​​the game score in the video frame image is often relatively fixed. In the embodiment of this specification, the text area can be detected based on the maximum extreme stable region algorithm (MSER). Specifically, Figure 4 As shown, determining the text area in the video frame image may include:

[0057] S2031: Adjust the grayscale value of the video frame image to obtain a binary image.

[0058] S2033: Calculate the area change rate of each connected region in the binary image corresponding to the gray value adjustment.

[0059] S2035: Determine the connected region whose area change rate is less than or equal to a preset threshold as the text region.

[0060] In the embodiment of the present specification, the preset threshold may be a value greater than or equal to 0. Generally, the display area of ​​the game score in the video frame image is often relatively fixed, and the preset threshold may be set to a value slightly greater than 0, such as 0.05.

[0061] S205: Determine a plurality of candidate text regions from the text region according to a standard character aspect ratio.

[0062] In actual applications, different broadcasters often use different playback templates when playing different event videos. Accordingly, when different event broadcasters play different events, the aspect ratio of each digital character in the event score in the event video played is often fixed. Accordingly, multiple candidate text regions are filtered out from the text region according to the standard score character aspect ratio. Specifically, Figure 5 As shown, it may include:

[0063] S2051: Obtain the standard score character aspect ratio.

[0064] In the embodiment of the present specification, the standard score character aspect ratio may be the aspect ratio of each digital character corresponding to the event score in the playback template used by the broadcaster playing the target event when playing the target event.

[0065] S2053: Determine the aspect ratio of the text area.

[0066] S2055: Determine the text regions whose aspect ratios match the standard character aspect ratios as the multiple candidate text regions.

[0067] In a specific embodiment, a text area whose aspect ratio is consistent with the standard score character aspect ratio can be determined as the multiple candidate text areas; in another specific embodiment, a text area whose aspect ratio differs from the standard score character aspect ratio by a value less than or equal to a certain threshold can be determined as the multiple candidate text areas.

[0068] S207: Performing digital character recognition on the images of the multiple candidate text regions respectively based on a digital character recognition model to obtain multiple digital characters.

[0069] In the embodiment of the present specification, after filtering out candidate text regions that match the width and height of the score characters of the target event from the text region of the video frame image, considering that in addition to the numeric text region, there are often text regions of characters such as text, correspondingly, numeric characters can be recognized from the candidate text region based on a numeric character recognition model that has the ability to recognize numeric characters. In the embodiment of the present specification, the character recognition that can be recognized by the numeric character recognition model can include numeric characters from 0 to 9. In addition, when the image does not include numeric characters from 0 to 9, the recognition result can be other categories.

[0070] In an embodiment of the present specification, the digital character recognition model may be a model obtained by training and learning a preset deep learning model for digital character recognition based on a plurality of training digital character images with digital character annotations.

[0071] Specifically, in the embodiments of this specification, Figure 6 As shown, the preset deep learning model may include three convolutional layers and two fully connected layers; wherein each convolutional layer is sequentially followed by an activation layer and a pooling layer.

[0072] Specifically, the convolution layer can be used to extract image features. In a specific embodiment, the convolution kernel size of the convolution layer can be 3x3, with a step size of 1, wherein the convolution kernel parameters of the first convolution layer are 3x3x3x16, the convolution kernel parameters of the second convolution layer are 16x3x3x32, and the convolution kernel parameters of the third convolution layer are 32x3x3x32.

[0073] Specifically, the fully connected layer can be used as a connection layer between the nodes of the upper and lower layers, establish a connection relationship between the node data obtained from the upper and lower layers, and send the output value to the classifier (such as a softmax classifier). In a specific embodiment, the dimension of the first fully connected layer is 128 dimensions, and the dimension of the second fully connected layer is 11 dimensions.

[0074] Specifically, the pooling layer can perform a downsampling operation on the output of the previous layer, that is, return the maximum value in the sampling window as the downsampled output. On the one hand, it can make the image smaller and simplify the computational complexity; on the other hand, it can perform feature compression and extract the main features. In a specific embodiment, the step size of the pooling layer can be 2.

[0075] In practical applications, the output of each layer is a linear function of the input of the previous layer. Considering that data is often not linearly separable in practical applications, nonlinear factors can be introduced by adding activation functions. Take the activation function Relu (rectified linear unit) as an example. Specifically, Relu is a layer that corrects the data results of the previous layer. It can convert all inputs less than 0 in the previous layer into 0 before output, and the output greater than 0 remains unchanged.

[0076] The classification layer can be used to determine a probability value of a numeric character being included in an image, and determine the numeric characters contained in the image based on the probability value.

[0077] In a specific embodiment, Figure 7 As shown, the training and learning of the digital character recognition model may include the following steps:

[0078] S701: Acquire multiple training digital character images;

[0079] In the embodiment of this specification, the training digital character image includes an image of a digital character. Specifically,

[0080] S703: performing digital character labeling on the training digital character image;

[0081] S705: Performing training and learning for digital character recognition based on the preset deep learning model of the plurality of training digital character image pairs, and obtaining a predicted probability value that the plurality of training digital character images include a specified digital character;

[0082] S707: Calculate the error values ​​between the predicted digital characters and the marked digital characters of the plurality of training digital character images based on a preset loss function and the predicted probability value.

[0083] In a specific embodiment, the preset loss function may be:

[0084]

[0085] Among them, loss represents the error value, k represents the digital character annotated in the training digital character image, and pk represents the probability value of including the digital character k in the training digital character image.

[0086] In addition, it should be noted that the preset loss function in the embodiments of this specification is not limited to the above In practical applications, other loss functions may also be included, and the embodiments of this specification are not limited to the above.

[0087] S709: Determine whether the error value meets a preset condition;

[0088] In the embodiment of the present specification, the preset condition can be satisfied when the error value corresponding to the second percentage of training digital character images in the multiple training digital character images is less than or equal to a specified threshold, or when the current error value corresponding to the second percentage of training digital character images in the multiple training digital character images and the error value after the last training learning are less than the specified threshold.

[0089] Specifically, the second percentage may be a value less than or equal to 100% that is set in combination with actual applications.

[0090] In the embodiments of this specification, the specified threshold can be set in combination with the requirements for the digital character recognition accuracy of the digital character recognition model in actual applications. Generally, the larger the specified threshold, the higher the recognition accuracy of the trained digital character recognition model, but the training time is longer; conversely, the smaller the specified threshold, the lower the recognition accuracy of the trained digital character recognition model, but the training time is shorter.

[0091] S711: When the error value does not meet the preset conditions, adjust the model parameters of the preset deep learning model based on the gradient descent method, and repeat the above training and learning steps;

[0092] S713: When the error value meets the preset condition, the current preset deep learning model is used as the digital character recognition model.

[0093] Specifically, Figure 8 As shown, Figure 8 A method provided in an embodiment of the present application is to perform digital character recognition on images of the multiple candidate text areas based on a digital character recognition model to obtain a schematic diagram of multiple digital characters. Specifically, in the process of performing digital character recognition on images of the multiple candidate text areas based on the digital character recognition model, after passing through each layer of the digital character recognition model, the last fully connected layer inputs the feature vector of each image to the classification layer, and the classification layer obtains the probability that the image of each candidate text area includes 0-9 and other types of characters, and takes the character with the highest probability value as the digital character included in the image of the candidate text area.

[0094] In the embodiments of the present specification, digital characters are recognized in the image of the candidate text area by a digital character recognition model that has the ability to recognize digital characters, so that the candidate text areas with non-digital characters in the candidate text areas can be filtered out, and the digital characters can be accurately recognized. The digital character recognition model used has a simple structure and can greatly improve the recognition speed.

[0095] S209: Convert the multiple numeric characters into game scores according to the position information of the candidate text regions corresponding to the multiple numeric characters in the video frame image.

[0096] In the embodiment of the present specification, after the digital characters are recognized, the multiple digital characters can be converted into the game scores according to the position information of the candidate text areas corresponding to the multiple digital characters in the video frame image. Specifically, Fig. 9 As shown, this may include:

[0097] S2091: Determine the position information of the candidate text regions corresponding to the multiple numeric characters in the video frame image.

[0098] S2093: Score the numeric characters corresponding to the adjacent candidate text regions according to the position information.

[0099] S2095: When the vertical coordinates in the position information corresponding to the two scores are consistent, the two scores are used as the game scores.

[0100] S211: Calculate the distance between the positions corresponding to two score values ​​in the game score.

[0101] S213: When the interval matches the standard score interval, determining the event score to be the event score of the target event.

[0102] In actual applications, since different broadcasters often use different playback templates when playing different event videos, in order to improve the accuracy of score recognition, in the embodiments of this specification, the event scores can be verified in combination with the standard score intervals corresponding to different playback templates. Specifically, the standard score interval can be the interval between two event scores in the playback template used by the broadcaster that plays the target event when playing the target event.

[0103] In a specific embodiment, when the distance between the positions corresponding to the two score values ​​in the event score is consistent with the standard score distance, the event score can be determined to be the event score of the target event. In another specific embodiment, when the distance between the positions corresponding to the two score values ​​in the event score is less than or equal to a certain threshold value and the standard score distance, the event score can be determined to be the event score of the target event.

[0104] In the embodiment of the present specification, the scores recognized in the corresponding video frame images when the target event video is played can be sorted according to the video playing time, thereby achieving the real-time score acquisition of the event.

[0105] It can be seen from the technical solutions provided by the above embodiments of this specification that in the embodiments of this specification, by determining the text area from the video corresponding to the video frame image, combining the standard score character aspect ratio to screen out multiple candidate text areas that meet the character size in the actual score, and then combining the digital character recognition model with the ability to recognize digital characters to recognize digital characters on the image of the candidate text area, the digital characters in the candidate text area can be accurately recognized; then, the digital characters are converted into event scores in combination with the position information of the candidate text area; finally, the event scores are verified in combination with the standard score spacing, effectively ensuring the accuracy of the recognized event scores. The technical solutions provided by the embodiments of this application can improve the accuracy of score recognition, simplify the complex network structure of the deep learning model, improve the processing speed, and meet the real-time requirements in actual business needs.

[0106] The present application also provides a device for identifying scores of sports events in a video. Fig.10 As shown, the device comprises:

[0107] The video decoding module 1010 is used to decode the video of the target event to obtain a video frame image;

[0108] A text region determination module 1020, configured to determine a text region in the video frame image;

[0109] A candidate text region determining module 1030, configured to determine a plurality of candidate text regions from the text region according to a standard character aspect ratio;

[0110] A digital character recognition module 1040 is used to perform digital character recognition on the images of the plurality of candidate text regions based on a digital character recognition model to obtain a plurality of digital characters;

[0111] A match score conversion module 1050, configured to convert the plurality of numeric characters into match scores according to position information of the candidate text regions corresponding to the plurality of numeric characters in the video frame image;

[0112] A spacing calculation module 1060, used to calculate the spacing between positions corresponding to two score values ​​in the game score;

[0113] The event score determination module 1070 is configured to determine that the event score is the event score of the target event when the interval matches the standard score interval.

[0114] In some embodiments, the text area determination module includes:

[0115] A binary image conversion unit, used for adjusting the gray value of the video frame image to obtain a binary image;

[0116] A grayscale value adjustment unit, used for adjusting the grayscale value of the binary image;

[0117] An area change rate calculation unit, used to calculate the area change rate of each connected region in the binary image when the gray value is adjusted;

[0118] The text region determining unit is used to determine the connected region whose area change rate is less than or equal to a preset threshold as the text region.

[0119] In some embodiments, the video decoding module includes:

[0120] A video segment acquisition unit, used to acquire video segments of a target event;

[0121] The video decoding unit is used to perform video decoding on the video slices to obtain the video frame images.

[0122] In some embodiments, the candidate text region determination module includes:

[0123] A standard score character aspect ratio acquisition unit, used to acquire the standard score character aspect ratio;

[0124] A text area aspect ratio determination unit, used to determine the aspect ratio of the text area;

[0125] The candidate text region determining unit is used to determine the text regions whose aspect ratio matches the standard character aspect ratio as the multiple candidate text regions.

[0126] In some embodiments, the event score conversion module includes:

[0127] A position information determining unit, used to determine position information of the candidate text regions corresponding to the plurality of numeric characters in the video frame image;

[0128] A score determination unit, configured to assign scores to the numeric characters corresponding to the adjacent candidate text regions according to the position information;

[0129] The match score determination unit is used to use the two scores as the match score when the ordinates in the position information corresponding to the two scores are consistent.

[0130] In some embodiments, the apparatus further comprises:

[0131] A training image acquisition module is used to acquire multiple training digital character images;

[0132] A digital character labeling module, used for labeling the training digital character images with digital characters;

[0133] A training and learning module, used for performing training and learning of digital character recognition based on a preset deep learning model of the plurality of training digital character image pairs, and obtaining a predicted probability value of a specified digital character being included in the plurality of training digital character images;

[0134] An error value calculation module, used for calculating the error values ​​between the predicted digital characters and the marked digital characters of the plurality of training digital character images based on a preset loss function and the predicted probability value;

[0135] A judging module, used to judge whether the error value meets a preset condition;

[0136] A model parameter adjustment module, used to adjust the model parameters of the preset deep learning model based on the gradient descent method when the error value does not meet the preset conditions, and repeat the above training and learning steps;

[0137] The digital character recognition model determination module is used to use the current preset deep learning model as the digital character recognition model when the error value meets the preset conditions.

[0138] In some embodiments, the preset deep learning model may include:

[0139] Three convolutional layers, two fully connected layers, and a classification layer;

[0140] Among them, each convolution layer is sequentially followed by an activation layer and a pooling layer.

[0141] The device and method embodiments in the device embodiments are based on the same application concept.

[0142] An embodiment of the present application provides a device for identifying a match score in a video, the device comprising a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement a method for identifying a match score in a video as provided in the above method embodiment.

[0143] The memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required for functions, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory can also include a memory controller to provide the processor with access to the memory.

[0144] The method embodiments provided in the embodiments of the present application can be executed in a client (mobile terminal, computer terminal), a server or a similar computing device. Taking running on a client as an example, Fig.11 is a schematic diagram of the structure of a client provided in an embodiment of the present application, such as Fig.11 As shown, the client can be used to implement the information interaction method provided in the above embodiment. Specifically:

[0145] The client may include components such as an RF (Radio Frequency) circuit 1110, a memory 1120 including one or more computer-readable storage media, an input unit 1130, a display unit 1140, a sensor 1150, an audio circuit 1160, a WiFi (wireless fidelity) module 1170, a processor 1180 including one or more processing cores, and a power supply 1190. Those skilled in the art will appreciate that Fig.11 The client structure shown in the figure does not constitute a limitation on the client, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0146] The RF circuit 1111 can be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information of the base station, it is handed over to one or more processors 1180 for processing; in addition, the data related to the uplink is sent to the base station. Generally, the RF circuit 1111 includes but is not limited to an antenna, at least one amplifier, a tuner, one or more oscillators, a user identity module (SIM) card, a transceiver, a coupler, an LNA (Low Noise Amplifier), a duplexer, etc. In addition, the RF circuit 1111 can also communicate with the network and other clients through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System of Mobilecommunication), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), LTE (Long Term Evolution), email, SMS (Short Messaging Service), etc.

[0147] The memory 1120 can be used to store software programs and modules, and the processor 1180 executes various functional applications and data processing by running the software programs and modules stored in the memory 1120. The memory 1120 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, application programs required for functions, etc.; the data storage area may store data created according to the use of the client, etc. In addition, the memory 1120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 1120 may also include a memory controller to provide the processor 1180 and the input unit 1130 with access to the memory 1120.

[0148] The input unit 1130 can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control. Specifically, the input unit 1130 may include a touch-sensitive surface 1131 and other input devices 1132. The touch-sensitive surface 1131, also known as a touch display screen or touch pad, can collect user touch operations on or near it (such as operations performed by users using fingers, styluses, or any other suitable objects or accessories on or near the touch-sensitive surface 1131), and drive corresponding connection devices according to a pre-set program. Optionally, the touch-sensitive surface 1131 may include a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into touch point coordinates, and then sends it to the processor 1180, and can receive and execute commands sent by the processor 1180. In addition, the touch-sensitive surface 1131 may be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface 1131, the input unit 1130 may further include other input devices 1132. Specifically, the other input devices 1132 may include, but are not limited to, one or more of a physical keyboard, a function key (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like.

[0149] The display unit 1140 can be used to display information input by the user or information provided to the user and various graphical user interfaces of the client, which can be composed of graphics, text, icons, videos and any combination thereof. The display unit 1140 may include a display panel 1141. Optionally, the display panel 1141 may be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode) and the like. Further, the touch-sensitive surface 1131 may cover the display panel 1141. When the touch-sensitive surface 1131 detects a touch operation on or near it, it is transmitted to the processor 1180 to determine the type of the touch event, and then the processor 1180 provides corresponding visual output on the display panel 1141 according to the type of the touch event. Among them, the touch-sensitive surface 1131 and the display panel 1141 can be two independent components to realize the input and output functions, but in some embodiments, the touch-sensitive surface 1131 and the display panel 1141 can also be integrated to realize the input and output functions.

[0150] The client may also include at least one sensor 1150, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1141 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 1141 and / or the backlight when the client moves to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary, which can be used for applications that identify the posture of the client (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that can also be configured on the client, they will not be repeated here.

[0151] The audio circuit 1160, the speaker 1161, and the microphone 1162 can provide an audio interface between the user and the client. The audio circuit 1160 can transmit the electrical signal converted from the received audio data to the speaker 1161, which is converted into a sound signal for output; on the other hand, the microphone 1162 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1160 and converted into audio data, and then the audio data is output to the processor 1180 for processing, and then sent to another client through the RF circuit 1111, or the audio data is output to the memory 1120 for further processing. The audio circuit 1160 may also include an earplug jack to provide communication between an external headset and the client.

[0152] WiFi is a short-range wireless transmission technology. The client can help users send and receive emails, browse web pages, and access streaming media through the WiFi module 1170, which provides users with wireless broadband Internet access. Fig.11 A WiFi module 1170 is shown, but it is understandable that it is not a necessary component of the client and can be omitted as needed without changing the essence of the invention.

[0153] The processor 1180 is the control center of the client, and uses various interfaces and lines to connect various parts of the entire client. By running or executing software programs and / or modules stored in the memory 1120, and calling data stored in the memory 1120, the processor 1180 performs various functions of the client and processes data, thereby monitoring the client as a whole. Optionally, the processor 1180 may include one or more processing cores; preferably, the processor 1180 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1180.

[0154] The client also includes a power supply 1190 (such as a battery) for supplying power to each component. Preferably, the power supply can be logically connected to the processor 1180 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions. The power supply 1190 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0155] Although not shown, the client may also include a camera, a Bluetooth module, etc., which will not be described in detail herein. Specifically in this embodiment, the display unit of the client is a touch screen display, and the client also includes a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors to execute instructions in the method embodiment of the present invention.

[0156] An embodiment of the present application also provides a storage medium, which can be set in a server to store at least one instruction, at least one program, a code set or an instruction set related to a method for identifying a match score in a video in a method embodiment. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method for identifying a match score in a video provided in the above method embodiment.

[0157] Optionally, in this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0158] It can be seen from the embodiments of the method, device, equipment or storage medium for identifying the scores of events in the video provided by the present application that in the present application, by determining the text area from the video frame image corresponding to the video, multiple candidate text areas that meet the character size in the actual score are screened out in combination with the standard score character aspect ratio, and then, the digital characters in the image of the candidate text area are recognized by combining the digital character recognition model with the ability to recognize digital characters, so that the digital characters in the candidate text area can be accurately recognized; then, the digital characters are converted into the scores of events in combination with the position information of the candidate text area; finally, the scores of events are verified in combination with the standard score spacing, effectively ensuring the accuracy of the recognized scores of events. The technical solution provided by the embodiments of the present application can improve the accuracy of score recognition, simplify the complex network structure of the deep learning model, improve the processing speed, and meet the real-time requirements in actual business needs.

[0159] It should be noted that the above-mentioned sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above-mentioned specific embodiments of this specification are described. Other embodiments are within the scope of the attached claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0160] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0161] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by instructing the relevant hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0162] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for identifying sports scores in a video, characterized in that: The method comprises: Decode the video of the target event to obtain a video frame image; Determine a text area in the video frame image; Get the standard score character aspect ratio; Determining the aspect ratio of the text area in the video frame image; Determine the text regions whose aspect ratios match the standard character aspect ratios as multiple candidate text regions; Based on the digital character recognition model, digital character recognition is performed on the images of the multiple candidate text regions respectively to obtain multiple digital characters; Convert the multiple digital characters into a game score according to the position information of the candidate text regions corresponding to the multiple digital characters in the video frame image; Calculate the distance between the positions corresponding to two score values ​​in the match score; When the interval matches the standard score interval, the event score is determined to be the event score of the target event, and the standard score interval is the interval between two event scores in the playback template used by the broadcaster that broadcasts the target event when broadcasting the target event.

2. The method according to claim 1, characterized in that Determining the text area in the video frame image includes: Adjusting the grayscale value of the video frame image to obtain a binary image; Adjusting the grayscale value of the binary image, and calculating the area change rate of each connected region in the binary image when the grayscale value is adjusted; A connected region whose area change rate is less than or equal to a preset threshold is determined as the text region.

3. The method according to claim 1, characterized in that The video decoding of the target event video to obtain the video frame image comprises: Get the video segments of the target event; The video slices are decoded to obtain the video frame images.

4. The method according to claim 1, characterized in that: The converting the plurality of digital characters into a match score according to the position information of the candidate text regions corresponding to the plurality of digital characters in the video frame image comprises: Determine position information of candidate text regions corresponding to the plurality of numeric characters in the video frame image; Score the numeric characters corresponding to the adjacent candidate text regions according to the position information; When the vertical coordinates in the position information corresponding to two scores are consistent, the two scores are used as the game scores.

5. The method according to claim 1, characterized in that The method further comprises: Obtain multiple training digital character images; Performing digital character labeling on the training digital character image; Performing training and learning for digital character recognition based on a preset deep learning model of the plurality of training digital character image pairs, and obtaining a predicted probability value of a specified digital character being included in the plurality of training digital character images; Calculating the error values ​​between the predicted digital characters and the marked digital characters of the plurality of training digital character images based on a preset loss function and the predicted probability value; Determining whether the error value meets a preset condition; When the error value does not meet the preset conditions, adjusting the model parameters of the preset deep learning model based on the gradient descent method, and repeating the above training and learning steps; When the error value meets the preset condition, the current preset deep learning model is used as the digital character recognition model.

6. The method according to claim 5, characterized in that The preset deep learning model includes: Three convolutional layers, two fully connected layers, and a classification layer; Among them, each convolution layer is sequentially followed by an activation layer and a pooling layer.

7. A device for identifying scores of sports events in a video, characterized in that: The device comprises: A video decoding module, used to decode the video of the target event to obtain a video frame image; A text area determination module, used to determine the text area in the video frame image; A candidate text area determination module is used to determine multiple candidate text areas from the text area according to the standard score character aspect ratio; the candidate text area determination module includes: a standard score character aspect ratio acquisition unit, used to acquire the standard score character aspect ratio; a text area aspect ratio determination unit, used to determine the aspect ratio of the text area; a candidate text area determination unit, used to determine the text area whose aspect ratio matches the standard score character aspect ratio as the multiple candidate text areas; A digital character recognition module, used to perform digital character recognition on the images of the plurality of candidate text regions respectively based on a digital character recognition model to obtain a plurality of digital characters; A match score conversion module, configured to convert the plurality of numeric characters into match scores according to position information of candidate text regions corresponding to the plurality of numeric characters in the video frame image; A spacing calculation module, used to calculate the spacing between positions corresponding to two score values ​​in the event score; The event score determination module is used to determine that the event score is the event score of the target event when the interval matches the standard score interval, and the standard score interval is the interval between two event scores in the playback template used by the broadcaster that broadcasts the target event when broadcasting the target event.

8. The device according to claim 7, characterized in that The text area determination module comprises: A binary image conversion unit, used for adjusting the gray value of the video frame image to obtain a binary image; A gray value adjustment unit, used for adjusting the gray value of the binary image; An area change rate calculation unit, used to calculate the area change rate of each connected region in the binary image when the gray value is adjusted; The text region determining unit is used to determine the connected region whose area change rate is less than or equal to a preset threshold as the text region.

9. The device according to claim 7, characterized in that The video decoding module comprises: A video segment acquisition unit, used to acquire video segments of a target event; The video decoding unit is used to perform video decoding on the video slices to obtain the video frame images.

10. The device according to claim 7, characterized in that The event score conversion module includes: A position information determining unit, used to determine position information of the candidate text regions corresponding to the plurality of numeric characters in the video frame image; A score determination unit, configured to assign scores to the numeric characters corresponding to the adjacent candidate text regions according to the position information; The match score determination unit is used to use the two scores as the match score when the ordinates in the position information corresponding to the two scores are consistent.

11. The device according to claim 7, characterized in that The device also includes: A training image acquisition module is used to acquire multiple training digital character images; A digital character labeling module, used for labeling the training digital character images with digital characters; A training and learning module, used for performing training and learning of digital character recognition based on a preset deep learning model of the plurality of training digital character image pairs, and obtaining a predicted probability value of a specified digital character being included in the plurality of training digital character images; An error value calculation module, used for calculating the error values ​​between the predicted digital characters and the marked digital characters of the plurality of training digital character images based on a preset loss function and the predicted probability value; A judging module, used to judge whether the error value meets a preset condition; A model parameter adjustment module, used to adjust the model parameters of the preset deep learning model based on the gradient descent method when the error value does not meet the preset conditions, and repeat the above training and learning steps; The digital character recognition model determination module is used to use the current preset deep learning model as the digital character recognition model when the error value meets the preset conditions.

12. The device according to claim 11, characterized in that The preset deep learning model includes: Three convolutional layers, two fully connected layers, and a classification layer; Among them, each convolution layer is sequentially followed by an activation layer and a pooling layer.

13. A device for identifying sports scores in a video, characterized in that: The device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method for identifying event scores in a video according to any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method for identifying event scores in a video as described in any one of claims 1 to 6.

Images