Live broadcast data processing method and device and related equipment

Abstract

The embodiment of the invention discloses a live broadcast data processing method and device and related equipment, and can be applied to the technical field of data processing. The method comprises the following steps: displaying a video live broadcast interface corresponding to a competitive live broadcast video; if the service mode of the video live broadcast interface is a content protection mode, obtaining first alignment information for data alignment based on the content protection mode; when second score data is obtained, performing data alignment on the second score data and the first score data based on the target version information and the first picture timestamp to obtain the second score data after data alignment; and when the video player plays the first picture frame, displaying the first score card data on the first picture frame in the internal area of the player, and displaying the second score card data after data alignment in the external area of the player. By adopting the embodiment of the invention, the accuracy of synchronous display of score card data inside and outside the video player is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a live broadcast data processing method, device and related equipment. Background Art

[0002] In a live broadcast scenario of a competitive game, for example, for a live broadcast client (e.g., a sports APP) corresponding to any viewer currently watching the live competitive video, the live broadcast client can display game data, such as scores, in the picture displayed in the internal area of ​​the player and in the external area of ​​the player.

[0003] However, the inventors have found in practice that, generally speaking, the live platform data obtained by the live client is faster than the picture progress inside the player. Therefore, the data displayed in the external area of ​​the player is earlier than the data displayed in the internal area of ​​the player. As a result, when watching the live video, the user can know the game situation in advance through the score information displayed in the external area of ​​the player, so that the game information is spoiled in advance. Therefore, how to improve the accuracy of the synchronous display of game data inside and outside the video player is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present application provide a live data processing method, apparatus and related equipment, which can align the scoreboard data to be displayed in the external area of ​​the player with the corresponding picture timestamp through the alignment information sent down, thereby helping to improve the accuracy of synchronous display of the scoreboard data inside and outside the video player.

[0005] On the one hand, an embodiment of the present application provides a live broadcast data processing method, the method comprising:

[0006] Display the video live broadcast interface corresponding to the competitive live broadcast video; the video live broadcast interface includes a player internal area and a player external area; the player internal area is used to play the competitive live broadcast video through the video player;

[0007] If the business mode of the video live broadcast interface is the content protection mode, first alignment information for data alignment is obtained based on the content protection mode; the first alignment information includes target version information and a first screen timestamp corresponding to the target version information; the target version information is determined based on second scoreboard data corresponding to a target competitive event to be displayed in an external area of ​​the player; the first screen timestamp refers to a timestamp carried by a first screen frame associated with the target competitive event; the first screen frame is used to identify the first scoreboard data to be displayed in an internal area of ​​the player in the competitive live broadcast video;

[0008] When the second scoreboard data is obtained, the second scoreboard data is data-aligned with the first scoreboard data based on the target version information and the first screen timestamp to obtain the second scoreboard data after data alignment; the timestamp corresponding to the second scoreboard data after data alignment and the timestamp corresponding to the first scoreboard data are both the first screen timestamp;

[0009] When the video player plays to the first picture frame, the first scoreboard data is displayed on the first picture frame in the inner area of ​​the player, and the second scoreboard data after data alignment is displayed in the outer area of ​​the player.

[0010] Among them, the video live broadcast interface corresponding to the competitive live broadcast video includes:

[0011] In response to a video triggering operation for a competitive live video in a live recommendation page, a live stream pull request is sent to a server; the live stream pull request is used to instruct the server to obtain a live video stream corresponding to the competitive live video;

[0012] The live video stream returned by the server is received, and when the video player is called to parse the live video stream to obtain the competitive live video, the video live interface corresponding to the competitive live video is displayed.

[0013] The live video interface includes a mode switching control for mode switching; the service mode corresponding to the mode switching control is a non-content protection mode;

[0014] Before acquiring first alignment information for data alignment based on the content protection mode, the method further includes:

[0015] In response to a first trigger operation on the mode switching control, the business mode of the live video broadcast interface is switched from a non-content protection mode to a content protection mode.

[0016] When the second scoreboard data is obtained, the second scoreboard data is aligned with the first scoreboard data based on the target version information and the first screen timestamp to obtain the aligned second scoreboard data, including:

[0017] Receive target business data associated with the target competitive event sent by the server, and add the target business data to the data processing queue; the target business data includes target version information and second scoreboard data;

[0018] Searching for business data matching the target business data in the data processing queue, and if business data matching the target business data is found, obtaining second scoreboard data from the target business data stored in the data processing queue;

[0019] Obtain a competitive event flow list corresponding to the target version information, and obtain an event occurrence timestamp of the target competitive event from the competitive event flow list;

[0020] Based on the event occurrence timestamp and the first screen timestamp, the second score board data is aligned with the first score board data on the business timeline corresponding to the live competitive video to obtain the aligned second score board data; the timestamp of the aligned second score board data on the business timeline and the timestamp of the first score board data on the business timeline are both the first screen timestamp.

[0021] Wherein, based on the event occurrence timestamp and the first screen timestamp, the second scoreboard data is aligned with the first scoreboard data on the business timeline corresponding to the live competitive video to obtain the aligned second scoreboard data, including:

[0022] When the event occurrence timestamp is earlier than the first picture timestamp, determining a time offset value between the event occurrence timestamp and the first picture timestamp;

[0023] Based on the time offset value on the business timeline, the timestamp corresponding to the second scoreboard data is delayed from the event occurrence timestamp to the first screen timestamp corresponding to the first scoreboard data, and the second scoreboard data with the timestamp delayed to the first screen timestamp is used as the second scoreboard data after data alignment.

[0024] Among them, the data page card corresponding to the content protection mode is the first data page card;

[0025] When the video player plays to the first frame, the first scoreboard data is displayed on the first frame in the inner area of ​​the player, and the second scoreboard data after data alignment is displayed in the outer area of ​​the player, including:

[0026] When the playing progress of the video player reaches the first picture timestamp corresponding to the first picture frame, the first picture frame is displayed in the internal area of ​​the player, and the first scoreboard data is displayed on the first picture frame;

[0027] When the data page card displayed in the external area of ​​the player is the first data page card, the second scoreboard data after the data whose timestamp is aligned with the timestamp of the first picture is displayed on the first data page card.

[0028] The target competitive event is the competitive event recorded in the competitive event flow list;

[0029] The method also includes:

[0030] In response to a second trigger operation on the mode switching control, switching the service mode of the live video broadcast interface from the content protection mode to the non-content protection mode;

[0031] When the third score board data associated with the competitive update event is obtained based on the non-content protection mode, the third score board data is displayed on the second data page card corresponding to the external area of ​​the player; the competitive update event is the next competitive event of the target competitive event recorded in the competitive event flow list, the third score board data is sent by the server through the data link, and the second data page card is the data page card corresponding to the non-content protection mode.

[0032] The picture frame associated with the competition update event is the second picture frame, and the second picture frame is used to indicate that the server has identified and obtained the fourth scoreboard data to be displayed in the internal area of ​​the player in the competition live video, and the second picture frame is a picture frame located after the first picture frame in the competition live video;

[0033] The method also includes:

[0034] When the playing progress of the video player reaches the second picture timestamp corresponding to the second picture frame, the second picture frame is displayed in the internal area of ​​the player, and the fourth scoreboard data is displayed on the second picture frame.

[0035] The method further includes:

[0036] If the first screen timestamp corresponding to the first screen frame belongs to the target hot spot moment, when the video player plays the first screen frame, the target barrage data bound to the target hot spot moment is displayed on the first screen frame through the barrage display mode corresponding to the video player.

[0037] The method further includes:

[0038] If the target competitive event is a competitive hot spot event, in response to a viewing trigger operation on a hot spot segment set in an area outside the player, hot event segments associated with the competitive hot spot event and segment evaluation values ​​corresponding to the hot spot event segments are displayed in the hot spot segment set; the hot event segment is a video segment cut from a competitive live video that matches the competitive hot spot event; the segment start timestamp corresponding to the hot event segment is the first screen timestamp in the first alignment information.

[0039] The method further includes:

[0040] When playing the live sports video through a video player, in response to a segment playing operation for a hot event segment, in the internal area of ​​the player, the playing content of the video player is switched from the live sports video to the hot event segment; or,

[0041] When the live sports video is played through the video player, in response to the segment play operation for the hot event segment, the hot video segment is played through the video player in the internal area of ​​the player.

[0042] On the one hand, an embodiment of the present application provides a live broadcast data processing method, the method comprising:

[0043] Acquire a live sports video, perform picture recognition on a first picture frame associated with a target sports event in the live sports video, obtain first scoreboard data to be displayed in an internal area of ​​a player, and use a timestamp corresponding to the first picture frame as a first picture timestamp corresponding to the first scoreboard data;

[0044] Obtain a competitive event flow list associated with the target competitive event, and parse the competitive event flow list to obtain an event occurrence timestamp and an event link timing timestamp corresponding to the target competitive event;

[0045] If a first screen timestamp matching the event link timing timestamp is found within the adjacent time range of the event occurrence timestamp, the scoreboard data corresponding to the event occurrence timestamp is used as the second scoreboard data to be displayed in the external area of ​​the player, and the target version information for the target competitive event is generated based on the second scoreboard data; the external area of ​​the player and the internal area of ​​the player are both areas in the video live broadcast interface corresponding to the competitive live broadcast video;

[0046] First alignment information for aligning the second score board data with the first score board data is determined through the target version information and the first picture timestamp, and the first alignment information is sent to the live broadcast client. The first alignment information is used to instruct the live broadcast client to align the second score board data with the first score board data based on the target version information and the first picture timestamp when the business mode is the content protection mode, so as to obtain the second score board data after the data alignment; the timestamp corresponding to the second score board data after the data alignment and the timestamp corresponding to the first score board data are both the first picture timestamp, and the first picture timestamp is used to instruct the live broadcast client to display the first score board data on the first picture frame in the internal area of ​​the player and display the second score board data after the data alignment in the external area of ​​the player when the first picture frame is played through the video player.

[0047] The method includes obtaining a live competitive video, performing picture recognition on a first picture frame associated with a target competitive event in the live competitive video, obtaining first scoreboard data to be displayed in an internal area of ​​a player, and using a timestamp corresponding to the first picture frame as a first picture timestamp corresponding to the first scoreboard data, including:

[0048] Receiving a video acquisition data stream sent by a picture acquisition terminal, decoding the video acquisition data stream, and obtaining competitive live broadcast data associated with a target competitive event; the video acquisition data stream is obtained by the picture acquisition terminal performing picture acquisition on the target competitive event;

[0049] Performing video encoding on the live broadcast data of the competition to obtain a live broadcast video of the competition carrying multiple picture frames, performing video encapsulation processing on the live broadcast video of the competition to obtain a monitoring video stream for the live broadcast video of the competition; one picture frame corresponds to one encoding timestamp; one encoding timestamp refers to a timestamp written when one picture frame is obtained by encoding;

[0050] Performing video unpacking processing on the monitoring video stream to obtain a live competitive video, and performing scoreboard recognition on each of a plurality of picture frames in the live competitive video through a first recognition service thread to obtain a scoreboard recognition result of each picture frame;

[0051] If the scoreboard recognition result of each picture frame contains identified scoreboard data associated with the target competitive event, the picture frame corresponding to the identified scoreboard data is used as the first picture frame, and when the scoreboard data identified in the first picture frame is used as the first scoreboard data to be displayed in the internal area of ​​the player, the encoded timestamp carried by the first picture frame is used as the first picture timestamp corresponding to the first scoreboard data.

[0052] The step of using the encoding timestamp carried by the first picture frame as the first picture timestamp corresponding to the first scoreboard data includes:

[0053] Establishing a first mapping relationship between first scoreboard data and first picture timestamp;

[0054] Based on the first mapping relationship, the first scoreboard data is used as the value in the key-value pair, and the first screen timestamp is used as the key in the key-value pair. Based on the key-value pair, the timestamp corresponding to the first scoreboard data is determined as the first screen timestamp.

[0055] The method further includes:

[0056] When the competitive live video is encapsulated and processed to obtain a monitoring video stream for the competitive live video, the live video stream for the competitive live video is encapsulated according to the data transmission protocol, and the live video stream is sent to the live client according to the data transmission protocol, so that when the live client parses the competitive live video from the live video stream based on the data transmission protocol to obtain the competitive live video, the competitive live video is played through a video player.

[0057] Among them, obtaining a competitive event flow list associated with the target competitive event, parsing the competitive event flow list to obtain the event occurrence timestamp and event link timing timestamp corresponding to the target competitive event, including:

[0058] Pulling a competitive event flow list from a data source interface provided by a data acquisition device; the competitive event flow list is determined by the event flow record data recorded by the data acquisition device for a target competitive event; the event flow record data includes an event occurrence timestamp of the target competitive event recorded by the data acquisition device and a competitive link to which the target competitive event belongs, as well as an event link timing timestamp corresponding to the competitive link;

[0059] Obtain the event flow record data in the competition event flow list, perform data conversion on the event flow record data to obtain the change flow structure data for the target competition event, and obtain the event occurrence timestamp corresponding to the target competition event and the event link timing timestamp corresponding to the target competition event during the competition link from the change flow structure data.

[0060] The plurality of frames included in the live competitive video also include a second frame other than the first frame, the second frame is a frame where a competitive update event occurs, the competitive update event is a next competitive event of the target competitive event, and the second frame is a frame after the first frame; the scoreboard recognition result of each frame includes a scoreboard recognition result corresponding to the second frame;

[0061] The method also includes:

[0062] If the score board recognition result corresponding to the second picture frame indicates that the score board data associated with the competitive update event is not recognized in the second picture frame, then based on the time difference between the first picture timestamp corresponding to the first picture frame and the event occurrence timestamp corresponding to the target competitive event, the picture timestamp corresponding to the second picture frame is time predicted to obtain a predicted timestamp for the second picture frame, and second alignment information for sending to the live broadcast client is constructed through the predicted timestamp and the updated version information corresponding to the competitive update event.

[0063] The method further includes:

[0064] When the predicted timestamp for the second picture frame is predicted, the scoreboard recognition is performed on the second picture frame by the second recognition service thread to obtain an auxiliary recognition result of the second picture frame; the second recognition service thread is a thread used to assist the first recognition service thread in performing the scoreboard recognition;

[0065] If the auxiliary recognition result indicates that the scoreboard data associated with the competition update event is recognized, the recognized scoreboard data is used as the fourth scoreboard data, and the encoding timestamp corresponding to the second picture frame is used as the second picture timestamp corresponding to the fourth scoreboard data;

[0066] The second picture timestamp is sent to the live broadcast client, so that the live broadcast client updates and replaces the picture frame timestamp corresponding to the second picture frame from the predicted timestamp to the second picture timestamp before playing the second picture frame through the video player;

[0067] The prediction timestamp is later than the second picture timestamp.

[0068] On the one hand, an embodiment of the present application provides a live broadcast data processing device, the device comprising:

[0069] The live broadcast interface display module is used to display the video live broadcast interface corresponding to the competitive live broadcast video; the video live broadcast interface includes a player internal area and a player external area; the player internal area is used to play the competitive live broadcast video through the video player;

[0070] An alignment information acquisition module is used to acquire first alignment information for data alignment based on the content protection mode if the business mode of the video live broadcast interface is the content protection mode; the first alignment information includes target version information and a first screen timestamp corresponding to the target version information; the target version information is determined based on second scoreboard data corresponding to a target competitive event to be displayed in an external area of ​​the player; the first screen timestamp refers to a timestamp carried by a first screen frame associated with the target competitive event; the first screen frame is used to identify the first scoreboard data to be displayed in an internal area of ​​the player in the competitive live broadcast video;

[0071] A data alignment module is used to align the second scoreboard data with the first scoreboard data based on the target version information and the first screen timestamp when the second scoreboard data is acquired, so as to obtain the second scoreboard data after data alignment; the timestamp corresponding to the second scoreboard data after data alignment and the timestamp corresponding to the first scoreboard data are both the first screen timestamp;

[0072] The scoreboard display module is used to display first scoreboard data on the first frame in the internal area of ​​the player when the video player plays the first frame, and to display second scoreboard data after data alignment in the external area of ​​the player.

[0073] On the one hand, an embodiment of the present application provides a live broadcast data processing device, the device comprising:

[0074] A picture recognition module is used to obtain a live sports video, perform picture recognition on a first picture frame associated with a target sports event in the live sports video, obtain first scoreboard data to be displayed in an internal area of ​​a player, and use a timestamp corresponding to the first picture frame as a first picture timestamp corresponding to the first scoreboard data;

[0075] A flow list acquisition module is used to acquire a competitive event flow list associated with a target competitive event, and parse the competitive event flow list to obtain an event occurrence timestamp and an event link timing timestamp corresponding to the target competitive event;

[0076] A version information generation module, for, if a first screen timestamp matching the event link timing timestamp is found within a time range adjacent to the event occurrence timestamp, using the scoreboard data corresponding to the event occurrence timestamp as second scoreboard data to be displayed in the player external area, and generating target version information for the target competitive event based on the second scoreboard data; both the player external area and the player internal area are areas in the video live broadcast interface corresponding to the competitive live broadcast video;

[0077] The alignment information sending module is used to determine the first alignment information used for aligning the second score board data with the first score board data through the target version information and the first picture timestamp, and send the first alignment information to the live broadcast client. The first alignment information is used to instruct the live broadcast client to align the second score board data with the first score board data based on the target version information and the first picture timestamp when the business mode is the content protection mode, so as to obtain the second score board data after the data alignment; the timestamp corresponding to the second score board data after the data alignment and the timestamp corresponding to the first score board data are both the first picture timestamp, and the first picture timestamp is used to instruct the live broadcast client to display the first score board data on the first picture frame in the internal area of ​​the player when playing to the first picture frame through the video player, and to display the second score board data after the data alignment in the external area of ​​the player.

[0078] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for being loaded and executed by a processor so that a computer device having the processor executes the method provided by the embodiment of the present application.

[0079] In one aspect, the present application provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in the present application.

[0080] By adopting the embodiment of the present application, in the video live broadcast interface associated with the competitive live broadcast video, when the business mode of the video live broadcast interface in the live broadcast client is the content protection mode, the alignment information (such as the first alignment information) issued by the server for data alignment can be obtained. Among them, the first alignment information here may include the target version information, the picture timestamp of the first picture frame required to be aligned with the target version data information (i.e., the first picture timestamp), and the first scoreboard data identified in the first picture frame is matched with the scoreboard data of the competitive event corresponding to the target version information. In this way, when the live broadcast client obtains the second scoreboard data corresponding to the target version information to be displayed in the external area of ​​the player, the second scoreboard data can be aligned with the first scoreboard data based on the target version information and the picture timestamp of the target picture frame. In this way, when the live broadcast client plays the first picture frame through the video player, the second scoreboard data after data alignment is displayed in the external area of ​​the player, thereby effectively improving the accuracy of synchronous display of the scoreboard data inside and outside the video player. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0082] Figure 1 It is a structural diagram of a data processing system provided in an embodiment of the present application;

[0083] Figure 2 It is a schematic diagram of the effect of a live broadcast data processing method provided by an embodiment of the present application;

[0084] Figure 3 It is a flowchart of a live broadcast data processing method provided by an embodiment of the present application;

[0085] Figure 4 It is a schematic diagram of a client data processing flow provided by an embodiment of the present application;

[0086] Figure 5 It is a flowchart of a live broadcast data processing method provided by an embodiment of the present application;

[0087] Figure 6 This is a schematic diagram of the effect of a video live broadcast interface provided in an embodiment of the present application;

[0088] Figure 7 This is a schematic diagram of the effect of a hot moment screen provided by an embodiment of the present application;

[0089] Figure 8 This is a schematic diagram of the effect of displaying a hotspot segment set provided by an embodiment of the present application;

[0090] Fig. 9 It is a flowchart of a live broadcast data processing method provided by an embodiment of the present application;

[0091] Fig.10 It is a flowchart of a picture recognition process provided by an embodiment of the present application;

[0092] Fig.11 It is a schematic diagram of a processing process of live video data provided by an embodiment of the present application;

[0093] Fig.12 This is a schematic diagram of the effect of data processing time provided by an embodiment of the present application;

[0094] Fig.13 It is a timing diagram of live data processing provided by the embodiment of the present application;

[0095] Fig.14 It is a structural diagram of a live broadcast data processing device provided in an embodiment of the present application;

[0096] Fig.15 It is a structural diagram of a live broadcast data processing device provided in an embodiment of the present application;

[0097] Fig.16 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0098] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0099] See also Figure 1 , Figure 1 Schematic diagram of a data processing system provided in an embodiment of the present application. Figure 1 As shown, the data processing system may include terminal devices (such as device 11a, device 12a, device 13a) and servers (such as server 110a, server 111a, server 112a, server 113a). It can be understood that Figure 1The number of terminal devices and servers in the embodiment is only for illustration, and any number of terminal devices and servers may be provided according to the implementation requirements. The terminal devices (such as the device 11a, the device 12a, and the device 13a) may communicate with the server through a network (i.e., a medium providing a communication link through a wired or wireless communication link or an optical fiber cable, etc.) to transmit data.

[0100] It is understandable that a client may be running on a terminal device (such as device 12a), and the client may be a program that provides local services to a user (also referred to as a business object, an operation object). Server 110a may be a server corresponding to a client, and a program for providing resources, service data and other services to the client may be running in the server 110a. It is understandable that a client running on a terminal device may also be referred to as an application client, a business client, and the like. For example, a client running on the terminal device may be a client (also referred to as a live client) for live video broadcasting (such as live broadcasting of a competitive match, and the like), and the live client may pull a live video stream of a live video from a server (such as server 110a), so that the live client may parse the live video stream, obtain a competitive live video, and play the competitive live video through a video player in the live client.

[0101] It is understandable that the terminal device (such as device 11a) may include but is not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, aircraft, smart speakers, smart home appliances, etc., and there is no limitation here. The server 110a may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms, and there is no limitation here.

[0102] For further information, see Figure 2 , Figure 2 Schematic diagram of the effect of a live data processing method provided by an embodiment of the present application. Figure 2 As shown, the server (such as Figure 2The server 110a in the example can obtain a competitive live video 201a, which can be a video of a live competitive game. The competitive live video 201a can include a first picture frame associated with a target competitive event (which can be a current competitive event, such as a scoring event), and then, the first picture frame can be subjected to picture recognition to obtain scoreboard data in the first picture frame, i.e., first scoreboard data 202a. Among them, the scoreboard data (such as the first scoreboard data 202a here) can be used to record a variety of information related to the competitive game corresponding to the competitive live video. Different types of competitive games can have different information in the scoreboard data (i.e., competitive data). For example, in a basketball competitive game, the scoreboard data can include the team name, current score, link (such as the first quarter, the second quarter, the third quarter, the fourth quarter) of the team associated with the competitive game, the timing information of each link (i.e., the event link timing timestamp), the number of fouls of each team, and other information, which are not limited here. Moreover, each picture frame in the competitive live video can be associated with a corresponding picture timestamp. For example, the picture timestamp of the first picture frame is the first picture timestamp 203a. The picture timestamp is used to uniquely identify the time of a picture frame. The picture timestamp can be obtained when encoding the competitive live video.

[0103] It should be understood that the server 110a can also obtain a running list of competitive events (such as Figure 2 As shown in 204a in ). Among them, the competitive event flow list can be a list of competitive events for recording the competitive game corresponding to the competitive live video. The competitive event list can record the information of each competitive event in the competitive game in chronological order, that is, PBP (Play by Play). The information of each competitive event may include a variety of information, such as the time when the competitive event occurred (i.e., the event occurrence timestamp), the scoreboard data corresponding to the competitive event, the event description information of the competitive event, etc., which is not limited here. The event occurrence timestamp here may refer to the time when the competitive event occurred. The event description information may be information used to describe the content of the competitive event, such as which team scored less, which team committed a foul, which team requested a timeout, and other information. It should be understood that the competitive event flow list 204a may include the event occurrence timestamp of the target competitive event and second scoreboard data. The event occurrence timestamp may refer to the time when the target competitive event occurs as recorded in the competitive event flow list. The second scoreboard data may be scoreboard data updated when the target competitive event occurs. For example, when the target competitive event is a scoring event, the second scoreboard data may be scoreboard data corresponding to the updated score caused by the scoring event.

[0104] Furthermore, the server 110a may generate target version information 205a based on the second scoreboard data of the target competitive event, and the target version information may identify the time at which a competitive match is being played. For example, the target version number, i.e., the target version information, may be determined based on the current link in the second scoreboard data, the event link timing timestamp of the current link, the event sequence of the target competitive event, and other information.

[0105] It should be understood that the server 110a may generate first alignment information based on the target version information and the first picture timestamp, and send the first alignment information to the terminal device 11a (eg, Figure 2 As shown in S21 in FIG. 1 , for example, the first alignment information may be as follows: Figure 2 As shown in 206a in , which includes the target version information and the first picture timestamp. The terminal device 11a may run a live broadcast client, and a video live broadcast interface for the above-mentioned competitive live video is displayed in the live broadcast client, and the video live broadcast interface can play the competitive live video. It should be understood that the server 110a can also continuously send a live video stream to the terminal device 11a, and then after receiving the live video stream, the terminal device 11a parses the received live video stream through the live broadcast client to obtain the competitive live video, and plays the competitive live video through the video player in the video live broadcast interface.

[0106] Further, after acquiring the first alignment information 206a, the terminal device 11a may align the second scoreboard data with the first scoreboard data based on the target version information and the first screen timestamp to obtain the aligned second scoreboard data (eg Figure 2 The data alignment may refer to aligning the timestamp corresponding to the second scoreboard data with the timestamp of the first picture, so that when the video player plays the first picture frame, the second scoreboard data after data alignment is displayed in the external area of ​​the player (such as Figure 2 As shown in S22 in the figure), instead of the live broadcast client directly displaying the second scoreboard data immediately after obtaining the second scoreboard data, it is avoided that the scoreboard data of the picture frame that the video player has not played is displayed in advance in the external area of ​​the player, causing spoilers.

[0107] It should be understood that the terminal device 11a can display the picture frame and the second scoreboard data through the page 200a. Wherein, the page 200a can be a live video page, and the live video page can include a player internal area 208a (also referred to as the internal area) and a player external area 209a (also referred to as the external area). The player internal area 208a can be used to play the video screen by the video player, that is, to display the picture frame of the competitive live video, and the player external area 209a can be a region different from the player internal area. It is understood that a data page card for displaying the second scoreboard data can be included in the player external area. It is understood that, for the convenience of description, the scoreboard data included in the picture frame can be referred to as picture scoreboard data (i.e., scoreboard data identified from the picture frame), such as the above-mentioned first scoreboard data can be picture scoreboard data, and the scoreboard data used to be displayed in the player external area is referred to as running scoreboard data, such as the above-mentioned second scoreboard data can be running scoreboard data.

[0108] It can be seen that in the first picture frame displayed in the player internal area 208a on the page 200a, the first scoreboard data can be as follows: Figure 2 As shown in 210a in FIG. 1 , the second scoreboard data of the second picture frame displayed in the player outer area 209a can be as follows: Figure 2 The first scoreboard data 210a and the second scoreboard data 220a can be displayed synchronously. Thus, when the first picture frame is displayed, the corresponding second scoreboard data can be displayed in the outer area of ​​the player to avoid spoilers of the running scoreboard data displayed in the outer area in advance.

[0109] It should be noted that before collecting relevant data of users and during the process of collecting relevant data of users, this application can display a prompt interface, pop-up window or output voice prompt information, and the prompt interface, pop-up window or voice prompt information is used to prompt the user that its relevant data is currently being collected, so that this application only starts to execute the relevant steps of obtaining relevant data of users after obtaining the confirmation operation issued by the user to the prompt interface or pop-up window, otherwise (that is, when the confirmation operation issued by the user to the prompt interface or pop-up window is not obtained), the relevant steps of obtaining relevant data of users are terminated, that is, the relevant data of users are not obtained. In other words, all user data collected by this application are collected with the consent and authorization of the user, and the collection, use and processing of relevant user data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0110] It is understood that the above scenarios are only examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of the present application. The technical solutions of the present application can also be applied to other scenarios. For example, it is known to those skilled in the art that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0111] For further information, see Figure 3 , Figure 3 1 is a flowchart of a live broadcast data processing method provided in an embodiment of the present application. The method can be executed by a terminal device or a server. The embodiment of the present application is described by taking the execution of a terminal device as an example. A live broadcast client for playing live competitive video can be run in the terminal device. The method can at least include the following steps S101-S104.

[0112] S101, displaying a video live broadcast interface corresponding to the competitive live broadcast video; the video live broadcast interface includes a player internal area and a player external area; the player internal area is used to play the competitive live broadcast video through a video player.

[0113] The live competitive video may be a live competitive video of a competitive game, for example, a basketball game, a football game, etc., which are not limited here.

[0114] The video live broadcast interface may be an interface showing a live competitive video, and a user may watch the live competitive video through the video live broadcast interface. It should be understood that the video live broadcast interface includes a player internal area and a player external area.

[0115] The internal area of ​​the player is used to play the live broadcast video of the competition through the video player. The external area of ​​the player is an area in the live video interface that is different from the internal area of ​​the player. It should be understood that the video player can be a component for playing live video. For example, the internal area of ​​the player can be the above Figure 1 As shown in the player internal area 208a in the page 200a, the player external area can be the above Figure 1 As shown in the player outer area 209a in page 200a, no further details are given here.

[0116] It should be understood that in an embodiment of the present application, the live video stream of the competitive live video can be pulled from the server, and then the competitive live video can be displayed on the live video interface based on the pulled live video stream, that is, the live video interface corresponding to the competitive live video can be displayed.

[0117] Specifically, displaying the video live broadcast interface corresponding to the competitive live video may include the following steps: in response to a video trigger operation for the competitive live video in the live broadcast recommendation page, sending a live stream pull request to the server; the live stream pull request is used to instruct the server to obtain the live video stream corresponding to the competitive live video; further, receiving the live video stream returned by the server, and when calling the video player to parse the live video stream to obtain the competitive live video, displaying the video live broadcast interface corresponding to the competitive live video.

[0118] The live broadcast recommendation page may be a page for displaying recommended live broadcast content. It is understandable that the live broadcast recommendation page may include at least one recommended live broadcast content, including the live broadcast content corresponding to the competitive live broadcast video.

[0119] The video trigger operation may be an operation for triggering the display of a competitive live video. For example, the video trigger operation may be a touch operation (such as single click, double click, etc.) on the live content corresponding to the competitive live video displayed on the live recommendation page.

[0120] The live streaming pull request may be a request for obtaining a live video stream of a competitive live video from a server, that is, the live streaming pull request is used to instruct the server to obtain a live video stream corresponding to the competitive live video. A live video stream may refer to a data stream that transmits compressed video data through a network (such as the Internet, a local area network, a satellite network, or a cable TV network), which can be processed as a stable and continuous stream through the network. It should be understood that when the server obtains the live streaming pull request, the server can determine the live video stream corresponding to the competitive live video based on the live streaming pull request, and send the live video stream to the live client stably and continuously.

[0121] It should be understood that the live broadcast client can receive the live video stream returned by the server, and then call the video player to parse the live video stream to obtain the competitive live video, and then display the live video interface corresponding to the competitive live video, that is, play the competitive live video through the video player to display the live video interface corresponding to the competitive live video.

[0122] S102. If the business mode of the live video interface is a content protection mode, first alignment information for data alignment is obtained based on the content protection mode; the first alignment information includes target version information and a first picture timestamp corresponding to the target version information; the target version information is determined based on second scoreboard data corresponding to a target competitive event to be displayed in an area outside the player; the first picture timestamp refers to a timestamp carried by a first picture frame associated with the target competitive event; the first picture frame is used to identify first scoreboard data to be displayed in an internal area of ​​the player in the live competitive video.

[0123] It should be understood that the business mode may refer to a mode of whether content protection is performed in the video live interface of the competitive live video. Wherein, whether content protection is performed here may refer to the competitive data (i.e., scoreboard data) displayed in the internal area of ​​the player of the video live interface is not displayed in advance by the competitive data displayed in the external area of ​​the player (i.e., spoiler). Based on this, the business mode may be a content protection mode or a non-content protection mode. Wherein, the content protection mode may refer to a mode of performing content protection in the video live interface, that is, a mode in which the competitive data (i.e., scoreboard data) in the internal area of ​​the player in the video live interface is not displayed in advance by the competitive data (i.e., scoreboard data) in the external area of ​​the player. On the contrary, the non-content protection mode may refer to a mode in which content protection is not performed in the video live interface, that is, a mode in which the competitive data (i.e., scoreboard data) displayed in the internal area of ​​the player is allowed to be displayed in advance by the competitive data displayed in the external area of ​​the player in the video live interface. It should be understood that the business mode may be customized by the business object corresponding to the live client according to its own needs.

[0124] Among them, the scoreboard data can be used to record various information related to the competitive game corresponding to the competitive live video. Different types of competitive games can have different information in the scoreboard data (i.e., competitive data). For example, in a basketball competitive game, the scoreboard data can include the team name, current score, link (such as the first quarter, second quarter, third quarter, fourth quarter) of the team associated with the competitive game, timing information of each link (i.e., event link timing timestamp), number of fouls of each team, etc., which are not limited here.

[0125] It should be understood that the first alignment information may refer to data used to align the competitive data displayed in the external area of ​​the player with the competitive data (such as scoreboard data) displayed in the internal area of ​​the player. The first alignment information may include the target version information and the first screen timestamp corresponding to the target version information, and then the target version information may be aligned with the screen of the competitive live video at the first screen timestamp through the first screen timestamp, that is, the second scoreboard data to be displayed in the external area of ​​the video player is aligned with the first scoreboard data in the screen frame, so that when the competitive live video displays the screen corresponding to the first screen timestamp, the scoreboard data corresponding to the target version information is displayed in the external area of ​​the player.

[0126] Among them, the target version information is the version information determined by the second scoreboard data corresponding to the target competitive event to be displayed in the external area of ​​the player, that is, the target version information can be the version information of the target competitive event. It should be understood that a version information can refer to the version information corresponding to a competitive event. Among them, the target competitive event can refer to any competitive event to be processed. It is understandable that in competitive games, some competitive events can be generated to increase the scores of the teams associated with the competitive games. Different types of competitive games involve different competitive events, such as, in basketball competitive games, such as scoring, fouls, substitutions, suspensions, etc., which are not limited here. Then, the version information can be the information of the version determined by the scoreboard data corresponding to the competitive event currently occurring, and can be specifically determined based on the event link (also known as the competitive link), link timing timestamp, score and other information in the scoreboard data.

[0127] For example, the version information corresponding to a competitive event can be shown in the following table (1).

[0128] Table (1)

[0129]

[0130] Among them, the event link may refer to the event link corresponding to a version, the link timing minute may refer to the minute information in the link timing timestamp of a version in the event link, the link timing second may refer to the second information in the link timing timestamp of a version in the event link, and the link timing millisecond may refer to the millisecond information in the link timing timestamp of a version in the event link. Fallback may refer to whether there is a fallback behavior during the game. The event sequence may refer to the order of occurrence of competitive events (that is, it may be the order in the competitive event flow list used to record competitive events). The total score may refer to the score corresponding to the competitive event at the time of this version. The total may refer to the total length of a version information. It should be understood that the version information shown in Table (1) is only an exemplary description, and the version information can also be determined in other formats, which is not limited here.

[0131] It should be understood that the second scoreboard data is scoreboard data for display in the external area of ​​the player determined based on the target competitive event. For example, the target competitive event is a scoring competitive event. Before the target competitive event occurs, the score of the competitive game is team D1: team D2 = 50:60. After team D1 scores (i.e., the target competitive event occurs), the score in the scoreboard data of the competitive game is updated to team D1: team D2 = 52:60. Then, the score in the second scoreboard data can be 52:60. It should be understood that the target version information can be determined by the server based on the second scoreboard data. The specific determination process can be referred to the above-mentioned related description, which will not be repeated here.

[0132] It should be understood that the first picture timestamp refers to the timestamp carried by the first picture frame associated with the target competitive event. Among them, the timestamp is a complete and verifiable data that can indicate that a piece of data already existed before a certain time, and uniquely identifies the time of a certain moment. Then, the picture timestamp refers to the timestamp corresponding to the picture frame in the live video. The picture timestamp is written to the picture frame by the encoder during video encoding, and is used to guide the behavior of the playback end (i.e., the live client here) (i.e., the picture frames can be played in sequence according to the picture timestamp). It can be understood that the picture timestamp of a picture frame (such as the first picture timestamp of the first picture frame) can be associated with the picture frame as supplementary enhancement information (i.e., SEI, Supplemental Enhancement Information) of the picture frame. Supplementary enhancement information can provide additional information and functions for the picture frame in the video stream, such as the color space, scene information, specific encoding parameters, time information (i.e., picture timestamp), etc. of the video, thereby enhancing the quality and function of the video and improving the compression efficiency and reliability of the video. Therefore, in an embodiment of the present application, when encoding a video, the picture timestamp of the picture frame can be determined, and the picture timestamp can be used as supplementary enhancement information of the picture frame. Then, when determining the alignment information, the picture timestamp of the picture frame (such as the first picture timestamp of the first picture frame) can be determined from the supplementary enhancement information of the picture frame to facilitate subsequent data alignment.

[0133] It should be understood that the first screen timestamp may refer to the timestamp corresponding to the first screen frame, and the first screen frame may be a screen frame associated with the target competitive event, that is, the first screen frame may be a screen frame when the scoreboard data in the screen is the scoreboard data updated based on the target competitive event. It should be understood that the first screen frame is used to identify the first scoreboard data to be displayed in the internal area of ​​the player in the competitive live video, and the first scoreboard data may be the scoreboard data in the internal area of ​​the player identified in the first screen frame, that is, the first scoreboard data may be the scoreboard data displayed in the internal area of ​​the player updated based on the target competitive event.

[0134] It should be understood that the first scoreboard data and the second scoreboard data can be understood as the scoreboard data for being displayed in different regions (i.e., the internal area of ​​the player and the external area of ​​the player) associated with the same target competitive event. The first scoreboard data and the second scoreboard data can be the same, or, can also be different, and will not be repeated here. It should be understood that the first scoreboard data are different from the second scoreboard data, and can refer to the different types of information included in the first scoreboard data and the second scoreboard data, such as more information can be included in the first scoreboard data than in the second scoreboard data. For example, the first scoreboard data can include the team name, current score, the link (such as the first quarter, the second quarter, the third quarter, the fourth quarter) of the associated team, the timing information of each link, the number of fouls of each team, and the second scoreboard data only include the team name, current score, the link (such as the first quarter, the second quarter, the third quarter, the fourth quarter) of the team.

[0135] S103. When the second score board data is obtained, the second score board data is aligned with the first score board data based on the target version information and the first screen timestamp to obtain the aligned second score board data; the timestamp corresponding to the aligned second score board data and the timestamp corresponding to the first score board data are both the first screen timestamp.

[0136] It should be understood that aligning the second score board data with the first score board data may refer to determining the timestamp corresponding to the second score board data as the first picture timestamp, so that after the picture played in the internal area of ​​the player reaches the first picture timestamp, the second score board data is displayed in the external area of ​​the player.

[0137] It should be understood that the timestamp corresponding to the second scoreboard data after data alignment and the timestamp corresponding to the first scoreboard data are both first picture timestamps. It should be understood that the timestamp corresponding to the first scoreboard data may refer to the timestamp corresponding to the first picture frame where the first scoreboard data is located.

[0138] Specifically, when the live broadcast client obtains the second scoreboard data, based on the target version information and the first screen timestamp, the second scoreboard data is aligned with the first scoreboard data to obtain the aligned second scoreboard data, which may include the following steps: receiving the target business data associated with the target competitive event issued by the server, and adding the target business data to the data processing queue; the target business data includes the target version information and the second scoreboard data; further, searching the data processing queue for business data matching the target business data, and if the business data matching the target business data is found, obtaining the second scoreboard data from the target business data stored in the data processing queue; further, obtaining the competitive event flow list corresponding to the target version information, and obtaining the event occurrence timestamp of the target competitive event from the competitive event flow list; further, based on the event occurrence timestamp and the first screen timestamp, the second scoreboard data is aligned with the first scoreboard data on the business timeline corresponding to the competitive live broadcast video to obtain the aligned second scoreboard data; the timestamp of the second scoreboard data on the business timeline after the data alignment and the timestamp of the first scoreboard data on the business timeline are both the first screen timestamp.

[0139] It should be understood that the target business data can be used for data for scoreboard display in the external area of ​​the player. It should be understood that the target business data may include the above-mentioned target version information and the second scoreboard data, so that the second scoreboard data can be aligned with the first picture frame (i.e., aligned with the first scoreboard data) through the target version information, and then the aligned second scoreboard data is displayed in the external area of ​​the player. It is understandable that the target business data may only include the target version information and the second scoreboard data, or it may also include the second scoreboard data and target version information for the target competitive event included in the competitive event flow list and other information (such as the above-mentioned event description information of the competitive event), so as to determine the full data and target version information of the target competitive event as the target business data, and then send it to the live broadcast client.

[0140] Among them, the data processing queue may refer to a queue maintained by the live client for processing the scoreboard data displayed in the external area of ​​the player. It should be understood that the data processing queue may include various business data to be processed, and then after the live client receives the target business data sent by the server, the target business data can be added to the data processing queue. Further, when the target business data needs to be processed, the live client can search for business data matching the target business data in the data processing queue, thereby reading the found business data (specifically, the scoreboard data in the business data) for processing. It should be understood that if business data matching the target business data is found in the data processing queue, it means that the target business data has been added to the data processing queue, and then the data can be read from the data processing queue for data processing. On the contrary, if the business data matching the target business data is not found in the data processing queue, it means that the target business data has not been successfully added to the data processing queue, and then it can be added again.

[0141] Among them, the competitive event flow list can be a list of competitive events for recording the competitive game corresponding to the competitive live video. The competitive event list can record the information of each competitive event in the competitive game in chronological order, i.e., PBP (Play by Play). The information of each competitive event can include a variety of information, such as the time when the competitive event occurred (i.e., the event occurrence timestamp), the scoreboard data of the competitive event, the event description information of the competitive event, etc., which is not limited here. The event occurrence timestamp here can refer to the time when the competitive event occurred. The event description information can be information used to describe the content of the competitive event, such as which team scored less, which team committed a foul, which team requested a timeout, and other information.

[0142] It can be understood that the competitive event flow list can be continuously updated as the competitive events occur. Therefore, the competitive event flow list corresponding to the specified version information (such as the target version information) can be obtained based on the specified version information, and the version information can be generated based on the competitive events that occurred. Then, the competitive event flow list corresponding to a certain version information can be a competitive event flow list containing information about the corresponding competitive events. For example, the competitive event flow list corresponding to the target version information contains information about the target competitive event.

[0143] It is understandable that the competitive event flow list corresponding to the target version information may include information of the target competitive event, and further, the event occurrence timestamp of the target competitive event may be determined from the competitive event flow list corresponding to the obtained target version information.

[0144] Among them, the business timeline may refer to the timeline for business processing in the live client. It should be understood that the timestamp of the second scoreboard data on the business timeline after data alignment and the timestamp of the first scoreboard data on the business timeline are both the first screen timestamp. That is to say, the second scoreboard data is data aligned with the first scoreboard data on the business timeline corresponding to the live client, and the timestamp of the second scoreboard data can be adjusted to the same timestamp as the first screen timestamp on the business timeline corresponding to the live client. For example, the timestamp of the first scoreboard data on the business timeline is time T1, and the event occurrence timestamp of the second scoreboard data is time T2, but the timestamp of the second scoreboard data on the business timeline after data alignment will be adjusted to time T1.

[0145] Specifically, based on the event occurrence timestamp and the first screen timestamp, the second score board data is aligned with the first score board data on the business timeline corresponding to the live competitive video to obtain the aligned second score board data, which may include the following steps: when the event occurrence timestamp is earlier than the first screen timestamp, determining the time offset value between the event occurrence timestamp and the first screen timestamp; further, based on the time offset value, on the business timeline, delaying the timestamp corresponding to the second score board data from the event occurrence timestamp to the first screen timestamp corresponding to the first score board data, and using the second score board data with the timestamp delayed to the first screen timestamp as the aligned second score board data.

[0146] Among them, it should be understood that there is a certain delay from the occurrence of the competitive event to the screen display when the competitive event occurs to the video live broadcast interface. Then, the competitive event occurrence timestamp of the competitive event is earlier than the first screen timestamp, and the event occurrence timestamp and the first screen timestamp have a certain time difference, so that the time offset value between the event occurrence timestamp and the first screen timestamp can be determined, and the time offset value is also the time difference between the event occurrence timestamp and the first screen timestamp. For example, the event occurrence timestamp is V1, and the first screen timestamp is T1, then the time difference between the event occurrence timestamp and the first screen timestamp can be T1-V1.

[0147] It should be understood that if the competitive event occurrence timestamp of the competitive event is later than the first screen timestamp, it means that the event occurrence timestamp of the competitive event recorded in the competitive event flow list is wrong, and thus the erroneous competitive event may not be processed.

[0148] Furthermore, based on the time offset value on the business timeline, the timestamp corresponding to the second score board data is delayed from the event occurrence timestamp to the first screen timestamp corresponding to the first score board data. This may mean adding the time offset value to the event occurrence timestamp to obtain the first screen timestamp, thereby delaying the competitive event to be displayed at the first screen timestamp. Its essence is to adjust the timestamp corresponding to the second score board data to the first screen timestamp.

[0149] S104: When the video player plays the first picture frame, the first scoreboard data is displayed on the first picture frame in the inner area of ​​the player, and the second scoreboard data after data alignment is displayed in the outer area of ​​the player.

[0150] It should be understood that, based on the above description, the first picture frame can refer to any picture frame associated with the target competitive event. It should be understood that the first picture frame can display the first scoreboard data, that is, when the video player plays the first picture frame, the first scoreboard data can be displayed on the first picture frame in the internal area of ​​the player, that is, when the video player plays the picture frame corresponding to the first picture timestamp, the first scoreboard data can be displayed in the picture displayed in the internal area of ​​the player.

[0151] It should be understood that since the timestamp of the second scoreboard data is aligned with the timestamp of the first picture, when the first picture frame is displayed in the internal area of ​​the player, that is, when the current time reaches the timestamp of the first picture, the second scoreboard data after data alignment can be displayed in the external area of ​​the player. Based on this, the scoreboard data displayed in the external area of ​​the player will not be earlier than the scoreboard data displayed in the internal area of ​​the player, thereby preventing spoilers.

[0152] Specifically, the data page card corresponding to the content protection mode is the first data page card; then, when the video player plays to the first picture frame, the first scoreboard data is displayed on the first picture frame in the internal area of ​​the player, and the second scoreboard data after data alignment is displayed in the external area of ​​the player, which may include the following steps: when the playback progress of the video player reaches the first picture timestamp corresponding to the first picture frame, the first picture frame is displayed in the internal area of ​​the player, and the first scoreboard data is displayed on the first picture frame; further, when the data page card displayed in the external area of ​​the player is the first data page card, the second scoreboard data after data alignment with the timestamp of the first picture timestamp is displayed on the first data page card.

[0153] The first data page card may be a data page card in a content protection mode. The data page card may be a page card in an external area of ​​the player for displaying scoreboard data.

[0154] It should be understood that, based on the above description, when the video player plays the first frame, it means that the playback progress of the video player reaches the first frame timestamp corresponding to the first frame. The playback progress here can be used to indicate the time progress of the frame played by the video player.

[0155] It should be understood that when the first data page card is displayed in the external area of ​​the player, the second scoreboard data after the data with the timestamp aligned with the first screen timestamp can be displayed on the first data page card. Conversely, if the first data page card is not displayed in the external area of ​​the player, but other types of page cards (such as comment page cards) are displayed, the corresponding types of page cards can be displayed in the external area of ​​the player, which will not be described in detail here. For example, when the comment page card is displayed in the external area of ​​the player, the comments on the live broadcast video of the competition are displayed in the comment page card.

[0156] For example, see Figure 4 , Figure 4 Schematic diagram of a client data processing flow provided by an embodiment of the present application. Figure 4 As shown, the live broadcast client can obtain the flow scoreboard data (such as the second scoreboard data mentioned above) and the corresponding version information, that is, the target service data mentioned above, from the server; the live broadcast client can also obtain the picture timestamp (such as the first picture timestamp mentioned above) and the corresponding version information (such as the target version information mentioned above), that is, alignment information, such as the first alignment information mentioned above. The target service data and the first alignment information can be actively pushed by the server through the TCP protocol (such as Figure 4 Optionally, the live client can also poll the server (as shown in 401a in FIG. Figure 4 402a in the figure), so that the full data and the corresponding version information can be obtained. The full data can be the full data corresponding to each competitive event in the competitive event flow list, or the full data and the corresponding version information can also be the target business data. It should be understood that after the live broadcast client obtains the target business data, it can add the target business data to the data processing queue 403a maintained by the live broadcast client. In addition, the live broadcast client can perform data alignment based on the obtained alignment information (i.e., picture timestamp + version information) (such as Figure 4 Further, the playing progress of the live sports video in the private room in the internal area of ​​the player can be detected (such as Figure 4 405a in FIG), and then the live broadcast client can determine the aligned stream scoreboard data of the current playback progress (such as Figure 4 As shown in 406a in FIG. 4 ), the running scoreboard data may be, for example, the aligned second scoreboard data described above, so that the first data page card (such as Figure 4 407a in the figure), which may specifically be the aligned running score board data displayed in the first data page card.

[0157] For further information, see Figure 5 , Figure 5 1 is a flow chart of a live broadcast data processing method provided in an embodiment of the present application. The method can be executed by a live broadcast client, for example, the live broadcast client can be a live broadcast client running in the device 11a. The method can at least include the following steps S201-S205.

[0158] S201, displaying a video live broadcast interface corresponding to the competitive live broadcast video; the video live broadcast interface includes a player internal area and a player external area; the player internal area is used to play the competitive live broadcast video through a video player.

[0159] Among them, the introduction of step S201 can refer to the above related description, and will not be repeated here.

[0160] S202: In response to a first trigger operation on the mode switching control, the service mode of the live video broadcast interface is switched from a non-content protection mode to a content protection mode.

[0161] It should be understood that, based on the above description, the business mode may include a content protection mode and a non-content protection mode, and the business mode may be custom configured by a business object.

[0162] Optionally, the live video interface includes a mode switching control for mode switching, wherein the mode switching control may be a control for switching a business mode.

[0163] It should be understood that the first trigger operation can be an operation for switching the mode switching control to the content protection mode. It should be understood that the premise for detecting the first trigger operation is that the business mode corresponding to the mode switching control is the non-content protection mode, so that the business mode can be switched from the non-content protection mode to the content protection mode through the first trigger operation on the mode switching control. Further, in the content protection mode, the scoreboard data in the internal area of ​​the player can be protected to prevent spoilers in the external area of ​​the player in advance.

[0164] For example, see Figure 2. Figure 6 , Figure 6 Schematic diagram of the effect of a video live broadcast interface provided by an embodiment of the present application. Figure 6As shown, page 600a may be a video live broadcast interface, which may include a player internal area 601a and a player external area 602a. The player internal area 601a may display screen scoreboard data (such as Figure 6 603a in the figure), the player external area 602a can display the running score board data (such as Figure 6 It should be understood that a data page card is displayed in the player outer area 602a, and the live video interface may include a mode switching control for switching the service mode (such as Figure 6 As shown in 607a in FIG. 1 , the mode switching control can be used to switch the service mode, for example, Figure 6 607a in the figure may be a mode switching control in content protection mode, which is obtained in response to a trigger operation on a mode switching control in non-content protection mode. Further, in response to the trigger operation on the mode switching control 607a, the business mode may be switched from content protection mode to non-content protection mode. Optionally, the data page card in the player external area 602a may also be used to display some other data information, for example, the score information corresponding to each competition link (such as Figure 6 For example, the operation statistics of each player in the competition can also be displayed (such as Figure 6 For example, the total number of points scored by each player and the statistical value of the number of various technical actions performed can be recorded.

[0165] S203. If the business mode of the live video interface is the content protection mode, first alignment information for data alignment is obtained based on the content protection mode; the first alignment information includes target version information and a first picture timestamp corresponding to the target version information; the target version information is determined based on second scoreboard data corresponding to a target competitive event to be displayed in an external area of ​​the player; the first picture timestamp refers to a timestamp carried by a first picture frame associated with the target competitive event; the first picture frame is used to identify the first scoreboard data to be displayed in an internal area of ​​the player in the live competitive video.

[0166] S204. When the second score board data is obtained, the second score board data is aligned with the first score board data based on the target version information and the first screen timestamp to obtain the aligned second score board data; the timestamp corresponding to the aligned second score board data and the timestamp corresponding to the first score board data are both the first screen timestamp.

[0167] S205: When the video player plays the first picture frame, the first scoreboard data is displayed on the first picture frame in the inner area of ​​the player, and the second scoreboard data after data alignment is displayed in the outer area of ​​the player.

[0168] For steps S203-S205, reference may be made to the relevant description of steps S102-S104, which will not be repeated here.

[0169] Optionally, in an embodiment of the present application, the business mode can also be switched to a non-content protection mode, so that the scoreboard data in the screen may not be protected, thereby being more adaptable to the needs of various users and improving the user experience of live broadcasting.

[0170] Specifically, the target competitive event is a competitive event recorded in the competitive event flow list; then, the embodiment of the present application may further include the following steps: in response to a second trigger operation on the mode switching control, the business mode of the video live broadcast interface is switched from the content protection mode to the non-content protection mode; further, when the third score board data associated with the competitive update event is obtained based on the non-content protection mode, the third score board data is displayed on the second data page card corresponding to the external area of ​​the player; the competitive update event is the next competitive event of the target competitive event recorded in the competitive event flow list, the third score board data is sent by the server through the data link, and the second data page card is a data page card corresponding to the non-content protection mode.

[0171] It should be understood that the second trigger operation can be an operation for switching the mode switching control to a non-content protection mode. It should be understood that the premise for detecting the second trigger operation is that the business mode corresponding to the mode switching control is a content protection mode, so that the business mode can be switched from a content protection mode to a non-content protection mode through the second trigger operation on the mode switching control. Further, in the non-content protection mode, the scoreboard data in the internal area of ​​the player is not protected in the subsequent non-content protection mode. Instead, after acquiring new scoreboard data, the acquired scoreboard data is directly displayed in the external area of ​​the player without aligning the acquired scoreboard data with the picture frame in the live competitive video, so that the scoreboard data associated with a competitive event can be displayed in advance in the external area of ​​the player, producing a spoiler effect, so that the user can understand the progress of the competitive game ahead of the currently displayed picture.

[0172] It should be understood that the third scoreboard data can be the running scoreboard data associated with the competitive update event, that is, the scoreboard data updated by the competitive update event. The competitive update event is the next competitive event of the target competitive event recorded in the competitive event running list, and the third scoreboard data can be issued by the server through a data link, specifically, it can be issued through the basic data link of the server. The basic data link can be a link in the server for sending business data to the live broadcast client. The data link can be a link for processing the data of the competitive game (such as the competitive event running list).

[0173] It should be understood that the second data page card is a data page card corresponding to the non-content protection mode, and thus the third scoreboard data can be directly displayed in the second data page card corresponding to the external area of ​​the player.

[0174] It is understandable that the scoreboard style of the scoreboard data displayed in the second data page card can be the same as the scoreboard style of the scoreboard data displayed in the first data page card. Alternatively, the scoreboard style of the scoreboard data displayed in the second data page card can also be different from the scoreboard style of the scoreboard data displayed in the first data page card, thereby allowing the business object to more intuitively understand the current business model through the difference in scoreboard styles, thereby improving the user experience.

[0175] It is understandable that, in the non-content protection mode, when the second picture timestamp associated with the picture frame (i.e., the second picture frame) associated with the competitive update event is reached, the second picture frame can be displayed in the internal area of ​​the player. The second picture frame is also the picture frame associated with the competitive update event. The second picture timestamp is the timestamp associated with the second picture frame.

[0176] Specifically, the picture frame associated with the competitive update event is the second picture frame, and the second picture frame is used to indicate that the server has identified and obtained the fourth score board data to be displayed in the internal area of ​​the player in the competitive live video. The second picture frame is a picture frame located after the first picture frame in the competitive live video. Then, the embodiment of the present application may also include the following steps: when the playback progress of the video player reaches the second picture timestamp corresponding to the second picture frame, the second picture frame is displayed in the internal area of ​​the player, and the fourth score board data is displayed on the second picture frame.

[0177] It should be understood that the second screen frame may be a screen frame associated with the competition update event. The second screen frame may include fourth scoreboard data, and the fourth scoreboard data may refer to the scoreboard data displayed in the second screen frame, and the timestamp associated with the fourth scoreboard data may be the second screen timestamp.

[0178] It is understandable that the second picture frame is used to instruct the server to identify the fourth scoreboard data to be displayed in the internal area of ​​the player in the live competitive video. It is understandable that since the needs of the business object may change at will, the business object may switch the business mode at will in the live video interface. In order to ensure that the business mode can be quickly switched from the non-content protection mode to the content protection mode, in the non-content protection mode, the server can also continue to identify the scoreboard data in each picture frame, and determine the alignment information based on the identified scoreboard data and send it to the live broadcast client for backup, so that when the business mode is switched from the non-content protection mode to the content protection mode, the scoreboard data displayed in the external area of ​​the player can be quickly aligned with the scoreboard data displayed in the internal area of ​​the player based on the alignment information, thereby improving the efficiency of switching the business mode.

[0179] It should be understood that the timing of displaying the second frame in the internal area of ​​the player and the timing of displaying the third scoreboard data in the external area of ​​the player are not limited, and can be specifically determined based on the data processing efficiency of the third scoreboard data. For example, if the timing of displaying the third scoreboard data in the external area of ​​the player is earlier than the timing of displaying the second frame in the internal area of ​​the player, a spoiler effect will be caused; or, if the timing of displaying the third scoreboard data in the external area of ​​the player is later than the timing of displaying the second frame in the internal area of ​​the player, no spoiler effect will be caused; or, the timing of displaying the third scoreboard data in the external area of ​​the player can also be synchronized with the timing of displaying the second frame in the internal area of ​​the player.

[0180] Optionally, in an embodiment of the present application, various special effects barrages can be automatically triggered at hot moments in competitive games (such as a buzzer beater, the end of a game, a game reversal, etc.), and the need to combine the special effects barrages with the live stream can be met during the live broadcast, thereby attracting user interaction and enhancing the viewing experience.

[0181] Specifically, the embodiment of the present application may also include the following steps: if the first screen timestamp corresponding to the first screen frame belongs to the target hot spot moment, when the video player plays the first screen frame, the target barrage data bound to the target hot spot moment is displayed on the first screen frame through the barrage display mode corresponding to the video player.

[0182] The target hotspot moment may be the moment indicated by the screen timestamp associated with the competitive hotspot event. The competitive hotspot event may refer to an event that is likely to generate higher attention, for example, the competitive hotspot event may be an event at the end of a game, an event at which a game is reversed, etc., which is not limited here.

[0183] Among them, the barrage display mode can be the style of the barrage displayed by the target hotspot moment. It is understandable that the target hotspot moment can be bound with the corresponding barrage data, and then when the timestamp of the currently displayed picture frame is the target hotspot moment, the target barrage data bound to the target hotspot moment can be automatically displayed. For example, when the target hotspot moment is the moment when the game ends, the moment when the game ends can be bound with barrage data used to wish a certain team to win the championship, so that when the timestamp of the current picture reaches the target hotspot moment, the target barrage data can be automatically sent, thereby improving interactivity. It is understandable that the barrage display mode triggered by the target hotspot moment can be different from the conventional barrage style, thereby triggering a special barrage display style to highlight the particularity of the target hotspot moment, thereby enhancing interactivity.

[0184] For example, see Figure 7 , Figure 7 1 is a schematic diagram of the effect of a hot moment screen provided by an embodiment of the present application. Figure 7 As shown, the timestamp corresponding to the picture frame in the picture 700a is the target hotspot moment, then, the barrage data of a special barrage display style can be displayed in the picture 700a, for example, Figure 7 701a in the figure may be barrage data displayed based on a special barrage display style, and the barrage content may be "Congratulations to team D1 for winning the championship".

[0185] Optionally, in an embodiment of the present application, clips of hot events in the live competitive video can be automatically edited to obtain hot event clips, thereby increasing the production speed of videos of hot competitive events of competitive games, so that each user can pay attention to the hot events of competitive games more quickly and increase the attention to competitive games.

[0186] Specifically, the embodiments of the present application may further include the following steps: if the target competitive event is a competitive hot spot event, then in response to a view trigger operation on a hot spot segment set in an area outside the player, hot event segments associated with the competitive hot spot event and segment evaluation values ​​corresponding to the hot event segments are displayed in the hot spot segment set; the hot event segment is a video segment cut from a competitive live video and matching the competitive hot spot event; the segment start timestamp corresponding to the hot event segment is the first screen timestamp in the first alignment information.

[0187] Among them, the introduction of the competitive hot event can be found in the above-mentioned related description, which will not be repeated here.

[0188] It should be understood that the hot segment set may be a set of video segments of competitive hot events (i.e., hot event segments). The viewing trigger operation may be an operation for viewing the hot segment set. For example, the player external area may include a hot viewing control for viewing the hot segment set, and the viewing trigger operation may be a touch operation on the hot viewing control, such as clicking the hot viewing control.

[0189] It is understandable that the hot event segment is a video segment that matches the competitive hot event cut from the competitive live video; the segment start timestamp corresponding to the hot event segment is the first screen timestamp in the first alignment information. Among them, the segment start timestamp can be the timestamp of the first screen frame of an event segment. It is understandable that during the playback of the live video, it is possible to continuously detect whether the competitive event corresponding to the current screen frame is a competitive hot event, so that when the current screen frame is detected as a competitive hot event, it means that the current screen frame is the first screen frame of a competitive hot event, so that the current screen frame can be used as the segment start timestamp of the corresponding hot event segment, so that the screen frame within a certain length of time after the segment start timestamp is cut with the segment start timestamp as the start, as the hot event segment. It is understandable that the duration of the hot event segment cut can be a preset duration, or it can be based on the information in the competitive event stream list to determine the segment start timestamp and the segment end timestamp, and then the segment with the segment start timestamp and the segment end timestamp quality inspection is used as the hot event segment. The segment end timestamp may be determined based on a screen timestamp corresponding to a next competitive event of the hot competitive event.

[0190] It should be understood that the segment evaluation value corresponding to the hot event segment can be the heat level of the corresponding competitive hot event. The segment evaluation value can be determined based on the type of competitive hot event (for example, the segment evaluation value of the hot event segment of the event type of scoring is higher than that of the event type of error), or it can be determined based on the heat of the players associated with the competitive event (for example, the greater the heat of the players associated with the competitive event, the greater the segment evaluation value of the hot event segment), or it can be determined based on the timing of the game in which the competitive event is located (for example, the closer the competitive event is to the match point, the greater the segment evaluation value of the corresponding hot event segment), which is not limited here.

[0191] See also Figure 8 , Figure 8 Schematic diagram of the effect of displaying a hotspot segment set provided by an embodiment of the present application. Figure 8As shown, page 800a may be a video live broadcast interface, which may include an internal player area 801a and an external player area 802a. In the external player area 802a, a data page card may be displayed, and a hotspot viewing control 803a for triggering the display of a hotspot segment set may also be included. Further, in response to a trigger operation on control 803a (step S81), that is, a view trigger operation on a hotspot segment set, a hotspot segment set is displayed in the external player area. For example, the hotspot segment set displayed in the external player area may include hot event segments of event 1, hot event segments of event 2, hot event segments of event 3, hot event segments of event 4, and so on. For example, the hot event segment of event 1 may be displayed as follows. Figure 8 As shown in 804a in FIG. 1 , the cover of event 1, the title of event 1, and the number of views of the hot event clip (ie, the number of viewers) and other information may be displayed.

[0192] Furthermore, the embodiments of the present application may also include the following steps: when playing the live competitive video through a video player, in response to a clip playback operation for a hot event clip, in an internal area of ​​the player, the playback content of the video player is switched from the live competitive video to the hot event clip; or, after the live competitive video is played through the video player, in response to a clip playback operation for a hot event clip, the hot video clip is played through the video player in the internal area of ​​the player.

[0193] The clip playback operation may be an operation for playing a hot event clip. The clip playback operation may be a touch operation on the area of ​​the hot event clip displayed in the external area of ​​the player (such as clicking on the area where the hot event clip is located). Furthermore, in response to the clip playback operation on the hot event clip, the playback content of the video player may be switched from the live sports video to the hot event clip in the internal area of ​​the player.

[0194] It is understandable that after playing the live sports video through the video player, the hot video segment can also be played in the internal area of ​​the player in response to the segment play operation for the hot event segment. That is, during the playing of the live sports video, or after playing the live sports video, the hot video segment can be played in the internal area of ​​the player through the video player in response to the segment play operation for the hot event segment.

[0195] For further information, see Fig. 9 , Fig. 9 is a flow chart of a live broadcast data processing method provided by an embodiment of the present application. The method can be executed by a server, for example, the server can be the above-mentioned Figure 1The method may at least include the following steps S301 to S304.

[0196] S301, obtaining a live sports video, performing picture recognition on a first picture frame associated with a target sports event in the live sports video, identifying and obtaining first scoreboard data to be displayed in an internal area of ​​a player, and using a timestamp corresponding to the first picture frame as a first picture timestamp corresponding to the first scoreboard data.

[0197] The competitive live video is a live video of a competitive match. The target competitive event and the first picture frame can be found in the above-mentioned related descriptions, which will not be repeated here.

[0198] It should be understood that performing picture recognition on the first picture frame may refer to identifying the scoreboard data in the first picture frame, thereby identifying the first scoreboard data to be displayed in the internal area of ​​the player, and the timestamp corresponding to the first picture frame may be used as the first picture timestamp corresponding to the first scoreboard data.

[0199] Specifically, obtaining a live competitive video, performing picture recognition on a first picture frame associated with a target competitive event in the live competitive video, identifying first scoreboard data to be displayed in an internal area of ​​a player, and using a timestamp corresponding to the first picture frame as a first picture timestamp corresponding to the first scoreboard data, may include the following steps: receiving a video acquisition data stream sent by a picture acquisition terminal, performing video decoding on the video acquisition data stream, and obtaining live competitive data associated with a target competitive event; the video acquisition data stream is obtained by the picture acquisition terminal performing picture acquisition on the target competitive event; further, performing video encoding on the live competitive data, and obtaining a live competitive video carrying multiple picture frames; performing video encapsulation processing on the live competitive video, and obtaining a monitoring video stream for the live competitive video; a picture acquisition terminal may be used to obtain a video stream for monitoring the live competitive video. A face frame corresponds to a coding timestamp; a coding timestamp refers to a timestamp written when a picture frame is obtained by encoding; further, the monitoring video stream is unpacked to obtain a live competitive video, and a scoreboard is identified for each of the multiple picture frames in the live competitive video through a first identification service thread to obtain a scoreboard identification result for each picture frame; further, if the scoreboard identification result of each picture frame contains identified scoreboard data associated with a target competitive event, the picture frame corresponding to the identified scoreboard data is used as the first picture frame, and when the scoreboard data identified from the first picture frame is used as the first scoreboard data to be displayed in the internal area of ​​the player, the coding timestamp carried by the first picture frame is used as the first picture timestamp corresponding to the first scoreboard data.

[0200] It is understandable that the picture acquisition terminal can be a terminal device for collecting videos of competitive games. It should be understood that the picture acquisition terminal can send the collected video acquisition data stream to the server. The video acquisition data stream is obtained by the picture acquisition terminal performing picture acquisition for the target competitive event, that is, the video acquisition data stream includes the data of the picture frame corresponding to the target competitive event.

[0201] It should be understood that the competitive live broadcast data can be data obtained by decoding the video acquisition data stream. It should be understood that after obtaining the video acquisition data stream, the server can decode the video acquisition data stream to obtain the competitive live broadcast data associated with the target competitive event (i.e., the competitive live broadcast data containing the picture frames corresponding to the target competitive event). It can be understood that the video decoding here can refer to the process of restoring the compressed data stream to video data.

[0202] It should be understood that the live competitive video may refer to a video obtained by video encoding based on the live competitive data. The live competitive video carries multiple frames, and the multiple frames may include frames corresponding to the above-mentioned target competitive events. Among them, video encoding processing refers to the process of converting the original video format file into another video format file through compression technology. The main purpose of video encoding is to achieve compatibility and efficiency of video data, and is suitable for various applications and hardware.

[0203] It should be understood that in a live competitive video, one picture frame corresponds to one coding timestamp, and a coding timestamp refers to the timestamp written when a picture frame is encoded. It is understandable that since the actual occurrence time of the competitive game is earlier than the time when the corresponding picture frame is encoded, the actual occurrence time of a picture is earlier than the coding timestamp of the corresponding picture frame. In the live broadcast system, since the video encodes a picture frame and sends a picture frame when it obtains a picture frame, the encoder will generate a coding timestamp when encoding. The main function of the coding timestamp is to tell the video player when the picture frame should be displayed. It timestamps each picture frame based on the codec, thereby achieving synchronous playback of audio and video.

[0204] The monitoring video stream may be a video stream obtained by encapsulating the live competitive video. The monitoring video stream may be a video stream used for image recognition, and then the image frames in the monitoring video stream may be recognized. Video encapsulation processing may refer to storing the encoded and compressed video data (such as live competitive video) into a file in a certain format. This process does not involve editing or modifying the video content, but simply integrates the encoded video and audio data into a specific file format for easy playback, storage and transmission.

[0205] It can be understood that the server may include a codec component, and then the picture acquisition terminal can send the video acquisition data stream to the codec component in the server, and the codec component can decode the video acquisition data stream to obtain competitive live broadcast data, and then perform video encoding based on the competitive live broadcast data to obtain competitive live video, and then perform video encapsulation processing on the competitive live broadcast video to obtain a video stream.

[0206] It is understandable that when performing picture recognition, the monitoring video stream can be unpacked to obtain the live competitive video, and then each picture frame in the live competitive video obtained by video unpacking can be recognized. The video unpacking process here can refer to decomposing an already packaged video file (such as MP4 or MKV) into its original components, such as video data (i.e., the above-mentioned multiple picture frames), audio data, subtitle data, etc. This process is the opposite of the video packaging process.

[0207] Among them, the picture recognition of the picture frames in the live competitive video can be to perform scoreboard recognition on each picture frame in the live competitive video, from to the scoreboard recognition result of the picture frame. The scoreboard recognition result can be the result of recognizing the scoreboard in the picture frame. If the scoreboard data of the picture frame is recognized, the scoreboard recognition result of the picture frame includes the recognized scoreboard data (i.e., the picture scoreboard data); if the scoreboard data of the picture frame is not recognized, the scoreboard recognition result of the picture frame is used to indicate that the scoreboard data of the picture frame is not recognized. It can be understood that the scoreboard recognition of the picture frame can be the data obtained by performing image optical character recognition (OCR recognition) on the picture frame. For example, a pre-trained image recognition model can be called to perform scoreboard recognition on the picture frame to obtain the recognized scoreboard recognition result.

[0208] It is understood that in the embodiment of the present application, the server may further include a screen recognition component, which may be used to perform screen recognition on each screen frame corresponding to the monitored video stream, obtain the scoreboard recognition results of each screen frame, and may be used to determine the screen timestamp corresponding to the scoreboard data in the scoreboard recognition results recognized for each screen frame. It should be understood that the processing logic of the codec component and the above-mentioned screen recognition component may be decoupled from each other, and the two may be deployed on the same server or on different servers, which is not limited here.

[0209] It is understandable that the monitoring video stream in the embodiment of the present application can be a video stream encapsulated in a corresponding data format. For example, the monitoring video stream can be an HLS stream (a video stream in a data format). Then, the monitoring video stream can be a video stream parsed into segmented data (such as TS segments) in a timed manner, and then distributed to the receiving end (such as a screen recognition component) through the segmented data. It should be understood that the receiving end of the monitoring video stream (i.e., the screen recognition component) can use the segmented data to generate corresponding pictures (i.e., picture frames) and SEI information (i.e., supplementary enhancement information) corresponding to the pictures at a frequency of one second.

[0210] It is understandable that the queryable structure data of the picture frame can be determined based on the scoreboard recognition result of the determined picture frame, and the queryable structure data can be used to query the scoreboard data of the picture frame (i.e., the picture scoreboard data). In the queryable data structure, the identified scoreboard data and the associated information of the picture frame can be included. If the scoreboard recognition result includes the scoreboard data of the picture frame, the queryable data structure corresponding to the picture frame can include the identified scoreboard data, and if the scoreboard recognition result indicates that the scoreboard data of the picture frame is not recognized, the scoreboard data in the queryable data structure corresponding to the picture frame can be a null value. It should be understood that the associated information of the picture frame can be information associated with the picture frame itself, such as the associated information can include the frame identification of the picture frame, the picture timestamp of the picture frame, wherein the frame identification of the picture is used to uniquely identify a picture frame, and the introduction of the picture timestamp of the picture frame can refer to the above-mentioned related description. Optionally, when the monitoring video stream is sent to the picture recognition component through the fragment data, then the queryable structure data can also include the name of the fragment data to which the picture frame belongs (i.e., the fragment name).

[0211] For example, the queryable structure data of the picture frame can refer to the following table (2):

[0212] Table (2)

[0213]

[0214] The name of the slice to which the picture frame belongs refers to the name of the slice data to which the picture frame belongs when the picture frame is parsed from the monitoring video stream. The scoreboard parsing mode may refer to a mode for recognizing a picture frame, such as a mode in which the scoreboard is centered, to the left, or to the right in the picture frame. When recognizing the scoreboard, the position of the scoreboard can be quickly determined based on the scoreboard mode, thereby performing OCR recognition on the text in the scoreboard to obtain scoreboard data, which helps to improve the efficiency of picture recognition.

[0215] It is understandable that in the picture recognition component, an independent thread (such as the first recognition service thread) can be used to specifically recognize the scoreboard of the picture frame. In other words, the first recognition service thread can be a thread used to recognize the scoreboard of the picture frame, so that the scoreboard recognition result of each picture frame can be obtained through the first recognition service.

[0216] It can be understood that it is possible to detect whether there is scoreboard data associated with the target competitive event in the scoreboard recognition results of each picture frame. If there is identified scoreboard data associated with the target competitive event in the scoreboard recognition results of each picture frame, the picture frame corresponding to the identified scoreboard data (i.e., the identified scoreboard data associated with the target competitive event) can be used as the first picture frame, and the scoreboard data identified from the first picture frame can be used as the first scoreboard data to be displayed in the internal area of ​​the player, and the encoded timestamp carried by the first picture frame can be used as the first picture timestamp corresponding to the first scoreboard data.

[0217] Specifically, using the encoded timestamp carried by the first picture frame as the first picture timestamp corresponding to the first score board data may include the following steps: establishing a first mapping relationship between the first score board data and the first picture timestamp; further, based on the first mapping relationship, using the first score board data as the value in a key-value pair, and using the first picture timestamp as the key in the key-value pair, and determining, based on the key-value pair, that the timestamp corresponding to the first score board data is the first picture timestamp.

[0218] Among them, the first mapping relationship can be a mapping relationship between the first scoreboard data and the first picture timestamp. That is to say, the first picture timestamp can establish a corresponding relationship with the scoreboard data (i.e., the first scoreboard data) identified in the corresponding picture frame. Further, a key-value pair can be established to store the mapping relationship. Specifically, the first mapping relationship can use the first scoreboard data as the value in the key-value pair, and the first picture timestamp as the key in the key-value pair. For example, the encoding timestamp of the first picture frame is T1, and the first scoreboard data can include {score: 60:70; section time: 5'45'}, then, in the key-value pair corresponding to the first mapping relationship, key = T1, value = {score: 60:70; section time: 5'45'}.

[0219] For example, see Fig.10 , Fig.10 FIG. 1 is a flow chart of a picture recognition process provided by an embodiment of the present application. Fig.10 As shown, first, the codec component can send the monitoring video stream data to the picture recognition component, Fig.10901a) in the video stream is processed by slicing to obtain slicing data. For example, each time a certain time length (such as 2S, which can be preset according to actual needs) of video stream data is obtained, a slicing data is determined based on the video stream data of the certain time length, and the slicing data is sent to the receiving end (such as the screen recognition component) to send the monitoring video stream data to the receiving end. For example, as time accumulates, slicing 902a, slicing 903a and slicing 904a can be generated in sequence, and more slicing data can be generated later and sent to the screen recognition component. It can be understood that, in each generated fragment data, multiple fragment packages can be included, and one fragment package can be used to include audio and video data corresponding to the corresponding picture. For example, fragment 902a can include fragment package TS11, fragment package TS12, fragment package TS13, fragment package TS14, etc., fragment 903a can include fragment package TS21, fragment package TS22, fragment package TS23, fragment package TS24, etc., and fragment 904a can include fragment package TS31, fragment package TS32, fragment package TS33, fragment package TS34, etc.

[0220] Furthermore, the codec component can send the slice data to the picture recognition component, and then the picture recognition component can process each slice data in chronological order. Specifically, each slice data can be parsed to obtain multiple picture frames corresponding to the slice data. For example, Fig.10 The multiple picture frames shown in 905a in FIG. 9 may be multiple picture frames obtained by parsing the slice 903a, and the multiple picture frames may include a picture frame F1 (such as Fig.10 906a in FIG. 1 ), picture frame F2 (as shown in FIG. 1 ). Fig.10 907a in FIG. 1 ), picture frame F3 (as shown in FIG. 1 ). Fig.10 908a in FIG. 1 ), frame F4 (as shown in FIG. 1 ). Fig.10 909a in the figure) and so on. Further, each picture frame can be framed to obtain the scoreboard recognition result of each picture frame, for example, the scoreboard recognition result 910a of the picture frame F1, the scoreboard recognition result 911a of the picture frame F2, the scoreboard recognition result 912a of the picture frame F3, and the scoreboard recognition result 913a of the picture frame F4 can be obtained. Further, the mapping relationship between the picture timestamp of each picture frame and the scoreboard data in the corresponding scoreboard recognition result can be stored (such as Fig.10 As shown in 914a in the figure), for example, it can be stored through key-value pairs. For details, please refer to the above-mentioned related description, which will not be repeated here.

[0221] It is understandable that, while the monitoring video stream is encapsulated by the codec component, the live video stream can also be encapsulated at the same time, and the live video stream can be directly sent to the live client.

[0222] Specifically, the embodiments of the present application may further include the following steps: when performing video encapsulation processing on the competitive live video to obtain a monitoring video stream for the competitive live video, encapsulating it according to the data transmission protocol to obtain a live video stream for the competitive live video, and sending the live video stream to the live client according to the data transmission protocol, so that when the live client parses the competitive live video from the live video stream based on the data transmission protocol to obtain the competitive live video, the competitive live video is played through a video player.

[0223] Among them, the data transmission protocol may refer to a protocol for transmitting a live video stream. For example, the data transmission protocol may include transmission protocols such as RTMP, HLS, etc., which are not limited here. Further, a live video stream for a competitive live video can be obtained by encapsulating it according to the data transmission protocol. The method for encapsulating the live video stream can refer to the relevant description of encapsulating the above-mentioned monitoring video stream, which is not repeated here. Among them, the live video stream can be a data stream obtained by video encapsulation of the competitive live video. It should be understood that the server can send the live video stream to the live client, so that when the live client parses the competitive live video from the live video stream based on the data transmission protocol, the competitive live video is played through the video player, that is, the competitive live video is played in the video live interface in the live client. Its specific display method can refer to the above-mentioned related description, which is not repeated here.

[0224] S302: Obtain a competitive event flow list associated with a target competitive event, and parse the competitive event flow list to obtain an event occurrence timestamp and an event link timing timestamp corresponding to the target competitive event.

[0225] It should be understood that the introduction of the competitive event flow list can refer to the above related description, which will not be repeated here. The introduction of the event occurrence timestamp can refer to the above related description, which will not be repeated here.

[0226] It should be understood that the event link timing timestamp may refer to the timing information of the competitive link to which the event belongs. It should be understood that a competitive game may generally include multiple competitive links, for example, a basketball competitive game may include four competitive links, each of which has a corresponding timing.

[0227] Specifically, obtaining a competitive event flow list associated with a target competitive event, parsing the competitive event flow list to obtain an event occurrence timestamp and an event link timing timestamp corresponding to the target competitive event, may include the following steps: pulling the competitive event flow list from a data source interface provided by a data acquisition device; the competitive event flow list is determined by event flow record data recorded by the data acquisition device for the target competitive event; the event flow record data includes the event occurrence timestamp of the target competitive event and the competitive link to which the target competitive event belongs, as well as the event link timing timestamp corresponding to the competitive link, recorded by the data acquisition device; further, obtaining the event flow record data in the competitive event flow list, performing data conversion on the event flow record data, converting to obtain change flow structure data for the target competitive event, and obtaining the event occurrence timestamp corresponding to the target competitive event and the event link timing timestamp corresponding to the target competitive event during the competitive link from the change flow structure data.

[0228] It is understandable that the data acquisition device can be a device for collecting data related to competitive games, and the data acquisition device can provide a data source interface for other devices to obtain data. The data source interface can be an interface for collecting data related to competitive games. It should be understood that the data server can obtain a competitive event flow list from the data acquisition device based on the data source interface. It should be understood that the competitive event flow list is determined by the event flow record data recorded by the data acquisition device for the target competitive event; the event flow record data includes the event occurrence timestamp of the target competitive event recorded by the data acquisition device and the scoreboard data of the target competitive event (such as the competitive link to which the target competitive event belongs, the event link timing timestamp corresponding to the competitive link, the score, etc.), as well as the event description information of the competitive event, etc.

[0229] It should be understood that the change pipeline structure data can be the data obtained by data conversion of event pipeline record data. Data conversion of event pipeline record data can refer to difference calculation of event pipeline record data to obtain the change pipeline (i.e. change pipeline structure data) of each competitive event in the competitive event pipeline list. For example, the competitive event pipeline list includes competitive event J1 and competitive event J2, and competitive event J1 is the previous competitive event of competitive event J2. Then, when the time pipeline record data in the competitive event pipeline list is converted, the difference information of the scoreboard data between competitive event J2 and competitive event J1 can be determined, thereby determining the change pipeline structure data based on the determined difference information. The change pipeline structure data can represent the information of the changes produced by each competitive event. Further, a matching operation can be triggered based on the change pipeline structure data, and the matching operation can refer to the operation of matching the picture timestamp associated with each competitive event. Furthermore, the event occurrence timestamp corresponding to the target competitive event and the event link timing timestamp corresponding to the target competitive event during the competitive link can be obtained from the change pipeline structure data to determine the screen timestamps required to be aligned for the competitive event.

[0230] It is understood that in the embodiment of the present application, the server may further include a data processing component, which may be used to receive a list of competitive event streams sent by a data acquisition terminal and determine the screen timestamps required to be aligned for each competitive event. It is understood that the processing logic of the data processing component, the encoding and decoding component, and the screen recognition component may be decoupled from each other, and the three may be deployed on the same server or on different servers, which is not limited here.

[0231] S303. If a first screen timestamp matching the event link timing timestamp is found within a time range adjacent to the event occurrence timestamp, the scoreboard data corresponding to the event occurrence timestamp is used as second scoreboard data to be displayed in the external area of ​​the player, and target version information for the target competitive event is generated based on the second scoreboard data; both the external area of ​​the player and the internal area of ​​the player are areas in the video live broadcast interface corresponding to the competitive live broadcast video.

[0232] It should be understood that, because the scoreboard style for displaying the scoreboard data in the picture frame is different and diverse due to different broadcasters, it is difficult to identify and use. And the event link timing timestamp in the scoreboard data is usually standardized and unified in format, so the embodiment of the present application can obtain the identified event link timing timestamp (qt) by regular expression matching, and store it together with the picture timestamp of the corresponding picture. However, because there may be multiple competitive links in each competitive game, such as at least four sections in a basketball game, the same event link timing timestamp (qt) will appear at least four times. Therefore, it is necessary to query the picture timestamp near the event occurrence timestamp (i.e., within the adjacent time range of the event occurrence timestamp) and the event link timing timestamp is the same according to the event occurrence timestamp of the competitive event, to determine the picture timestamp corresponding to the competitive event.

[0233] It should be understood that the adjacent time range can be a time range within a certain range near the event occurrence timestamp, for example, the adjacent time range can include a 2S time period of the event occurrence timestamp. It should be understood that the first screen timestamp that matches the event link timing timestamp can refer to the event link timing timestamp included in the identified scoreboard data, and the screen timestamp of the screen frame that is the same as the event link timing timestamp of the target competitive event in the competitive event flow list.

[0234] It should be understood that if a first picture timestamp matching the event link timing timestamp is found within the adjacent time range of the event occurrence timestamp, it is explained that the first picture timestamp is the picture frame corresponding to the target competitive event, and then the first picture timestamp can be used as the picture timestamp required to align the second score board data corresponding to the target competitive event. Further, the score board data corresponding to the event occurrence timestamp can be used as the second score board data to be displayed in the external area of ​​the player, and the target version information for the target competitive event is generated based on the second score board data. The introduction of the target version information can refer to the above-mentioned related description, which is not repeated here. It is understandable that if a first picture timestamp matching the event link timing timestamp is not found within the adjacent time range of the event occurrence timestamp, it is explained that the version information corresponding to the event occurrence timestamp does not need to be aligned with the first picture timestamp, and then the version information corresponding to the next picture frame of the first picture frame is determined, that is, the version information corresponding to the picture timestamp of the next picture frame is determined.

[0235] S304. Determine first alignment information for aligning the second score board data with the first score board data through the target version information and the first picture timestamp, and send the first alignment information to the live broadcast client. The first alignment information is used to instruct the live broadcast client to align the second score board data with the first score board data based on the target version information and the first picture timestamp when the service mode is the content protection mode, so as to obtain the second score board data after data alignment. The timestamp corresponding to the second score board data after data alignment and the timestamp corresponding to the first score board data are both the first picture timestamp. The first picture timestamp is used to instruct the live broadcast client to display the first score board data on the first picture frame in the internal area of ​​the player and display the second score board data after data alignment in the external area of ​​the player when the first picture frame is played through the video player.

[0236] The first alignment information can be introduced by referring to the above related description, which will not be described in detail here. The method for the live broadcast client to align the second scoreboard data with the first scoreboard data based on the first alignment information can be introduced by referring to the above related description, which will not be described in detail here.

[0237] Optionally, when the server fails to recognize scoreboard data in the picture frame (i.e., the second picture frame) corresponding to the competitive event after the current competitive event (i.e., the target competitive event), it can predict and generate a virtual SEI (i.e., predict the predicted timestamp of the second picture frame, also called the virtual timestamp) based on the first picture timestamp (i.e., the picture SEI1 information) corresponding to the currently recognized first scoreboard data and the physical occurrence event (i.e., the event occurrence timestamp) of the current competitive event (i.e., the target competitive event), and generate another alignment information (i.e., the second alignment information) through the virtual SEI (i.e., the predicted timestamp of the second picture frame) and the updated version information corresponding to the competitive update event.

[0238] Specifically, the multiple picture frames included in the competitive live video also include a second picture frame in addition to the first picture frame, the second picture frame is a picture frame in which a competitive update event occurs, the competitive update event is the next competitive event of the target competitive event, and the second picture frame is a picture frame after the first picture frame; the score board recognition result of each picture frame includes the score board recognition result corresponding to the second picture frame; then, the embodiment of the present application may also include the following steps: if the score board recognition result corresponding to the second picture frame indicates that the score board data associated with the competitive update event is not recognized in the second picture frame, then based on the time difference between the first picture timestamp corresponding to the first picture frame and the event occurrence timestamp corresponding to the target competitive event, the picture timestamp corresponding to the second picture frame is time predicted to obtain a predicted timestamp for the second picture frame, and the second alignment information for sending to the live broadcast client is constructed through the predicted timestamp and the updated version information corresponding to the competitive update event.

[0239] The second frame may be a frame different from the first frame. The competition update event may be the next competition event of the target competition event, the second frame may be a frame where the competition update event occurs, and the second frame may be a frame after the first frame.

[0240] It should be understood that the score board recognition result of each picture frame includes the score board recognition result corresponding to the second picture frame. The method for determining the score board recognition result can refer to the above related description, which will not be repeated here.

[0241] It should be understood that the score board recognition result corresponding to the second picture frame indicates that the score board data associated with the competition update event is not recognized in the second picture frame. This may be due to the current slow picture recognition, slow data transmission, etc., which leads to the failure to timely recognize the score board recognition result.

[0242] It is understandable that if a certain picture frame is not identified to the corresponding scoreboard data, the event occurrence timestamp of the competitive event corresponding to the version information in the alignment information previously determined and the picture timestamp in the alignment information can be used for prediction. The prediction method is to calculate the time difference between the previous picture timestamp and the physical time of the event (i.e., the event occurrence timestamp), and add the time difference to the event occurrence timestamp of the competitive event corresponding to the picture frame that has not been identified to the corresponding picture timestamp, and obtain the predicted timestamp, that is, generate a virtual SEI for distribution. The predicted timestamp can be understood as a virtual timestamp used to determine the version information corresponding to the picture frame, and the predicted timestamp may be different from the real picture timestamp of the picture frame.

[0243] It can be understood that the calculation of the predicted timestamp of the second picture frame can be determined based only on the time difference between the picture timestamp of the first picture frame and the event occurrence timestamp of the target competitive event, or it can be determined based on the time difference between the event occurrence timestamps of a larger number of updated competitive events between the target competitive events and their corresponding picture timestamps.

[0244] For example, the predicted timestamp of the second picture frame can be determined based on the time difference between the picture timestamp of the picture frame corresponding to the competitive event and the corresponding event occurrence timestamp in the first n-1 versions of information. For example, the predicted timestamp of the second picture frame corresponding to the nth competitive event can be determined by the following formula (1).

[0245] SEI n =((SEI1-TV1)+(SEI2-TV2)+......+(SEI n-1 -TV n-1 )) / n+TV n

[0246] Formula (1)

[0247] Among them, SEI n is the predicted timestamp of the picture frame corresponding to the competitive event (such as competitive update event) corresponding to the version information in the nth alignment information, TV n is the event occurrence timestamp of the competitive event corresponding to the version information in the nth alignment information. SEI1 is the picture timestamp of the picture frame corresponding to the competitive event corresponding to the version information in the first alignment information, and TV1 is the event occurrence timestamp of the competitive event corresponding to the version information in the first alignment information. Then, SEI1-TV I It is the time difference between the screen timestamp in the first alignment information and the event occurrence timestamp of the competitive event corresponding to the version information in the alignment information.

[0248] Furthermore, the updated version information may refer to the version information corresponding to the predicted timestamp and the competitive update event. The method for determining the updated version information may refer to the relevant description of the above-mentioned target version information, which will not be elaborated here.

[0249] The second alignment information may refer to alignment information determined based on the update version information and the predicted timestamp corresponding to the second picture frame. The second alignment information may be used to align the scoreboard data corresponding to the competition update event with the predicted timestamp corresponding to the second picture frame.

[0250] It should be understood that, when the predicted timestamp of the second picture frame (i.e., the virtual SEI) is predicted, another thread (i.e., the second identification service thread) can be started to continue the identification, and before the virtual SEI takes effect (i.e., before the video player plays the second picture frame), the encoding timestamp of the second picture frame where the scoreboard data identified for the second picture frame is located (i.e., the second picture timestamp, for example, SEI2) is sent to the live broadcast client, so as to replace the virtual SEI in the original alignment information (i.e., the second alignment information) with the sent second picture timestamp (e.g., SEI2).

[0251] Specifically, the embodiment of the present application may further include the following steps: when a predicted timestamp for a second picture frame is predicted, a scoreboard is identified on the second picture frame through a second identification service thread to obtain an auxiliary identification result of the second picture frame; the second identification service thread is a thread used to assist the first identification service thread in identifying the scoreboard; further, if the auxiliary identification result indicates that scoreboard data associated with a competitive update event is identified, the identified scoreboard data is used as fourth scoreboard data, and the encoded timestamp corresponding to the second picture frame is used as the second picture timestamp corresponding to the fourth scoreboard data; further, the second picture timestamp is sent to a live broadcast client, so that the live broadcast client updates and replaces the picture frame timestamp corresponding to the second picture frame from the predicted timestamp to the second picture timestamp before playing the second picture frame through a video player; wherein the predicted timestamp is later than the second picture timestamp.

[0252] Among them, the second identification service thread can be a thread for assisting the first identification service thread in scoreboard identification, and can be used to separately process the picture timestamps corresponding to the competitive events for which the picture timestamps are not identified, thereby improving the processing efficiency of the competitive events for which the picture timestamps are not identified. It can be understood that the second identification service thread and the above-mentioned first identification service thread can both be threads in the component in the picture identification. Based on this, the picture frames for which the scoreboard data is not identified by the first identification service thread can be assisted in identification by the second service thread, so that without affecting the normal picture recognition processing of the first identification service thread, the picture frames with picture recognition errors can be additionally and independently re-identified, so that without affecting the recognition of most of the picture frames, the actual scoreboard data can be determined as much as possible, so as to determine the alignment information with higher accuracy based on the identified scoreboard data, so as to ensure the alignment of the picture with the external data as much as possible.

[0253] Among them, the auxiliary recognition result can be the result obtained by the second recognition service thread performing scoreboard recognition on the second picture frame. The method for determining the auxiliary recognition result of the second picture frame can refer to the relevant description of determining the scoreboard recognition result of the first picture frame, which is not repeated here. It should be understood that if the auxiliary recognition result indicates that the scoreboard data in the second picture frame is not recognized, the second picture frame may no longer be subjected to picture recognition.

[0254] It should be understood that the fourth scoreboard data may be scoreboard data identified from the second picture frame. Further, the encoding timestamp corresponding to the second picture frame may be used as the second picture timestamp corresponding to the fourth scoreboard data. The second picture timestamp may then be sent to the live broadcast client, so that before the live broadcast client plays the second picture frame through the video player, the picture frame timestamp corresponding to the second picture frame is updated and replaced by the predicted timestamp to the second picture timestamp, that is, the timestamp in the alignment information associated with the second picture frame may be replaced by the predicted timestamp to the second picture timestamp, and then the timestamp aligned with the version information in the alignment information associated with the second picture frame may be adjusted from the predicted timestamp to the second picture timestamp, and then when the second picture frame is played through the video player, the running scoreboard data corresponding to the version information aligned with the second picture timestamp is displayed.

[0255] It is understandable that the predicted timestamp is later than the second picture timestamp, so that even if the scoreboard data of the second picture frame is not determined, the live broadcast client will not display the running scoreboard data of the corresponding version information in advance when playing the second picture frame (because the version information corresponding to the second picture frame is aligned and bound to the corresponding predicted timestamp), thereby avoiding the scoreboard data in the external area of ​​the player from being displayed to the user in advance.

[0256] For example, see Fig.11 , Fig.11 FIG. 1 is a schematic diagram of a video live broadcast data processing process provided by an embodiment of the present application. Fig.11 As shown, first of all, we can start from competitive games (such as Fig.11 1001a in the figure) to obtain a data source, which may specifically include performing data collection (such as Fig.11 1002a in FIG. 1002a ) and picture acquisition (as shown in FIG. Fig.11 As shown in 1003a), data collection can obtain a list of competitive event streams, and picture collection can obtain a collected video stream of a competitive live video.

[0257] Furthermore, the data link of the server may include a data processing link for performing data processing in a non-content protection mode (which may be referred to as a basic data link) and a data processing link for performing data alignment (which may be referred to as a data alignment link). It should be understood that the basic data link may include data acquisition (such as a data alignment link). Fig.11 1004a in ) and data processing (as shown in Fig.11 1005a in FIG. 1005 ), the data here is obtained (as shown in FIG. Fig.11 1004a in FIG. 1004b) may be a running list of competitive events obtained from a data collection terminal, where data processing (such as Fig.11 The method may include determining the business data corresponding to the competitive event based on the collected competitive event flow list (which may be the version information of the competitive event and the scoreboard data corresponding to the competitive event, or the version information and the full data of the competitive event), and sending the business data to the live broadcast client, so that the live broadcast client can directly display the scoreboard data in the received business data on the second data page card in the external area of ​​the player when the business mode is the non-content protection mode. Fig.11 The data alignment link may include data alignment (such as Fig.11 ), the data alignment here may refer to determining each competitive event in the competitive event flow list, the screen timestamps to be aligned, and the version information corresponding to each competitive event (such as the target version information of the target competitive event), so as to determine the alignment information based on the version information of the competitive event and the corresponding screen timestamp, and send the alignment information to the live broadcast client. It can be understood that in the embodiment of the present application, on the basis of the basic data link, the data alignment link is additionally added to send the alignment information, so that the accuracy of the synchronous display of the scoreboard data inside and outside the player can be improved in the live broadcast client, and the scoreboard data in the screen frame can be protected from being spoiled in advance.

[0258] At the same time, the image link of the server may include encoding and decoding the acquired video stream (such as Fig.11 1010a in ), and then a monitoring video stream can be generated (such as Fig.11 1011a in FIG), thereby performing picture recognition based on the monitored video stream (such as Fig.11), the scoreboard recognition results of each picture frame are obtained, and the mapping relationship between the picture timestamp of each picture frame and the scoreboard data in the corresponding scoreboard recognition result (i.e., the picture scoreboard data) can be determined, and then the data link can determine the picture timestamp corresponding to the competitive event based on the mapping relationship between the picture timestamp determined in the picture link and the picture scoreboard data, so that the data link can determine the alignment information based on the mapping relationship. It should be understood that in the embodiment of the present application, the processing processes of the picture link and the data link are independent of each other, and the data link can obtain the mapping relationship between the scoreboard data of the picture frame (i.e., the picture scoreboard data) and the picture timestamp from the picture link. It should be understood that the picture link can also encode and decode the acquired captured video stream to obtain a live video stream (such as Fig.11 1013a in the figure), so as to send the live video stream to the live client so that the live client can play the live video through the video player (such as Fig.11 In addition, the live broadcast client obtains the alignment information determined by the data link (such as 1014a in FIG. 1 ). Fig.11 After that, when the business mode is the content protection mode, the version information of the competitive event of the alignment information can be aligned with the picture timestamp in the alignment information, and then when the picture timestamp of the picture frame played in the video player reaches the picture timestamp aligned with a certain version information, the first data page card (such as Fig.11 The running score board data corresponding to the version information is displayed in 1009a), and the running score board data corresponding to the version information can be determined by the business data issued by the above-mentioned basic data link.

[0259] In addition, in the data link, you can also use the dot service pair (such as Fig.11 1015a in FIG. 1015b) performs a check on the competitive hot events in the competitive event flow list, and sends the check information of the checked competitive hot events (such as Fig.11 1016a in FIG. 1016a ) is sent to the live broadcast client so that the live broadcast client can play the video in the video player (such as Fig.11 When the picture frame displayed in the image frame is the target hotspot moment corresponding to the competitive hotspot event, the interactive special effect (such as Fig.11 As shown in 1017a in the figure), the interactive special effect may refer to displaying the target barrage data bound to the target hot spot moment in a special barrage display style in the video player. The specific display process refers to the above-mentioned related description and will not be repeated here.

[0260] See also Fig.12 , Fig.12Schematic diagram of the effect of data processing time provided by the embodiment of the present application. Fig.12 As shown, in the real world, multiple competitive events can occur in a competitive game, for example, competitive event J1 (such as Fig.12 1101a), competitive events J2 (such as Fig.12 1102a), competitive events J3 (such as Fig.12 1103a), competitive events J4 (such as Fig.12 Further, the server can obtain a list of competitive events from the data acquisition device. The list of competitive events can include information about each competitive event (such as event occurrence timestamp, scoreboard data, and some event description information, etc.), and the server can then determine the version information corresponding to each competitive event, such as version V1 corresponding to competitive event J1 (such as Fig.12 1105a in), version V2 corresponding to competition event J2 (such as Fig.12 1106a in), version V3 corresponding to competition event J3 (such as Fig.12 1107a in), version V4 corresponding to competition event J4 (such as Fig.12 1108a in the figure). It is understandable that in the competitive event flow list obtained by the server, the time interval between two adjacent competitive events and the time interval between two corresponding competitive events in the real world may have certain errors due to some data processing reasons (such as errors in the event occurrence timestamps of manually entered competitive events). For example, the time interval between the two competitive events J1 and J2 in the real world is 10S, then the version V1 of event J1 obtained by the server (such as Fig.12 1105a) and version V2 of event J2 (such as Fig.12 The time interval between the event occurrence timestamps corresponding to 1106a) is 11S.

[0261] Furthermore, the server can perform picture recognition on each collected picture frame, and can determine the picture timestamp corresponding to each competitive event based on the scoreboard recognition result of the picture recognition, thereby generating alignment information based on the picture timestamp and version information corresponding to the competitive event. It should be understood that if a picture frame does not recognize the corresponding scoreboard recognition result, a predicted timestamp of the picture frame can be generated based on the alignment information determined before the picture frame, thereby generating alignment information based on the predicted timestamp and the corresponding version information of the competitive event for distribution.

[0262] It should be understood that the server can send the business data corresponding to each competitive event to the live broadcast client, and the business data sent includes the version information of the competitive event and the corresponding running scoreboard data (such as the second scoreboard data). Fig.12 1109a in FIG. 1109 ), business data corresponding to version V2 (such as Fig.12 1110a in FIG. 1110a), business data corresponding to version V3 (such as Fig.12 1111a in), business data corresponding to version V4 (such as Fig.12 1112a in FIG. 1112b).

[0263] Furthermore, the server can send the alignment information corresponding to each competitive event to the live broadcast client. For example, when the picture timestamp SEI1 corresponding to the version V1 of the event J1 is identified, the alignment information V1+SEI1 (such as Fig.12 Similarly, when the picture timestamp SEI2 corresponding to the version V2 of the event J2 is identified, the alignment information V2+SEI2 (as shown in 1113a) can be determined. Fig.12 As shown in 1114a in FIG. 1 , when the picture timestamp SEI4 corresponding to the version V4 of the event J4 is identified, the alignment information V4+SEI4 (as shown in FIG. 1114b ) can be determined. Fig.12 However, for the version V3 of the event J1, the picture timestamp corresponding to the version V3 is not identified, and the corresponding prediction timestamp SEI3 can be generated, so as to determine the alignment information V3+SEI3 of the version V3 of the event J1 (as shown in FIG. Fig.12 1115a in FIG. 1115b).

[0264] Furthermore, in the live broadcast client, the competitive live broadcast video can be played through a video player. The played competitive live broadcast video is a recorded picture of the real world, which may also include pictures of various competitive events, such as competitive event J1 (such as Fig.12 1117a), competitive events J2 (such as Fig.12 1118a), competitive events J3 (such as Fig.12 1119a), competitive events J4 (such as Fig.121120a in). For the played picture, the time interval between the two competitive events is consistent with the event interval of the two competitive events in the real world. In the live broadcast client, various alignment information can be obtained, and then the client can align the scoreboard data to be displayed in the external area of ​​the player based on the various alignment information, so that the corresponding running scoreboard data can be displayed in the external area of ​​the player only when the picture timestamp of the picture frame played in the internal area of ​​the player reaches the picture timestamp aligned with the running scoreboard data. It is understandable that if the live broadcast client does not receive the alignment information of a certain competitive event, such as not receiving the alignment information V4+SEI4 of the competitive event J4, then after receiving the business data of the competitive event J4, the live broadcast client directly displays the scoreboard data in the business data of the competitive event J4 in the first data page card.

[0265] See also Fig.13 , Fig.13 This is a timing diagram of live data processing provided by the embodiment of the application. Specifically, at the scene of a competition, a competition event occurs (such as Fig.13 Then, the picture acquisition terminal can collect pictures related to the competition event from the competition scene (such as Fig.13 In S1202), the data collection terminal can collect a list of competitive events associated with the competitive events from the competitive game site (such as Fig.13 Further, the picture acquisition terminal may send a video acquisition data stream (such as Fig.13 In S1204), the data collection terminal can send a list of competitive events to the data processing component in the server (such as Fig.13 (S1205 in).

[0266] Furthermore, when the codec component obtains the video acquisition data stream, it can encode and decode the video acquisition data stream (such as Fig.13 S1206 in the figure), and then the picture timestamp can be written into the picture frame (such as Fig.13 In addition, the codec component can generate a monitoring video stream and send the monitoring video stream to the picture recognition component (such as S1207). Fig.13 Further, the picture recognition component can perform picture recognition on each picture frame in the monitoring video stream (such as Fig.13 S1209 in the figure) obtains the scoreboard data recognized by each picture frame (i.e., the scoreboard recognition result), thereby storing the mapping relationship between the picture timestamp and the recognized scoreboard data (e.g., Fig.13S1210 in the figure), such as the first mapping relationship described above. Further, the data processing component can query the screen timestamp (such as the screen timestamp corresponding to the scoreboard data collected by the screen recognition component) from the screen recognition component. Fig.13 S1211 in the example), and then the alignment information of the version information and the picture timestamp can be determined (such as Fig.13 Further, the data processing component may send alignment information to the live broadcast client (eg Fig.13 In addition, the data processing component can also send the collected scoreboard data and corresponding version information (ie, target business data) to the live broadcast client (eg Fig.13 (S1214 in).

[0267] In addition, the codec component can send a live video stream to the live client (such as Fig.13 Further, the live broadcast client can display the picture and data (such as Fig.13 Specifically, the live broadcast video can be displayed in the inner area of ​​the player, and the scoreboard data after data alignment can be displayed in the outer area of ​​the player. The specific alignment display method can refer to the above-mentioned related description, which will not be repeated here.

[0268] See also Fig.14 , Fig.14 Schematic diagram of the structure of a live data processing device provided by an embodiment of the present application. Fig.14 As shown, the live data processing device 1 can be a computer program running on a live client (for example, the live client in the terminal device 11a above), for example, the live data processing device 1 is an application software; it can be understood that the live data processing device 1 can be used to execute the corresponding steps of the live data processing method provided in the embodiment of the present application. Fig.14 As shown, the live broadcast data processing device 1 may include: a live broadcast interface display module 11, an alignment information acquisition module 12, a data alignment module 13, and a scoreboard display module 14;

[0269] The live broadcast interface display module 11 is used to display the video live broadcast interface corresponding to the competitive live broadcast video; the video live broadcast interface includes a player internal area and a player external area; the player internal area is used to play the competitive live broadcast video through the video player;

[0270] The alignment information acquisition module 12 is used to acquire first alignment information for data alignment based on the content protection mode if the business mode of the video live broadcast interface is the content protection mode; the first alignment information includes target version information and a first picture timestamp corresponding to the target version information; the target version information is determined based on the second scoreboard data corresponding to the target competitive event to be displayed in the external area of ​​the player; the first picture timestamp refers to the timestamp carried by the first picture frame associated with the target competitive event; the first picture frame is used to identify the first scoreboard data to be displayed in the internal area of ​​the player in the competitive live broadcast video;

[0271] The data alignment module 13 is used to align the second scoreboard data with the first scoreboard data based on the target version information and the first screen timestamp when the second scoreboard data is acquired, so as to obtain the second scoreboard data after data alignment; the timestamp corresponding to the second scoreboard data after data alignment and the timestamp corresponding to the first scoreboard data are both the first screen timestamp;

[0272] The scoreboard display module 14 is used to display first scoreboard data on the first frame in the internal area of ​​the player when the video player plays the first frame, and to display second scoreboard data after data alignment in the external area of ​​the player.

[0273] Among them, the live broadcast interface display module 11 is specifically used for:

[0274] In response to a video triggering operation for a competitive live video in a live recommendation page, a live stream pull request is sent to a server; the live stream pull request is used to instruct the server to obtain a live video stream corresponding to the competitive live video;

[0275] The live video stream returned by the server is received, and when the video player is called to parse the live video stream to obtain the competitive live video, the video live interface corresponding to the competitive live video is displayed.

[0276] The live video interface includes a mode switching control for mode switching; the service mode corresponding to the mode switching control is a non-content protection mode;

[0277] The live data processing device 1 further includes: a mode switching module 15;

[0278] The mode switching module 15 is specifically used for:

[0279] In response to a first trigger operation on the mode switching control, the business mode of the live video broadcast interface is switched from a non-content protection mode to a content protection mode.

[0280] The data alignment module 13 is specifically used for:

[0281] Receive target business data associated with the target competitive event sent by the server, and add the target business data to the data processing queue; the target business data includes target version information and second scoreboard data;

[0282] Searching for business data matching the target business data in the data processing queue, and if business data matching the target business data is found, obtaining second scoreboard data from the target business data stored in the data processing queue;

[0283] Obtain a competitive event flow list corresponding to the target version information, and obtain an event occurrence timestamp of the target competitive event from the competitive event flow list;

[0284] Based on the event occurrence timestamp and the first screen timestamp, the second score board data is aligned with the first score board data on the business timeline corresponding to the live competitive video to obtain the aligned second score board data; the timestamp of the aligned second score board data on the business timeline and the timestamp of the first score board data on the business timeline are both the first screen timestamp.

[0285] Among them, the data page card corresponding to the content protection mode is the first data page card;

[0286] The scoreboard display module 14 is specifically used for:

[0287] When the playing progress of the video player reaches the first picture timestamp corresponding to the first picture frame, the first picture frame is displayed in the internal area of ​​the player, and the first scoreboard data is displayed on the first picture frame;

[0288] When the data page card displayed in the external area of ​​the player is the first data page card, the second scoreboard data after the data whose timestamp is aligned with the timestamp of the first picture is displayed on the first data page card.

[0289] The target competitive event is the competitive event recorded in the competitive event flow list;

[0290] The live data processing device 1 further includes: a non-content protection module 16;

[0291] The non-content protection module 16 is specifically used for:

[0292] In response to a second trigger operation on the mode switching control, switching the service mode of the live video broadcast interface from the content protection mode to the non-content protection mode;

[0293] When the third score board data associated with the competitive update event is obtained based on the non-content protection mode, the third score board data is displayed on the second data page card corresponding to the external area of ​​the player; the competitive update event is the next competitive event of the target competitive event recorded in the competitive event flow list, the third score board data is sent by the server through the data link, and the second data page card is the data page card corresponding to the non-content protection mode.

[0294] The picture frame associated with the competition update event is the second picture frame, and the second picture frame is used to indicate that the server has identified and obtained the fourth scoreboard data to be displayed in the internal area of ​​the player in the competition live video, and the second picture frame is a picture frame located after the first picture frame in the competition live video;

[0295] The non-content protection module 16 is further used for:

[0296] When the playing progress of the video player reaches the second picture timestamp corresponding to the second picture frame, the second picture frame is displayed in the internal area of ​​the player, and the fourth scoreboard data is displayed on the second picture frame.

[0297] The live broadcast data processing device 1 further includes: a bullet screen display module 17;

[0298] The bullet screen display module 17 is specifically used for:

[0299] If the first screen timestamp corresponding to the first screen frame belongs to the target hot spot moment, when the video player plays the first screen frame, the target barrage data bound to the target hot spot moment is displayed on the first screen frame through the barrage display mode corresponding to the video player.

[0300] The live data processing device 1 further includes: a hot event segment module 18;

[0301] Among them, the hot event segment module 18 is specifically used for:

[0302] If the target competitive event is a competitive hot spot event, in response to a viewing trigger operation on a hot spot segment set in an area outside the player, hot event segments associated with the competitive hot spot event and segment evaluation values ​​corresponding to the hot spot event segments are displayed in the hot spot segment set; the hot event segment is a video segment cut from a competitive live video that matches the competitive hot spot event; the segment start timestamp corresponding to the hot event segment is the first screen timestamp in the first alignment information.

[0303] See also Fig.15 , Fig.15 Schematic diagram of the structure of a live data processing device provided by an embodiment of the present application. Fig.15As shown, the live data processing device 2 may be a computer program running on a server (e.g., the server 110a described above), for example, the live data processing device 2 is an application software; it is understandable that the live data processing device 2 may be used to execute the corresponding steps in the live data processing method provided in the embodiment of the present application. Fig.15 As shown, the live broadcast data processing device 2 may include: a picture recognition module 21, a flow list acquisition module 22, a version information generation module 23, and an alignment information delivery module 24;

[0304] The picture recognition module 21 is used to obtain a live sports video, perform picture recognition on a first picture frame associated with a target sports event in the live sports video, recognize and obtain first scoreboard data to be displayed in an internal area of ​​the player, and use a timestamp corresponding to the first picture frame as a first picture timestamp corresponding to the first scoreboard data;

[0305] A flow list acquisition module 22 is used to acquire a competitive event flow list associated with a target competitive event, and parse the competitive event flow list to obtain an event occurrence timestamp and an event link timing timestamp corresponding to the target competitive event;

[0306] The version information generating module 23 is used for, if a first screen timestamp matching the event link timing timestamp is found within the adjacent time range of the event occurrence timestamp, using the scoreboard data corresponding to the event occurrence timestamp as the second scoreboard data to be displayed in the player external area, and generating target version information for the target competitive event based on the second scoreboard data; the player external area and the player internal area are both areas in the video live broadcast interface corresponding to the competitive live broadcast video;

[0307] The alignment information sending module 24 is used to determine the first alignment information used to align the second score board data with the first score board data through the target version information and the first picture timestamp, and send the first alignment information to the live broadcast client. The first alignment information is used to instruct the live broadcast client to align the second score board data with the first score board data based on the target version information and the first picture timestamp when the business mode is the content protection mode, so as to obtain the second score board data after the data alignment; the timestamp corresponding to the second score board data after the data alignment and the timestamp corresponding to the first score board data are both the first picture timestamp, and the first picture timestamp is used to instruct the live broadcast client to display the first score board data on the first picture frame in the internal area of ​​the player when playing to the first picture frame through the video player, and to display the second score board data after the data alignment in the external area of ​​the player.

[0308] The picture recognition module 21 is specifically used for:

[0309] Receiving a video acquisition data stream sent by a picture acquisition terminal, decoding the video acquisition data stream, and obtaining competitive live broadcast data associated with a target competitive event; the video acquisition data stream is obtained by the picture acquisition terminal performing picture acquisition on the target competitive event;

[0310] Performing video encoding on the live broadcast data of the competition to obtain a live broadcast video of the competition carrying multiple picture frames, performing video encapsulation processing on the live broadcast video of the competition to obtain a monitoring video stream for the live broadcast video of the competition; one picture frame corresponds to one encoding timestamp; one encoding timestamp refers to a timestamp written when one picture frame is obtained by encoding;

[0311] Performing video unpacking processing on the monitoring video stream to obtain a live competitive video, and performing scoreboard recognition on each of a plurality of picture frames in the live competitive video through a first recognition service thread to obtain a scoreboard recognition result of each picture frame;

[0312] If the scoreboard recognition result of each picture frame contains identified scoreboard data associated with the target competitive event, the picture frame corresponding to the identified scoreboard data is used as the first picture frame, and when the scoreboard data identified in the first picture frame is used as the first scoreboard data to be displayed in the internal area of ​​the player, the encoded timestamp carried by the first picture frame is used as the first picture timestamp corresponding to the first scoreboard data.

[0313] The picture recognition module 21 is specifically used for:

[0314] Establishing a first mapping relationship between first scoreboard data and first picture timestamp;

[0315] Based on the first mapping relationship, the first scoreboard data is used as the value in the key-value pair, and the first screen timestamp is used as the key in the key-value pair. Based on the key-value pair, the timestamp corresponding to the first scoreboard data is determined as the first screen timestamp.

[0316] The live data processing device 2 includes: a live video sending module 25;

[0317] The live video delivery module 25 is specifically used for:

[0318] When the competitive live video is encapsulated and processed to obtain a monitoring video stream for the competitive live video, the live video stream for the competitive live video is encapsulated according to the data transmission protocol, and the live video stream is sent to the live client according to the data transmission protocol, so that when the live client parses the competitive live video from the live video stream based on the data transmission protocol to obtain the competitive live video, the competitive live video is played through a video player.

[0319] The flow list acquisition module 22 is specifically used for:

[0320] Pulling a competitive event flow list from a data source interface provided by a data acquisition device; the competitive event flow list is determined by the event flow record data recorded by the data acquisition device for a target competitive event; the event flow record data includes an event occurrence timestamp of the target competitive event recorded by the data acquisition device and a competitive link to which the target competitive event belongs, as well as an event link timing timestamp corresponding to the competitive link;

[0321] Obtain the event flow record data in the competition event flow list, perform data conversion on the event flow record data to obtain the change flow structure data for the target competition event, and obtain the event occurrence timestamp corresponding to the target competition event and the event link timing timestamp corresponding to the target competition event during the competition link from the change flow structure data.

[0322] The plurality of frames included in the live competitive video also include a second frame other than the first frame, the second frame is a frame where a competitive update event occurs, the competitive update event is a next competitive event of the target competitive event, and the second frame is a frame after the first frame; the scoreboard recognition result of each frame includes a scoreboard recognition result corresponding to the second frame;

[0323] The live data processing device 2 further includes: a prediction timestamp module 26;

[0324] The prediction timestamp module 26 is specifically used for:

[0325] If the score board recognition result corresponding to the second picture frame indicates that the score board data associated with the competitive update event is not recognized in the second picture frame, then based on the time difference between the first picture timestamp corresponding to the first picture frame and the event occurrence timestamp corresponding to the target competitive event, the picture timestamp corresponding to the second picture frame is time predicted to obtain a predicted timestamp for the second picture frame, and second alignment information for sending to the live broadcast client is constructed through the predicted timestamp and the updated version information corresponding to the competitive update event.

[0326] The live data processing device 2 further includes: a timestamp updating module 27;

[0327] The timestamp updating module 27 is specifically used for:

[0328] When the predicted timestamp for the second picture frame is predicted, the scoreboard recognition is performed on the second picture frame by the second recognition service thread to obtain an auxiliary recognition result of the second picture frame; the second recognition service thread is a thread used to assist the first recognition service thread in performing the scoreboard recognition;

[0329] If the auxiliary recognition result indicates that the scoreboard data associated with the competition update event is recognized, the recognized scoreboard data is used as the third scoreboard data, and the encoding timestamp corresponding to the second picture frame is used as the second picture timestamp corresponding to the third scoreboard data;

[0330] The second picture timestamp is sent to the live broadcast client, so that the live broadcast client updates and replaces the picture frame timestamp corresponding to the second picture frame from the predicted timestamp to the second picture timestamp before playing the second picture frame through the video player;

[0331] The predicted timestamp is later than the second picture timestamp.

[0332] See also Fig.16 , Fig.16 1 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. The computer device may be the terminal device for running the live broadcast client mentioned above, or may be the server mentioned above, which is not limited here. Fig.16 As shown, the computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, the above-mentioned computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, the user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may optionally also be at least one storage device located away from the aforementioned processor 1001. As Fig.16 As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

[0333] In such Fig.16 In the computer device 1000 shown, the network interface 1004 can provide a network communication function; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to execute the description of the live data processing method in any of the corresponding embodiments above, which will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated.

[0334] In addition, it should be pointed out here that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores the computer program executed by the live data processing device 1 and the live data processing device 2 mentioned above, and the computer program includes computer instructions. When the processor executes the computer instructions, it can execute the description of the live data processing method in the above embodiment, so it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0335] The computer-readable storage medium may be the live data processing device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Further, the computer-readable storage medium may also include both the internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0336] In addition, it should be noted here that: the embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided by any of the corresponding embodiments in the foregoing text. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer program product or computer program embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0337] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0338] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, device, product, or equipment that includes a series of steps or units is not limited to the listed steps or modules, but optionally includes steps or modules that are not listed, or optionally includes other step units inherent to these processes, methods, devices, products, or equipment.

[0339] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0340] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A live broadcast data processing method, characterized in that: The method comprises: Display a video live broadcast interface corresponding to the competitive live broadcast video; the video live broadcast interface includes a player internal area and a player external area; the player internal area is used to play the competitive live broadcast video through a video player; If the business mode of the video live broadcast interface is a content protection mode, first alignment information for data alignment is obtained based on the content protection mode; the first alignment information includes target version information and a first screen timestamp corresponding to the target version information; the target version information is determined based on second scoreboard data corresponding to a target competitive event to be displayed in an area outside the player; the first screen timestamp refers to a timestamp carried by a first screen frame associated with the target competitive event; the first screen frame is used to identify the first scoreboard data to be displayed in an area inside the player in the competitive live broadcast video; When the second scoreboard data is acquired, the second scoreboard data is data-aligned with the first scoreboard data based on the target version information and the first screen timestamp to obtain the second scoreboard data after data alignment; the timestamp corresponding to the second scoreboard data after data alignment and the timestamp corresponding to the first scoreboard data are both the first screen timestamp; When the video player plays the first picture frame, the first scoreboard data is displayed on the first picture frame in the inner area of ​​the player, and the second scoreboard data after the data alignment is displayed in the outer area of ​​the player.

2. The method according to claim 1, characterized in that The video live broadcast interface corresponding to the competitive live broadcast video includes: In response to a video triggering operation on the competitive live video in the live recommendation page, a live stream pull request is sent to a server; the live stream pull request is used to instruct the server to obtain a live video stream corresponding to the competitive live video; The live video stream returned by the server is received, and when the video player is called to parse the live video stream to obtain the competitive live video, a live video interface corresponding to the competitive live video is displayed.

3. The method according to claim 1, characterized in that The video live broadcast interface includes a mode switching control for mode switching; the service mode corresponding to the mode switching control is a non-content protection mode; Before acquiring first alignment information for data alignment based on the content protection mode, the method further includes: In response to a first trigger operation on the mode switching control, the business mode of the live video interface is switched from the non-content protection mode to the content protection mode.

4. The method according to claim 1, characterized in that: When the second scoreboard data is acquired, aligning the second scoreboard data with the first scoreboard data based on the target version information and the first screen timestamp to obtain the aligned second scoreboard data, including: receiving target business data associated with the target competitive event sent by a server, and adding the target business data to a data processing queue; the target business data includes the target version information and the second scoreboard data; Searching for business data matching the target business data in the data processing queue, and if business data matching the target business data is found, acquiring the second scoreboard data from the target business data stored in the data processing queue; Obtaining a competitive event flow list corresponding to the target version information, and obtaining an event occurrence timestamp of the target competitive event from the competitive event flow list; Based on the event occurrence timestamp and the first screen timestamp, the second score board data is aligned with the first score board data on the business timeline corresponding to the live competitive video to obtain the aligned second score board data; the timestamp of the aligned second score board data on the business timeline and the timestamp of the first score board data on the business timeline are both the first screen timestamp.

5. The method according to claim 1, characterized in that The data page card corresponding to the content protection mode is the first data page card; When the video player plays the first picture frame, the first scoreboard data is displayed on the first picture frame in the internal area of ​​the player, and the second scoreboard data after the data alignment is displayed in the external area of ​​the player, including: When the playing progress of the video player reaches the first picture timestamp corresponding to the first picture frame, the first picture frame is displayed in the internal area of ​​the player, and the first scoreboard data is displayed on the first picture frame; When the data page card displayed in the external area of ​​the player is the first data page card, the second scoreboard data after the data whose timestamp is the first picture timestamp is aligned is displayed on the first data page card.

6. The method according to claim 1, characterized in that The target competitive event is a competitive event recorded in the competitive event flow list; The method further comprises: In response to a second trigger operation on the mode switching control, switching the service mode of the live video interface from the content protection mode to a non-content protection mode; When the third score board data associated with the competitive update event is obtained based on the non-content protection mode, the third score board data is displayed on the second data page card corresponding to the external area of ​​the player; the competitive update event is the next competitive event of the target competitive event recorded in the competitive event flow list, the third score board data is sent by the server through the data link, and the second data page card is the data page card corresponding to the non-content protection mode.

7. The method according to claim 6, characterized in that The picture frame associated with the competitive update event is a second picture frame, the second picture frame is used to indicate that the server has identified and obtained fourth scoreboard data to be displayed in the internal area of ​​the player in the competitive live video, and the second picture frame is a picture frame located after the first picture frame in the competitive live video; The method further comprises: When the playing progress of the video player reaches the second picture timestamp corresponding to the second picture frame, the second picture frame is displayed in the internal area of ​​the player, and the fourth scoreboard data is displayed on the second picture frame.

8. The method according to claim 1, characterized in that The method further comprises: If the first screen timestamp corresponding to the first screen frame belongs to the target hot spot moment, when the video player plays the first screen frame, the target barrage data bound to the target hot spot moment is displayed on the first screen frame through the barrage display mode corresponding to the video player.

9. The method according to claim 1, characterized in that: The method further comprises: If the target competitive event is a competitive hot spot event, in response to a viewing trigger operation on a hot spot segment set in the external area of ​​the player, the hot event segment associated with the competitive hot spot event and the segment evaluation value corresponding to the hot event segment are displayed in the hot spot segment set; the hot event segment is a video segment cut from the competitive live video and matching the competitive hot spot event; the segment start timestamp corresponding to the hot event segment is the first screen timestamp in the first alignment information.

10. A live broadcast data processing method, characterized in that: The method comprises: Acquire a live competitive video, perform picture recognition on a first picture frame associated with the target competitive event in the live competitive video, obtain first scoreboard data to be displayed in an internal area of ​​a player, and use a timestamp corresponding to the first picture frame as a first picture timestamp corresponding to the first scoreboard data; Acquire a competitive event flow list associated with the target competitive event, and parse the competitive event flow list to obtain an event occurrence timestamp and an event link timing timestamp corresponding to the target competitive event; If the first screen timestamp matching the event link timing timestamp is found within the adjacent time range of the event occurrence timestamp, the scoreboard data corresponding to the event occurrence timestamp is used as the second scoreboard data to be displayed in the external area of ​​the player, and the target version information for the target competitive event is generated based on the second scoreboard data; the external area of ​​the player and the internal area of ​​the player are both areas in the video live broadcast interface corresponding to the competitive live broadcast video; First alignment information for aligning the second scoreboard data with the first scoreboard data is determined by using the target version information and the first picture timestamp, and the first alignment information is sent to a live broadcast client, wherein the first alignment information is used to instruct the live broadcast client to align the second scoreboard data with the first scoreboard data based on the target version information and the first picture timestamp when the service mode is a content protection mode, so as to obtain the second scoreboard data after data alignment; the timestamp corresponding to the second scoreboard data after data alignment and the timestamp corresponding to the first scoreboard data are both the first picture timestamp, and the first picture timestamp is used to instruct the live broadcast client to display the first scoreboard data on the first picture frame in the internal area of ​​the player and to display the second scoreboard data after data alignment in the external area of ​​the broadcaster when the first picture frame is played through the video player.

11. The method according to claim 10, characterized in that The method of acquiring a live competitive video, performing picture recognition on a first picture frame associated with the target competitive event in the live competitive video, identifying and obtaining first scoreboard data to be displayed in an internal area of ​​a player, and using a timestamp corresponding to the first picture frame as a first picture timestamp corresponding to the first scoreboard data, includes: Receiving a video acquisition data stream sent by a picture acquisition terminal, performing video decoding on the video acquisition data stream, and obtaining competitive live broadcast data associated with the target competitive event; the video acquisition data stream is obtained by the picture acquisition terminal performing picture acquisition on the target competitive event; Performing video encoding on the live broadcast data of the competition to obtain a live broadcast video of the competition carrying multiple picture frames, performing video encapsulation processing on the live broadcast video of the competition to obtain a monitoring video stream for the live broadcast video of the competition; one picture frame corresponds to one encoding timestamp; one encoding timestamp refers to a timestamp written when one picture frame is obtained by encoding; Performing video unpacking processing on the monitoring video stream to obtain the competitive live video, and performing scoreboard recognition on each of the multiple picture frames in the competitive live video through a first recognition service thread to obtain a scoreboard recognition result of each picture frame; If the scoreboard recognition result of each picture frame contains identified scoreboard data associated with the target competitive event, the picture frame corresponding to the identified scoreboard data is used as the first picture frame, and when the scoreboard data identified in the first picture frame is used as the first scoreboard data to be displayed in the internal area of ​​the player, the encoded timestamp carried by the first picture frame is used as the first picture timestamp corresponding to the first scoreboard data.

12. The method according to claim 11, characterized in that Using the encoding timestamp carried by the first picture frame as the first picture timestamp corresponding to the first scoreboard data includes: Establishing a first mapping relationship between the first scoreboard data and the first picture timestamp; Based on the first mapping relationship, the first score board data is used as a value in a key-value pair, and the first screen timestamp is used as a key in the key-value pair. Based on the key-value pair, the timestamp corresponding to the first score board data is determined as the first screen timestamp.

13. The method according to claim 11, characterized in that The method further comprises: When the competitive live video is encapsulated and a monitoring video stream for the competitive live video is obtained, the live video stream for the competitive live video is encapsulated according to the data transmission protocol, and the live video stream is sent to the live client according to the data transmission protocol, so that when the live client parses the competitive live video from the live video stream based on the data transmission protocol to obtain the competitive live video, the competitive live video is played through the video player.

14. The method according to claim 10, characterized in that The step of obtaining a competitive event flow list associated with the target competitive event and parsing the competitive event flow list to obtain an event occurrence timestamp and an event link timing timestamp corresponding to the target competitive event includes: Pulling a competitive event flow list from a data source interface provided by a data acquisition device; the competitive event flow list is determined by the event flow record data recorded by the data acquisition device for the target competitive event; the event flow record data includes an event occurrence timestamp of the target competitive event and a competitive link to which the target competitive event belongs, as well as an event link timing timestamp corresponding to the competitive link, recorded by the data acquisition device; Obtain the event flow record data in the competitive event flow list, perform data conversion on the event flow record data to obtain the changed flow structure data for the target competitive event, and obtain the event occurrence timestamp corresponding to the target competitive event and the event link timing timestamp corresponding to the target competitive event during the competitive link from the changed flow structure data.

15. The method according to claim 11, characterized in that The plurality of frames included in the live competitive video further include a second frame except the first frame, the second frame being a frame in which a competitive update event occurs, the competitive update event being a next competitive event of the target competitive event, and the second frame being a frame subsequent to the first frame; The scoreboard recognition result of each picture frame includes the scoreboard recognition result corresponding to the second picture frame; The method further comprises: If the score board recognition result corresponding to the second screen frame indicates that the score board data associated with the competitive update event is not recognized in the second screen frame, then based on the time difference between the first screen timestamp corresponding to the first screen frame and the event occurrence timestamp corresponding to the target competitive event, the screen timestamp corresponding to the second screen frame is time predicted to obtain a predicted timestamp for the second screen frame, and second alignment information for sending to the live broadcast client is constructed through the predicted timestamp and the updated version information corresponding to the competitive update event.

16. The method according to claim 15, characterized in that The method further comprises: When the prediction timestamp for the second picture frame is obtained by the prediction, a scoreboard is recognized on the second picture frame by a second recognition service thread to obtain an auxiliary recognition result of the second picture frame; the second recognition service thread is a thread used to assist the first recognition service thread in recognizing the scoreboard; If the auxiliary recognition result indicates that the scoreboard data associated with the competition update event is recognized, the recognized scoreboard data is used as the fourth scoreboard data, and the encoding timestamp corresponding to the second picture frame is used as the second picture timestamp corresponding to the fourth scoreboard data; The second picture timestamp is sent to the live broadcast client, so that the live broadcast client updates and replaces the picture frame timestamp corresponding to the second picture frame from the predicted timestamp to the second picture timestamp before playing the second picture frame through the video player; The predicted timestamp is later than the second picture timestamp.

17. A live broadcast data processing device, characterized in that: The device comprises: A live broadcast interface display module, used to display a video live broadcast interface corresponding to the competitive live broadcast video; the video live broadcast interface includes a player internal area and a player external area; the player internal area is used to play the competitive live broadcast video through a video player; An alignment information acquisition module is used for acquiring first alignment information for data alignment based on the content protection mode if the business mode of the video live broadcast interface is the content protection mode; the first alignment information includes target version information and a first screen timestamp corresponding to the target version information; the target version information is determined based on second scoreboard data corresponding to a target competitive event to be displayed in an external area of ​​the player; the first screen timestamp refers to a timestamp carried by a first screen frame associated with the target competitive event; the first screen frame is used to identify the first scoreboard data to be displayed in an internal area of ​​the player in the competitive live broadcast video; a data alignment module, configured to, when acquiring the second scoreboard data, align the second scoreboard data with the first scoreboard data based on the target version information and the first screen timestamp, to obtain the second scoreboard data after data alignment; the timestamp corresponding to the second scoreboard data after data alignment and the timestamp corresponding to the first scoreboard data are both the first screen timestamp; The scoreboard display module is used to display the first scoreboard data on the first frame in the internal area of ​​the player when the video player plays the first frame, and to display the second scoreboard data after the data is aligned in the external area of ​​the player.

18. A live broadcast data processing device, characterized in that: The device comprises: A picture recognition module is used to obtain a live competitive video, perform picture recognition on a first picture frame associated with the target competitive event in the live competitive video, recognize and obtain first scoreboard data to be displayed in an internal area of ​​a player, and use a timestamp corresponding to the first picture frame as a first picture timestamp corresponding to the first scoreboard data; A flow list acquisition module is used to acquire a competitive event flow list associated with the target competitive event, and parse the competitive event flow list to obtain an event occurrence timestamp and an event link timing timestamp corresponding to the target competitive event; A version information generating module, configured to, if the first screen timestamp matching the event link timing timestamp is found within a time range adjacent to the event occurrence timestamp, use the scoreboard data corresponding to the event occurrence timestamp as second scoreboard data to be displayed in an area outside the player, and generate target version information for the target competitive event based on the second scoreboard data; both the area outside the player and the area inside the player are areas in a video live broadcast interface corresponding to the competitive live broadcast video; The alignment information sending module is used to determine the first alignment information used to align the second score board data with the first score board data through the target version information and the first picture timestamp, and send the first alignment information to the live broadcast client, wherein the first alignment information is used to instruct the live broadcast client to align the second score board data with the first score board data based on the target version information and the first picture timestamp when the service mode is the content protection mode, so as to obtain the second score board data after the data alignment; the timestamp corresponding to the second score board data after the data alignment and the timestamp corresponding to the first score board data are both the first picture timestamp, and the first picture timestamp is used to instruct the live broadcast client to display the first score board data on the first picture frame in the internal area of ​​the player and display the second score board data after the data alignment in the external area of ​​the broadcaster when the first picture frame is played through the video player.

19. A computer device, characterized in that: include: Processor and memory; The processor is connected to the memory, wherein the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method according to any one of claims 1 to 16.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 16.

Images