A data processing method, device, computer equipment and storage medium

Abstract

The present disclosure provides a data processing method, apparatus, computer equipment and storage medium, wherein the method is applied to a time calibration server, and comprises: obtaining a first relative time of a video stream and a second relative time of a data stream; the data stream is used to record various events occurring in a video scene corresponding to the video stream; the first relative time and the second relative time are the time when a target activity in the relative video scene starts; the video stream and the data stream come from different data sources; when the difference between the obtained first relative time of the video stream and the second relative time of the data stream is less than or equal to a preset difference, determining a target time difference between a first absolute time of the video stream and a second absolute time of the data stream; and synchronously pushing the data stream and the corresponding target time difference to a user end, so that the user end uses the target time difference to time-align the event content corresponding to the data stream with the video picture corresponding to the video stream.

Description

Technical Field

[0001] The present disclosure relates to the technical field of streaming media data live broadcast, and in particular to a data processing method, device, computer equipment and storage medium. Background Art

[0002] In live broadcast scenarios such as sports events, the video stream corresponding to the video screen of the target activity and the data stream describing the specific events occurring under the target activity are usually pushed, so as to display the video screen using the video stream and display the event content of each event through the data stream.

[0003] Since the video stream and data stream have different sending sources, and the video encoding, network transmission and other processing processes during the push of the video stream will cause push delays, the video stream and data stream display content will be out of sync, which will cause the displayed video images and event content to not conform to the display logic. Summary of the invention

[0004] The embodiments of the present disclosure at least provide a data processing method, apparatus, computer equipment, and storage medium.

[0005] In a first aspect, an embodiment of the present disclosure provides a data processing method, which is applied to a time synchronization server, and the method comprises: obtaining a first relative time of a video stream and a second relative time of a data stream; the data stream is used to record various events occurring in a video scene corresponding to the video stream; the first relative time and the second relative time are the time relative to the start of a target activity in the video scene; the video stream and the data stream come from different data sources; when the difference between the obtained first relative time of the video stream and the second relative time of the data stream is less than or equal to a preset difference, determining a target time difference between the first absolute time of the video stream and the second absolute time of the data stream; and synchronously pushing the data stream and the corresponding target time difference to a user end, so that the user end uses the target time difference to time-align the event content corresponding to the data stream with the video picture corresponding to the video stream.

[0006] In an optional implementation, the first absolute time of the video stream is determined by encoding the source server that provides the video stream; the first relative time of the video stream at any moment is determined in the following manner: determining the video screen corresponding to the video stream at that moment; performing character recognition on the video screen to obtain the first relative time of the video stream at that moment.

[0007] In an optional embodiment, when the difference between the acquired first relative time of the video stream and the second relative time of the data stream is less than or equal to a preset difference, a target time difference between the first absolute time of the video stream and the second absolute time of the data stream is determined, including: based on the first absolute time corresponding to each event in the video stream and the second absolute time corresponding to each event occurring in a historical time period, determining the absolute time difference corresponding to each event; and taking the average of the absolute time differences corresponding to each event as the target time difference.

[0008] In an optional implementation, the average value of the absolute time differences corresponding to the respective events is used as the target time difference, including: in response to the absolute time differences corresponding to the respective events, if there is an absolute time difference that deviates from other absolute time differences by more than a preset time length, the absolute time difference is screened out, and the average value of the updated multiple absolute time differences is determined as the target time difference.

[0009] In a second aspect, the disclosed embodiment also provides another data processing method, which is applied to a user end, and the method includes: receiving a video stream, and receiving a data stream and a target time difference sent by a time calibration server; the data stream is used to record various events occurring in a video scene corresponding to the video stream; the target time difference is the difference between a first absolute time of the video stream and a second absolute time of the data stream; based on the first absolute time of the video stream and the target time difference, during the display of the video picture corresponding to the video stream, time-aligning the event content corresponding to the data stream with the video picture corresponding to the video stream.

[0010] In an optional implementation, based on the first absolute time of the video stream and the target time difference, during the display of the video screen corresponding to the video stream, the event content corresponding to the data stream and the video screen corresponding to the video stream are time-aligned, including: after displaying the video screen of the video stream at the first absolute time, waiting for the target time difference, and displaying the event content corresponding to the data stream in the video screen corresponding to the video stream in an associated manner.

[0011] In an optional embodiment, based on the first absolute time of the video stream and the target time difference, during the display of the video screen corresponding to the video stream, the event content corresponding to the data stream and the video screen corresponding to the video stream are time-aligned, including: after the first absolute time, at preset time intervals, calculating the absolute time difference between the first absolute time and the second absolute time corresponding to the data stream; in response to the first absolute time changing to an absolute time difference between the first absolute time and the second absolute time being greater than or equal to the target time difference, displaying the video screen corresponding to the video stream, and displaying the event content corresponding to the data stream in association with the video screen.

[0012] In the third aspect, the embodiment of the present disclosure also provides a data processing device, including: an acquisition module, used to acquire a first relative time of a video stream, and a second relative time of a data stream; the data stream is used to record various events occurring in a video scene corresponding to the video stream; the first relative time and the second relative time are the time relative to the start of a target activity in the video scene; the video stream and the data stream come from different data sources; a determination module, used to determine a target time difference between a first absolute time of the video stream and a second absolute time of the data stream when the difference between the acquired first relative time of the video stream and the second relative time of the data stream is less than or equal to a preset difference; a push module, used to synchronously push the data stream and the corresponding target time difference to a user end, so that the user end uses the target time difference to time align the event content corresponding to the data stream with the video picture corresponding to the video stream.

[0013] In an optional implementation, the first absolute time of the video stream is determined by encoding the source server that provides the video stream; the first relative time of the video stream at any moment is determined in the following manner: determining the video screen corresponding to the video stream at that moment; performing character recognition on the video screen to obtain the first relative time of the video stream at that moment.

[0014] In an optional embodiment, when the difference between the acquired first relative time of the video stream and the second relative time of the data stream is less than or equal to a preset difference, the determination module, when determining the target time difference between the first absolute time of the video stream and the second absolute time of the data stream, is used to: determine the absolute time difference corresponding to each event respectively based on the first absolute time corresponding to each event in the video stream and the second absolute time corresponding to each event occurring in a historical time period; and take the average of the absolute time differences corresponding to each event respectively as the target time difference.

[0015] In an optional embodiment, when the determination module uses the average value of the absolute time differences corresponding to the respective events as the target time difference, it is used to: in response to the existence of an absolute time difference corresponding to each event that deviates from other absolute time differences by more than a preset time length, filter out the absolute time difference, and use the average value of the updated multiple absolute time differences as the target time difference.

[0016] In a fourth aspect, an embodiment of the present disclosure further provides a data processing device, comprising: a receiving module, used to receive a video stream, and receive a data stream and a target time difference sent by a timing server; the data stream is used to record various events occurring in a video scene corresponding to the video stream; the target time difference is the difference between a first absolute time of the video stream and a second absolute time of the data stream; a time alignment module, used to time-align the event content corresponding to the data stream with the video picture corresponding to the video stream based on the first absolute time of the video stream and the target time difference during the display of the video picture corresponding to the video stream.

[0017] In an optional embodiment, the time alignment module, based on the first absolute time of the video stream and the target time difference, time-aligns the event content corresponding to the data stream with the video picture corresponding to the video stream during the display of the video picture corresponding to the video stream. The module is used to: after displaying the video picture of the video stream at the first absolute time, wait for the target time difference, and associate and display the event content corresponding to the data stream in the video picture corresponding to the video stream.

[0018] In an optional embodiment, the time alignment module, based on the first absolute time of the video stream and the target time difference, time-aligns the event content corresponding to the data stream with the video picture corresponding to the video stream during the display of the video picture corresponding to the video stream, and is used to: calculate the absolute time difference between the first absolute time and the second absolute time corresponding to the data stream at preset time intervals after the first absolute time; in response to the first absolute time changing to an absolute time difference between the first absolute time and the second absolute time being greater than or equal to the target time difference, display the video picture corresponding to the video stream, and display the event content corresponding to the data stream in association with the video picture.

[0019] In a fifth aspect, an optional implementation of the present disclosure further provides a computer device, a processor, and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the machine-readable instructions perform the steps in any possible implementation of the first aspect or the second aspect mentioned above.

[0020] In a sixth aspect, an optional implementation of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed, the steps in any possible implementation of the first aspect or the second aspect described above are executed.

[0021] The data processing method, device, computer equipment and storage medium provided by the embodiments of the present disclosure can determine the target time difference between the absolute time of the video stream and data stream with time difference sent by different data sources at the same time, while ensuring that the relative time difference between the two is small, so as to time-align the event content corresponding to the data stream and the video screen corresponding to the video stream through the target time difference. After the time alignment between the two is completed in absolute time, the event content played on the video screen will not be earlier than the time when the specific event is reflected on the video screen, so it is more in line with the display logic when displaying the video screen and the event content.

[0022] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0024] Figure 1 A flow chart of a data processing method provided by an embodiment of the present disclosure is shown;

[0025] Figure 2 A flowchart showing another data processing method provided by an embodiment of the present disclosure is shown;

[0026] Figure 3A flowchart of interaction between multiple servers in live broadcasting provided by an embodiment of the present disclosure is shown;

[0027] Figure 4 A schematic diagram of a data processing device provided by an embodiment of the present disclosure is shown;

[0028] Figure 5 A schematic diagram showing another data processing device provided by an embodiment of the present disclosure;

[0029] Figure 6 A schematic diagram of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the present disclosure claimed for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0031] Research has found that when live broadcasts of sports events are broadcast, the video images of the target activity and the event content of the specific event will be pushed through the video stream and data stream. However, since the data sources of the video stream are different during the push process and the video stream goes through multiple processing steps, the video stream and the data stream will be out of sync, causing the video image to be delayed. In this case, when the screen is displayed through the video stream and data stream, the event content will be displayed first, and then the video screen can display the specific screen of the event, which does not conform to the display logic.

[0032] Based on the above research, the present disclosure provides a data processing method. For video streams and data streams sent by different data sources and with time differences, the target time difference between the absolute times of the two can be determined while ensuring that the relative time difference between the two is small at the same time, so as to time-align the event content corresponding to the data stream and the video screen corresponding to the video stream through the target time difference. After completing the time alignment between the two in absolute time, the event content played on the video screen will not be earlier than the time when the specific event is reflected on the video screen, and therefore it is more in line with the display logic when displaying the video screen and the event content.

[0033] The defects existing in the above solutions are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the present disclosure for the above problems below should be the contributions made by the inventor to the present disclosure during the disclosure process.

[0034] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0035] To facilitate understanding of the present embodiment, a data processing method disclosed in the present embodiment is first introduced in detail. The execution subject of the data processing method provided in the present embodiment is generally a computer device with certain computing capabilities, and the computer device includes, for example: a terminal device or a server or other processing device, and the terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the data processing method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0036] First, the application scenario of the data processing method provided by the embodiment of the present disclosure is described. The data processing method provided by the embodiment of the present disclosure can be applied to live broadcast scenarios, and specifically applied to scenarios that include specific activity pictures and events in the live broadcast scenario, such as live broadcast of sports events, live broadcast of e-sports, etc. Live broadcast of sports events includes but is not limited to live broadcast of football matches, live broadcast of basketball matches, and live broadcast of boxing matches. Taking the live broadcast of football matches as an example, the specific activity pictures may include the game pictures obtained by photographing the stadium, and the events may specifically include penalty kicks, corner kicks, and referee penalties and other events that may occur in the game. The details are described in detail below and will not be repeated here.

[0037] Below, first, the data processing method provided by the embodiment of the present disclosure is explained by taking the execution subject as an example of a time calibration server. The time calibration server is different from the user end and the other end side outside the two source servers that send the video stream and the data stream. Specifically, it can receive the relative time and absolute time corresponding to the video stream and the data stream respectively sent by two different servers to determine the target time difference between the video stream and the data stream in absolute time. The target time difference can be used to time-align the event content corresponding to the data stream and the video picture corresponding to the video stream. After determining the target time difference, the time calibration server can synchronously push the target time difference to the user end, so that the user end uses the target time difference to complete the time alignment, thereby displaying the video picture and event content that conforms to the display logic.

[0038] See also Figure 1 FIG. 1 is a flowchart of a data processing method provided by an embodiment of the present disclosure, wherein the method includes steps S101 to S103, wherein:

[0039] S101: Obtain a first relative time of a video stream and a second relative time of a data stream; the data stream is used to record various events occurring in a video scene corresponding to the video stream; the first relative time and the second relative time are times relative to the start of a target activity in the video scene; the video stream and the data stream come from different data sources;

[0040] S102: When the difference between the first relative time of the video stream and the second relative time of the data stream obtained is less than or equal to a preset difference, determine a target time difference between the first absolute time of the video stream and the second absolute time of the data stream;

[0041] S103: Synchronously push the data stream and the corresponding target time difference to the user end, so that the user end uses the time difference to time-align the event content corresponding to the data stream with the video picture corresponding to the video stream.

[0042] The above S101 to S103 are described in detail below.

[0043] With respect to the above S101, the video stream and the data stream are first described. In a live broadcast scenario, the video scene can be filmed by video recording to record the target activity occurring therein. The target activity here can be the sports event or e-sports event described above.

[0044] When obtaining a video stream, for example, in a football match, a cameraman can hold a camera to shoot the playing field. Here, the game scene is also a video scene, and the video obtained by shooting can be used to obtain a video stream. In the video scene, events may occur, such as penalty kicks, corner kicks, penalties, etc. in football matches, or upgrades of virtual characters controlled by players in e-sports events. In order to guide users to pay attention to these events occurring in the video scene, the corresponding data stream can be determined to record and push these events. In one possible case, for sports events such as football matches, whether an event occurs is generally determined manually, so the relevant referee or recorder can record the event and generate a data stream. In another possible case, for e-sports events, whether a specific event occurs can be directly determined by collecting relevant data of the player controlling the virtual character, so it can also be directly generated.

[0045] For video streams and data streams, time information can be obtained in two time dimensions: relative time and absolute time. The difference between relative time and absolute time is reflected in the difference in reference time; specifically, the reference time of relative time is the start time of the target activity, such as the start time of the game. The reference time of absolute time can be a universal standard time, such as Beijing time.

[0046] For example, taking a football match as an example, if it is held at 1:00 pm on January 1, 2000 Beijing time, the data corresponding to Beijing time "2000-01-01 13:00:00" can be obtained. This moment is also the reference time of the relative time. Taking the match for 10 minutes as an example, it can be determined that after 10 minutes, the absolute time changes to "2000-01-01 13:10:00" corresponding to Beijing time, and the relative time changes to "10:00". Specifically, when determining the relative time and the absolute time, the accuracy can be to seconds or milliseconds.

[0047] For the sake of distinction, the relative time corresponding to the video stream is referred to as the first relative time in this application, and the relative time of the data stream is referred to as the second relative time in this application.

[0048] When obtaining the relative time of the video stream, in general, the event organizer will take the lead in shooting the event video and obtain the video stream, and then send the video stream to the servers corresponding to the different platforms that broadcast the event video. During the shooting process, the time can be synchronously marked on each frame of the video stream. For example, in the above example, if the game has been going on for 10 minutes, then at the moment of 10 minutes of the game, the corresponding video screen will be marked with "10:00" or other marking information expressing that the current video screen is the moment of 10 minutes of the game. Therefore, when confirming the first relative time of the video stream at any moment, the following method can be specifically adopted: determine the video screen corresponding to the video stream at that moment; perform character recognition on the video screen to obtain the first relative time of the video stream at that moment.

[0049] In one possible case, since the numbers appearing in the video screen may not only include numbers reflecting relative time, such as numbers displayed on billboards, scoreboards, etc., in order to avoid misrecognition, the focus position when performing character recognition can be determined in advance according to the fixed position where the relative time appears in the video screen, such as determining the recognition frame. When performing character recognition, optical character recognition (OCR) can be used to recognize the image area under the recognition frame to obtain the first relative time of the video screen corresponding to that moment.

[0050] When obtaining the second relative time of the data stream, in one possible case, the data stream recording the event is manually confirmed and generated, so the data stream itself can carry the relative time, and the obtained data stream can be directly parsed.

[0051] In this way, the time synchronization server can obtain the first relative time of the video stream and the second relative time of the data stream.

[0052] In order to time align the video stream and the data stream in S102, it is also necessary to determine the difference between the two based on a time. In this application, the relative time corresponding to the video stream and the data stream is used as a reference, and the time alignment is completed through the difference in absolute time between the two.

[0053] In a specific implementation, when the first relative time of the video stream and the second relative time of the data stream are obtained according to the above steps, it can be determined whether the difference between the first relative time of the video stream and the second relative time of the data stream obtained at the same time is less than or equal to the preset difference. Here, the preset difference is generally set to be small, such as 1 second or 2 seconds. Since the video stream has a certain delay when it is obtained compared to the data stream, it cannot be completely guaranteed that the first relative time of the video stream obtained at the same time can be completely consistent with the second relative time of the data stream. Therefore, it can be selected to reduce the restrictions on the judgment of the two relative times by using the preset difference.

[0054] In one possible case, if the difference between the determined first relative time and the second relative time is greater than a preset difference, and the data stream and the video stream cannot be "aligned" in relative time under the preset difference, for example, after several seconds of the actual event broadcast, it is still not detected that the time difference corresponding to the data stream and the video stream is less than or equal to the preset difference, then an error message can be generated, etc., so as to update the corresponding relative time of the data stream and / or video stream by re-establishing a connection with the platform server, etc., so as to normally complete the "alignment" operation of the data stream and the video stream in relative time.

[0055] When it is determined that the difference between the first relative time of the video stream and the second relative time of the data stream obtained at the same time is less than or equal to the preset difference, the target time difference between the absolute time corresponding to the video stream and the absolute time corresponding to the data stream can be determined. Here, for the sake of distinction, the absolute time corresponding to the video stream is referred to as the first absolute time in this application, and the absolute time of the data stream is referred to as the second absolute time in this application. The target time difference obtained is the time difference used to time-align the event content corresponding to the data stream with the video picture corresponding to the video stream.

[0056] Among them, the first absolute time corresponding to the video stream can be determined by encoding the source server that provides the video stream. The source server described here can specifically be the server of the event organizer described above. After the source server collects the captured video, it can directly encode the current time (such as a unix timestamp) to obtain the first absolute time. Exemplarily, the first absolute time can be represented by Abs_SEI_t, and when it corresponds to Beijing time "2022-10-01 10:00:00", its value is specifically encoded as 1664589600.

[0057] The second absolute time corresponding to the data stream can be directly parsed from the data stream in a manner similar to the above-mentioned method of determining the second relative time.

[0058] In a specific implementation, when determining the difference between the first absolute time of the video stream and the second absolute time of the data stream, the absolute time difference corresponding to each event can be determined based on the first absolute time corresponding to each event in the video stream and the second absolute time corresponding to each event occurring in a historical time period. The average value of the absolute time differences corresponding to each event is used as the target time difference.

[0059] Here, the historical time period may refer to the time period from the start of the game to the current time, such as the 20 minutes from the start of the game to the 20th minute of the current game. Alternatively, it may be a historical time period determined based on the current time and a preset time length as the length of the retrospective time, such as when the current game is at the 90th minute, the preset time length is selected as 30 minutes, and the historical time period is determined to be the 30 minutes from the 60th minute to the 90th minute of the game. Alternatively, it may represent the time elapsed under multiple recent events, such as when the current time is the 40th minute of the game, and it is determined that the multiple recent events are 3, which occurred at the 30th minute, the 33rd minute, and the 35th minute of the game, respectively, then the historical time period may be determined to be 5 minutes or 10 minutes from the 30th minute to the 35th minute or to the 40th minute.

[0060] In the specific implementation, the selection of the historical time period can be determined according to the actual situation. For example, for basketball games, events specifically include goals, which occur frequently. If the difference calculation and average calculation of the video stream and data stream in absolute time are performed for all the events that have occurred during the game, the amount of calculation will be large and the time consumption will be correspondingly long. Therefore, it is possible to select the historical time period to be set to the time elapsed under multiple events that have occurred recently, so as to minimize the consumption of computing resources and reduce the time consumption while ensuring that the target time difference can be accurately obtained. For football games, the number of events such as goals and substitute appearances in the 90 minutes of the game is very limited. Therefore, when selecting the historical time period, the time period from the start of the game to the current time can be selected as the historical time period to ensure that the number of events that have occurred is sufficient, so as to facilitate more accurate determination of the target time difference. For details, please refer to the following related instructions.

[0061] Specifically, the time-correction server can continuously parse the data corresponding to the events that occurred from the data stream after the game starts, so as to obtain the second absolute time corresponding to each event that occurred in the determined historical time period in the data stream, and can also determine the first absolute time of each event that was selected from the video stream accordingly. The absolute time difference between the first absolute time and the second absolute time determined under different events may be different, such as having a fluctuating change. Therefore, when determining the target time difference, the average value of the absolute time differences corresponding to each event can be determined as the target time difference.

[0062] Here, since the absolute time difference obtained from the absolute times corresponding to the video stream and the data stream when the video stream and the data stream are pushed normally will not produce a large range of fluctuations, or show a phenomenon of gradually tending to be stable, therefore, for the scenario corresponding to the football match listed in the above example, the more events that can be determined to have occurred, the more accurate the result obtained when calculating the target time difference. That is, the more events pushed during push, the stronger the fault tolerance, and the more accurate the target time difference obtained for time calibration.

[0063] In one possible case, when calculating the absolute time difference between the first absolute time and the second absolute time, there may be a situation where the absolute time difference determined in a certain event deviates greatly from the absolute time difference determined in other events. For example, taking a football match as an example, when playing a video through a video stream, a video playback scene may appear, such as the playback after a goal, which will be played back in time to a position earlier in the video stream, thus causing the relative time to regress, and then when the relative times corresponding to the video stream and the data stream are close, the absolute time difference between the two deviates greatly from the absolute time difference at other times.

[0064] In order to avoid the influence of such scenarios on the determination of the target time difference, when determining the target time difference by the average value of the absolute time differences corresponding to each event, the following method can be adopted: in response to the absolute time difference corresponding to each event, if there is an absolute time difference that deviates from other absolute time differences by more than a preset time length, the absolute time difference is screened out, and the average value of the updated multiple absolute time differences is used as the target time difference.

[0065] Here, the absolute time difference that deviates from other absolute time differences by more than the preset time length corresponds to an event such as the video playback event listed above. Since the numerical influence of the absolute time difference under these events is not the cause of the time asynchrony between the video stream and the data stream, when calculating the average value of the absolute time difference under each event to obtain the target time difference, the absolute time difference is selected to be screened out and does not participate in the calculation of the target time difference.

[0066] In this way, the timing server can determine the target time difference for time alignment.

[0067] With respect to the above S103, the specific task of the time calibration server is to determine the target time difference for time alignment, and the time alignment using the target time difference to complete the display of the video screen and event content can be handled by the user end. Therefore, after the time calibration server determines the target time difference, the target time difference can be synchronously pushed to the user end.

[0068] In addition, in a possible case, the server sending the data stream does not interact with the user end, so the time synchronization server can also undertake the task of forwarding the event content. For example, taking a football match as an example, the event content can specifically include the event type and the score, such as the event type is "goal" and the score is "home team score -3; away team score 2".

[0069] Next, another data processing method provided by the embodiment of the present disclosure is described with the user end that interacts with the time synchronization server as the execution subject. The user end is the end side that specifically watches the event, corresponding to the user watching the game. After the user end receives the video stream and the data stream, the video screen and event content can be displayed to the user.

[0070] See also Figure 2 FIG. 2 is a flowchart of another data processing method provided by an embodiment of the present disclosure, wherein the method includes steps S201 to S202, wherein:

[0071] S201: Receive a video stream, and receive a data stream and a target time difference sent by a time calibration server; the data stream is used to record various events occurring in a video scene corresponding to the video stream; the target time difference is a difference between a first absolute time of the video stream and a second absolute time of the data stream;

[0072] S202: Based on the first absolute time of the video stream and the target time difference, during display of the video frame corresponding to the video stream, time-align the event content corresponding to the data stream with the video frame corresponding to the video stream.

[0073] Regarding the above S201, in a possible case, the source server corresponding to the event organizer described above does not directly transmit data with the time synchronization server. For example, for large-scale events, different broadcast platforms have their own time synchronization servers. Therefore, the source server will first push the video stream to the platform server (or called the push stream server) corresponding to the different broadcast platforms, and then the time synchronization server will pull the video stream to obtain it. Therefore, for the user end, it can receive the video stream provided by the platform server.

[0074] As for the data stream and the target time difference, corresponding to the relevant description of the above step S103, when the time calibration server determines the target time difference, the data stream and the target time difference can be synchronously sent to the user end.

[0075] With respect to the above S202, in order to enable the user to synchronously display the event information of the event after the picture corresponding to the specific event is played in the video screen when watching the video screen, that is, to update the score at the moment of video and data calibration, in the embodiment of the present disclosure, the event content corresponding to the data stream and the video screen corresponding to the video stream can be time-aligned based on the first absolute time of the video stream and the target time difference.

[0076] Here, the user end can obtain the first absolute time corresponding to the video stream sent synchronously with the video stream through the streaming server, or can also receive the first absolute time forwarded by the time calibration server. The method for the user end to obtain the first absolute time is not limited here.

[0077] The following describes two time alignment methods provided in this application.

[0078] (a) Time alignment is performed by periodic waiting.

[0079] In this case, after displaying the video frame of the video stream at the first absolute time, the target time difference may be waited for, and the event content corresponding to the data stream may be displayed in association with the video frame corresponding to the video stream.

[0080] Specifically, since the target time difference represents the absolute time delay of the video stream compared to the data stream when relative time is aligned, the target time difference can be selected as the waiting time, so that after playing the video picture corresponding to the video stream at the first absolute time, first wait for the target time difference and then play the event content corresponding to the data stream.

[0081] For example, if the target time difference is 5 seconds, if the absolute time corresponding to the video stream and the data stream is aligned for playback, the following scenario may occur: when the data stream is used to show that a goal has occurred, the video screen displayed under the video stream is still a frame of video screen S1 corresponding to 5 seconds before the goal event occurs, such as the player is pushing the ball from the midfield to the goal. In this case, the target time difference of 5 seconds can be set as the waiting time. When the data stream represents that a goal event has occurred, the video screen S1 is being played, and the event content of the goal event is retained and not displayed. After the target time difference of 5 seconds has passed, the video screen displayed under the video stream is updated to S2, that is, the screen expressing the event of the player scoring a goal. At this time, the event content of the goal is associated with the video screen S2 for display, so that the event content and the video screen are logical when displayed.

[0082] (b) Time alignment through real-time monitoring.

[0083] In this case, the absolute time difference between the first absolute time and the second absolute time corresponding to the data stream can be calculated at preset time intervals after the first absolute time; in response to the absolute time difference between the first absolute time and the second absolute time being greater than or equal to the target time difference, the video screen corresponding to the video stream is displayed, and the event content corresponding to the data stream is displayed in association with the video screen.

[0084] Here, when the user terminal determines the second absolute time corresponding to the data stream, it can directly pull the data stream from the server providing the data stream and decode the data stream, or it can also obtain it from the time calibration server, and there is no specific limitation. The preset duration can be determined according to actual conditions. For example, when both the absolute time and the relative time are in seconds as the minimum unit, the preset duration is set to 1 second, or it is set to 0.5 seconds according to the specific requirements of time accuracy.

[0085] Exemplarily, the first absolute time is expressed as Abs_SEI_t, the second absolute time is expressed as incident_SEI_t, and the target time difference is expressed as diff, then it can be determined whether the following inequality holds:

[0086] Abs_SEI_t-incident_SEI_t≥diff

[0087] If it is established, that is, the absolute time difference between the first absolute time and the second absolute time is greater than or equal to the target time difference, then when the video screen and the event content are played, the event content can be delayed to at least be displayed when the video screen displays the event. Correspondingly, if the above inequality does not hold, after a preset time interval, the changed first absolute time Abs_SEI_t' is obtained, and the above first absolute time Abs_SEI_t is updated and replaced, and the inequality is continued to be judged whether it holds until the first absolute time changes to satisfy the above inequality.

[0088] Here, the inequality described above is determined only under the understanding that the difference between the first absolute time and the second absolute time should be changed to exceed the target time difference. It can also be transformed into other forms, such as expressing that the first absolute time of the video stream should be delayed compared to the second absolute time of the data stream, and the delay should not be less than the target time difference, for example, transformed into the following inequality:

[0089] Abs_SEI_t-diff≥incident_SEI_t

[0090] Under this inequality, it can be determined whether the absolute time difference between the first absolute time and the target time is greater than or equal to the target time difference, and when it is determined that the inequality is satisfied, the video screen corresponding to the video stream is displayed, and the event content corresponding to the data stream is displayed on the video screen.

[0091] In this way, through any of the above methods, the time alignment of the video stream and the data stream can be achieved, thereby preventing the situation where the event content is displayed first and then the video screen can display the specific screen of the event, thereby ensuring the user's viewing experience.

[0092] In the embodiment of the present disclosure, a specific embodiment of using data stream and video stream to allow users to watch live broadcast is also provided. Figure 3 As shown, it is a flowchart of interaction between multiple servers in live broadcast provided by an embodiment of the present disclosure. Among them, multiple servers specifically include a source server 31, a data server 32, a platform server 33, a time server 34 and a user terminal 35. Here, the server that generates and sends the data source is called the data server 32. The data transmission between servers is represented by directional arrows. The following is an explanation of a specific embodiment using a football match as an example.

[0093] The source server 31 is, for example, a server of a competition organizer, which specifically obtains a video stream synchronously when filming the competition and encodes the first absolute time in the video stream. After obtaining the video stream, the video stream is pushed to the platform server 33, and the time calibration server 34 pulls the stream from the platform server 33, so that the time calibration server obtains the first relative time of the video stream by character recognition.

[0094] The data server 32 specifically sends the obtained data stream to the time calibration server 34, and the time calibration server 34 can directly parse and obtain the second absolute time and the second relative time in the data stream. In addition, the time calibration server 34 can also obtain event content including event type and score from the data stream, and synchronously send the event content to the time calibration server 34.

[0095] After receiving the video stream sent by the platform server 33, the time-correction server 34 can parse the first absolute time of the obtained video stream, and obtain the first relative time by character recognition. In addition, after receiving the data stream sent by the data server 32, the second absolute time and the second relative time of the data stream can be obtained. Using the relative time and absolute time corresponding to the video stream and the data stream, respectively, the time-correction server 34 can process and obtain the target time difference, so as to time-align the event content corresponding to the data stream with the video screen corresponding to the video stream. The time-correction server 34 can push the target time difference to the user terminal 35, and can also synchronously send the event content sent by the data server 32 to the user terminal 35.

[0096] The user terminal 35 can pull the video stream from the platform server 33, and receive the target time difference sent by the time calibration server 34. The event content corresponding to the data stream can be time-aligned with the video screen corresponding to the video stream by decoding the first absolute time obtained from the video stream pulled, and the target time difference. When the time alignment is completed, the video screen and the event content under the video stream can be associated and played using the pulled video stream and the event content forwarded from the time calibration server 34, so that the user can watch the video screen containing the event content after calibration, and effectively prevent "spoilers", ensuring the user's live viewing experience.

[0097] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0098] Based on the same inventive concept, a data processing device corresponding to the above-mentioned first data processing method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned first data processing method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the above-mentioned first data processing method, and the repeated parts will not be repeated.

[0099] Reference Figure 4 FIG. 4 is a schematic diagram of a data processing device provided by an embodiment of the present disclosure, wherein the device comprises: an acquisition module 41, a determination module 42, and a push module 43; wherein:

[0100] The acquisition module 41 is used to acquire a first relative time of a video stream and a second relative time of a data stream; the data stream is used to record various events occurring in a video scene corresponding to the video stream; the first relative time and the second relative time are relative to the start time of a target activity in the video scene; the video stream and the data stream come from different data sources;

[0101] A determination module 42, configured to determine a target time difference between a first absolute time of the video stream and a second absolute time of the data stream when the difference between the acquired first relative time of the video stream and the acquired second relative time of the data stream is less than or equal to a preset difference;

[0102] The push module 43 is used to push the data stream and the corresponding target time difference to the user end synchronously, so that the user end can use the target time difference to time align the event content corresponding to the data stream with the video picture corresponding to the video stream.

[0103] In an optional implementation, the first absolute time of the video stream is determined by encoding the source server that provides the video stream; the first relative time of the video stream at any moment is determined in the following manner: determining the video screen corresponding to the video stream at that moment; performing character recognition on the video screen to obtain the first relative time of the video stream at that moment.

[0104] In an optional embodiment, when the difference between the acquired first relative time of the video stream and the second relative time of the data stream is less than or equal to a preset difference, the determination module 42, when determining the target time difference between the first absolute time of the video stream and the second absolute time of the data stream, is used to: determine the absolute time difference corresponding to each event respectively based on the first absolute time corresponding to each event in the video stream and the second absolute time corresponding to each event occurring in a historical time period; and take the average of the absolute time differences corresponding to each event respectively as the target time difference.

[0105] In an optional embodiment, when the determination module 42 uses the average value of the absolute time differences corresponding to the respective events as the target time difference, it is used to: in response to the existence of an absolute time difference corresponding to each event that deviates from other absolute time differences by more than a preset time length, filter out the absolute time difference, and use the average value of the updated multiple absolute time differences as the target time difference.

[0106] In addition, another data processing device corresponding to the above-mentioned second data processing method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned second data processing method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the above-mentioned second data processing method, and the repeated parts will not be repeated.

[0107] Reference Figure 5 FIG. 1 is a schematic diagram of a data processing device provided by an embodiment of the present disclosure, wherein the device comprises: a receiving module 51 and a time alignment module 52; wherein:

[0108] The receiving module 51 is used to receive the video stream, and receive the data stream and the target time difference sent by the time calibration server; the data stream is used to record various events occurring in the video scene corresponding to the video stream; the target time difference is the difference between the first absolute time of the video stream and the second absolute time of the data stream;

[0109] The time alignment module 52 is used to time align the event content corresponding to the data stream with the video frame corresponding to the video stream based on the first absolute time of the video stream and the target time difference during the display of the video frame corresponding to the video stream.

[0110] In an optional embodiment, the time alignment module 52, based on the first absolute time of the video stream and the target time difference, is used to time align the event content corresponding to the data stream with the video picture corresponding to the video stream during the display of the video picture corresponding to the video stream: after displaying the video picture of the video stream at the first absolute time, wait for the target time difference, and associate and display the event content corresponding to the data stream in the video picture corresponding to the video stream.

[0111] In an optional embodiment, the time alignment module 52, when time aligning the event content corresponding to the data stream with the video picture corresponding to the video stream based on the first absolute time of the video stream and the target time difference, during the display of the video picture corresponding to the video stream, is used to: calculate the absolute time difference between the first absolute time and the second absolute time corresponding to the data stream at preset time intervals after the first absolute time; in response to the first absolute time changing to an absolute time difference between the first absolute time and the second absolute time being greater than or equal to the target time difference, display the video picture corresponding to the video stream, and display the event content corresponding to the data stream in association with the video picture.

[0112] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0113] The present disclosure also provides a computer device, such as Figure 6 FIG. 1 is a schematic diagram of a computer device structure provided in an embodiment of the present disclosure, including:

[0114] A processor 10 and a memory 20; the memory 20 stores machine-readable instructions executable by the processor 10, and the processor 10 is used to execute the machine-readable instructions stored in the memory 20. When the machine-readable instructions are executed by the processor 10, the processor 10 performs the following steps:

[0115] Obtain a first relative time of a video stream and a second relative time of a data stream; the data stream is used to record various events occurring in a video scene corresponding to the video stream; the first relative time and the second relative time are relative to the start time of a target activity in the video scene; the video stream and the data stream come from different data sources; when the difference between the first relative time of the video stream and the second relative time of the data stream obtained is less than or equal to a preset difference, determine a target time difference between the first absolute time of the video stream and the second absolute time of the data stream; synchronously push the data stream and the corresponding target time difference to a user end, so that the user end uses the target time difference to time-align the event content corresponding to the data stream with the video picture corresponding to the video stream.

[0116] Alternatively, the processor 10 executes the following steps:

[0117] Receive a video stream, and receive a data stream and a target time difference sent by a time calibration server; the data stream is used to record various events occurring in a video scene corresponding to the video stream; the target time difference is the difference between a first absolute time of the video stream and a second absolute time of the data stream; based on the first absolute time of the video stream and the target time difference, during display of a video picture corresponding to the video stream, time align the event content corresponding to the data stream with the video picture corresponding to the video stream.

[0118] The above-mentioned memory 20 includes an internal memory 210 and an external memory 220; the internal memory 210 here is also called an internal memory, which is used to temporarily store the calculation data in the processor 10, as well as the data exchanged with the external memory 220 such as a hard disk. The processor 10 exchanges data with the external memory 220 through the internal memory 210.

[0119] The specific execution process of the above instructions can refer to the steps of the data processing method described in the embodiment of the present disclosure, which will not be repeated here.

[0120] The present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the data processing method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0121] The embodiments of the present disclosure also provide a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the data processing method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0122] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0123] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0124] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0125] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0126] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0127] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A data processing method, characterized in that: Applied to a time synchronization server, the method comprises: Obtaining a first relative time of a video stream and a second relative time of a data stream; the data stream is used to record various events occurring in a video scene corresponding to the video stream; the first relative time and the second relative time are times relative to the start of a target activity in the video scene; the video stream and the data stream come from different data sources; When the difference between the first relative time of the video stream and the second relative time of the data stream obtained is less than or equal to a preset difference, determining a target time difference between the first absolute time of the video stream and the second absolute time of the data stream; Synchronously push the data stream and the corresponding target time difference to the user end, so that the user end time-aligns the event content corresponding to the data stream with the video picture corresponding to the video stream according to the target time difference; The first relative time is the time marked on the video image of the video stream during the camera shooting process, and the first absolute time is the current system time of the source server when the source server collects the video image shot by the camera; The second relative time and the second absolute time of the data stream are obtained by parsing the data stream; Wherein, when the difference between the acquired first relative time of the video stream and the second relative time of the data stream is less than or equal to a preset difference, determining a target time difference between the first absolute time of the video stream and the second absolute time of the data stream includes: Determine the absolute time difference corresponding to each event respectively based on a first absolute time corresponding to each event in the video stream and a second absolute time corresponding to each event in the data stream; The average value of the absolute time differences corresponding to the events is taken as the target time difference.

2. The method according to claim 1, characterized in that The first absolute time of the video stream is determined by encoding the source server providing the video stream; the first relative time of the video stream at any time is determined in the following manner: Determine the video picture corresponding to the video stream at this moment; Character recognition is performed on the video image to obtain the first relative time of the video stream at that moment.

3. The method according to claim 1, characterized in that The average value of the absolute time differences corresponding to the events is used as the target time difference, including: In response to the absolute time difference corresponding to each event, if there is an absolute time difference that deviates from other absolute time differences by more than a preset time length, the absolute time difference is filtered out, and an average value of the updated multiple absolute time differences is determined as the target time difference.

4. A data processing method, characterized in that: Applied to a user terminal, the method includes: Receive a video stream, and receive a data stream and a target time difference sent by a time calibration server; the data stream is used to record various events occurring in a video scene corresponding to the video stream; the target time difference is the difference between a first absolute time of the video stream and a second absolute time of the data stream when the difference between a first relative time of the video stream and a second relative time of the data stream is less than or equal to a preset difference; Based on the first absolute time of the video stream and the target time difference, during display of the video picture corresponding to the video stream, time-align the event content corresponding to the data stream with the video picture corresponding to the video stream; The first relative time is the time marked on the video image of the video stream during the camera shooting process, and the first absolute time is the current system time of the source server when the source server collects the video image shot by the camera; The second relative time and the second absolute time of the data stream are obtained by parsing the data stream; The target time difference is the average of the absolute time differences between the first absolute time corresponding to each event occurring in the historical time period in the video stream and the second absolute time corresponding to each event in the data stream.

5. The method according to claim 4, characterized in that The step of performing time alignment between the event content corresponding to the data stream and the video picture corresponding to the video stream during display of the video picture corresponding to the video stream based on the first absolute time of the video stream and the target time difference comprises: After displaying the video frame of the video stream at the first absolute time, wait for the target time difference, and display the event content corresponding to the data stream in association with the video frame corresponding to the video stream.

6. The method according to claim 4, characterized in that The step of performing time alignment between the event content corresponding to the data stream and the video picture corresponding to the video stream during display of the video picture corresponding to the video stream based on the first absolute time of the video stream and the target time difference comprises: After the first absolute time, at a preset time interval, calculating an absolute time difference between the first absolute time and a second absolute time corresponding to the data stream; In response to the absolute time difference between the first absolute time and the second absolute time being greater than or equal to the target time difference, a video screen corresponding to the video stream is displayed, and event content corresponding to the data stream is displayed in association with the video screen.

7. A data processing device, characterized in that: Applied to a time synchronization server, the device comprises: An acquisition module, used for acquiring a first relative time of a video stream and a second relative time of a data stream; the data stream is used for recording various events occurring in a video scene corresponding to the video stream; the first relative time and the second relative time are times relative to the start of a target activity in the video scene; the video stream and the data stream are from different data sources; A determination module, configured to determine a target time difference between a first absolute time of the video stream and a second absolute time of the data stream when the difference between the acquired first relative time of the video stream and the acquired second relative time of the data stream is less than or equal to a preset difference; A push module, used to push the data stream and the corresponding target time difference to the user end synchronously, so that the user end uses the target time difference to time-align the event content corresponding to the data stream with the video picture corresponding to the video stream; The first relative time is the time marked on the video image of the video stream during the camera shooting process, and the first absolute time is the current system time of the source server when the source server collects the video image shot by the camera; The second relative time and the second absolute time of the data stream are obtained by parsing the data stream; Among them, the determination module is further configured to: determine the absolute time difference corresponding to each event based on the first absolute time corresponding to each event occurring in the historical time period in the video stream and the second absolute time corresponding to each event in the data stream; and take the average value of the absolute time differences corresponding to each event as the target time difference.

8. A data processing device, characterized in that: Applied to a user terminal, the device comprises: A receiving module, used for receiving a video stream, and receiving a data stream and a target time difference sent by a time calibration server; the data stream is used for recording various events occurring in a video scene corresponding to the video stream; the target time difference is the difference between a first absolute time of the video stream and a second absolute time of the data stream when the difference between a first relative time of the video stream and a second relative time of the data stream is less than or equal to a preset difference; A time alignment module, configured to time-align the event content corresponding to the data stream with the video picture corresponding to the video stream during display of the video picture corresponding to the video stream based on the first absolute time of the video stream and the target time difference; The first relative time is the time marked on the video image of the video stream during the camera shooting process, and the first absolute time is the current system time of the source server when the source server collects the video image shot by the camera; The second relative time and the second absolute time of the data stream are obtained by parsing the data stream; The target time difference is the average of the absolute time differences between the first absolute time corresponding to each event occurring in the historical time period in the video stream and the second absolute time corresponding to each event in the data stream.

9. A computer device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory. When the machine-readable instructions are executed by the processor, the processor performs the steps of the data processing method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program. When the computer program is executed by a computer device, the computer device executes the steps of the data processing method according to any one of claims 1 to 6.

Images