A football match data processing system and method based on big data

Abstract

The present application provides a football match data processing system and method based on big data. The system sets a data resource management pool based on historical processing task data; extracts the downward capture response amount and the upward transmission response amount of the current football match data acquisition terminal through transmission response data, thereby obtaining the current data backlog loss; determines the capture time span and cache time span of the current football match big data, and determines the data freshness of the data cache area based on the capture time span and cache time span; determines the resource adaptation coefficient of the current football match big data based on the data backlog loss and data freshness, and performs corresponding data resource configuration on the current football match big data using the resource processing adaptation block extracted from the resource management pool through the resource adaptation coefficient. The present application processes the current football match big data based on the resource processing adaptation block, which can achieve a balance of computing resources in data processing, thereby improving the quality of data processing.

Description

Technical Field

[0001] The present application relates to the technical field of event data processing, and more specifically, to a football event data processing system and method based on big data. Background Art

[0002] With the development of big data, the amount of data is growing exponentially, including structured data, semi-structured data and unstructured data. In event data processing, especially in data analysis and machine learning competitions, contestants are usually faced with the task of processing and analyzing large-scale data sets, which requires a lot of computing resources. With the development of data processing and machine learning algorithms, more complex algorithms are needed to improve the performance of the models. These algorithms may include deep learning models, ensemble learning algorithms, etc., which often require more computing resources for training and optimization. In addition, some events may have high requirements for real-time performance, such as real-time data analysis and real-time prediction. In order to meet these requirements, high-performance computing resources are needed for real-time data processing and analysis. In addition, the development of big data and cloud computing technology has also provided strong support for event data processing. Big data technology can process huge sets of event data and extract valuable information and patterns from them, providing a reliable infrastructure for event data processing.

[0003] Event data processing typically involves steps such as data collection, cleaning, storage, analysis, and visualization, utilizing various technologies and tools, including data mining, machine learning, and big data technologies. However, data processing may involve a wide range of sources and varying data types. Some data sources may provide a large amount of data, while others may provide a small amount, resulting in an uneven distribution of computing resources in the data processing process. Furthermore, during important matches or critical moments, the volume of data requests may surge, leading to an uneven distribution of computing resources in the data processing process. This uneven distribution of computing resources can lead to increased processing delays and response times, impacting data processing quality. Therefore, how to balance computing resources in data processing and thereby improve the quality of data processing has become a challenging issue facing the industry. Summary of the Invention

[0004] The present application provides a football match data processing system and method based on big data, which can achieve a balance of computing resources in data processing and thus improve the quality of data processing.

[0005] In a first aspect, the present application provides a method for processing football match data based on big data, comprising the following steps:

[0006] Acquire historical processing task data of football match big data, and set a resource management pool for football match data processing based on the historical processing task data;

[0007] Acquiring transmission response data of a current football match data collection terminal, extracting a downward capture response amount and an upward transmission response amount of the current football match data collection terminal through the transmission response data, and analyzing the downward capture response amount and the upward transmission response amount to obtain a current data backlog loss;

[0008] Determining a capture time span and a cache time span of the current football match big data, and then determining the data freshness of the data cache when storing the football match big data based on the capture time span and the cache time span;

[0009] determining a resource adaptation coefficient for processing the current football match big data according to the data backlog loss and the data freshness, and extracting a resource processing adaptation block for the current football match big data from the resource management pool based on the resource adaptation coefficient;

[0010] Based on the resource configuration information of the resource processing adapter block, corresponding data resource configuration is performed on the current football match big data.

[0011] In some embodiments, setting a resource management pool for football match data processing based on the historical processing task data specifically includes:

[0012] Obtaining a set of computing resource consumption corresponding to each processing task in the historical processing task data;

[0013] For each processing task, determine the processing resource segment set corresponding to the processing task according to the corresponding computing resource consumption set, and then obtain the processing resource segment set corresponding to each processing task;

[0014] Connect the processing resource segment set of each processing task with the corresponding resource segment to obtain all resource management blocks;

[0015] All resource management blocks are combined into a resource management pool for football match data processing.

[0016] In some embodiments, determining a processing resource segment set corresponding to a processing task based on a corresponding computing resource consumption set specifically includes:

[0017] Arrange the computing resource consumptions in the computing resource consumption set in ascending order to obtain a computing resource consumption sequence;

[0018] determining a plurality of division points of the computing resource consumption sequence;

[0019] The computing resource consumption sequence is divided by all division points to obtain a set of processing resource segments corresponding to the processing tasks.

[0020] In some embodiments, analyzing the current data backlog loss based on the downward capture response volume and the upward transmission response volume specifically includes:

[0021] Determine the capture interference rejection factor of the current football match data collection terminal when capturing football match big data;

[0022] Determine the transmission anti-interference factor when the current football match data collection terminal transmits football match big data;

[0023] Determining a downward capture response confidence value by multiplying the downward capture response value by the capture rejection factor;

[0024] determining an upward transmission response confidence value by multiplying the upward transmission response value by the transmission interference rejection factor;

[0025] The ratio of the downward capture response confidence to the upward transmission response confidence is used as the current data backlog loss.

[0026] In some embodiments, determining the capture time span and cache time span of the current football match big data specifically includes:

[0027] Get the capture arrival time and capture sending time of the current football match big data capture;

[0028] Determining a capture time span by the difference between the capture sending time point and the capture time arrival point;

[0029] Get the cache arrival time and cache sending time of the current football match big data cache;

[0030] The cache time span is determined by the difference between the cache sending time point and the cache arrival time point.

[0031] In some embodiments, the resource processing adaptation block for extracting the current football match big data from the resource management pool based on the resource adaptation coefficient specifically includes:

[0032] Obtain all resource management blocks in the resource management pool;

[0033] Determine a mapping identification section for each resource management block;

[0034] The resource adaptation coefficient is mapped to the mapping identification segment of each resource management block. If the mapping is successful, the resource management block corresponding to the mapping identification segment is used as the resource processing adaptation block of the current football match big data.

[0035] In some embodiments, the resource management pool includes a plurality of resource management blocks.

[0036] In a second aspect, the present application provides a football match data processing system based on big data, comprising:

[0037] An acquisition module is used to acquire historical processing task data of football match big data, and to set a resource management pool for football match data processing based on the historical processing task data;

[0038] a processing module configured to obtain transmission response data from a current football match data collection terminal, extract a downward capture response volume and an upward transmission response volume of the current football match data collection terminal from the transmission response data, and analyze the downward capture response volume and the upward transmission response volume to obtain a current data backlog loss;

[0039] The processing module is further configured to determine a capture time span and a cache time span of the current football match big data, and further determine the data freshness of the data cache when storing the football match big data based on the capture time span and the cache time span;

[0040] The processing module is further configured to determine a resource adaptation coefficient for processing the current football match big data based on the data backlog loss and the data freshness, and extract a resource processing adaptation block for the current football match big data from the resource management pool based on the resource adaptation coefficient;

[0041] The execution module is used to perform corresponding data resource configuration on the current football match big data based on the resource configuration information of the resource processing adaptation block.

[0042] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores code, and the processor is configured to obtain the code and execute the above-mentioned big data-based football match data processing method.

[0043] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned football match data processing method based on big data.

[0044] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0045] In the big data-based football match data processing system and method provided by the present application, first, historical processing task data of the football match big data is obtained, and a resource management pool for football match data processing is set based on the historical processing task data; second, transmission response data of the current football match data acquisition terminal is obtained, and the downward capture response volume and the upward transmission response volume of the current football match data acquisition terminal are extracted through the transmission response data. The current data backlog loss is analyzed based on the downward capture response volume and the upward transmission response volume; then, the capture time span and the cache time span of the current football match big data are determined, and then the data freshness of the data cache when storing the football match big data is determined based on the capture time span and the cache time span; a resource adaptation coefficient for processing the current football match big data is determined based on the data backlog loss and the data freshness, and a resource processing adaptation block for the current football match big data is extracted from the resource management pool based on the resource adaptation coefficient; finally, data resource configuration is performed accordingly for the current football match big data based on resource configuration information of the resource processing adaptation block.

[0046] It can be seen that the present application can achieve the balance of computing resources in data processing, thereby improving the quality of data processing; first, by analyzing the computing resource consumption of historical processing task data, a data resource management pool is set up, which can centrally manage and schedule computing resources and dynamically allocate resources according to actual needs, thereby maximizing resource utilization and resource balance; secondly, the data backlog loss in the data cache area is determined, and the data backlog loss can be used to measure the degree of loss in the current football match big data transmission and processing process, thereby avoiding the imbalance of computing resources caused by large changes in data volume; further, the data freshness of the data cache area when storing football match big data is determined, through The timeliness of real-time processing of football match big data can be measured through data freshness, so as to better analyze the processing resources required for the current football match big data, thereby avoiding waste and idleness of processing resources; then, the resource adaptation coefficient when processing the current football match big data is determined according to the data backlog loss and the data freshness, and the resource processing adaptation block of the current football match big data is extracted from the resource management pool according to the resource adaptation coefficient; finally, the current football match big data is configured with corresponding data resources based on the resource configuration information of the resource processing adaptation block; in summary, the technical solution provided by the present application can achieve a balance of computing resources in data processing, thereby improving the quality of data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is an exemplary flow chart of a method for processing football match data based on big data according to some embodiments of the present application;

[0048] Figure 2 is an exemplary flow chart for determining data backlog loss according to some embodiments of the present application;

[0049] Figure 3 is an exemplary flow chart of determining a resource processing adaptation block according to some embodiments of the present application;

[0050] Figure 4 1 is a schematic structural diagram of a football match data processing system based on big data according to some embodiments of the present application;

[0051] Figure 5 It is a structural diagram of a computer device for implementing a football match data processing method based on big data as shown in some embodiments of the present application. DETAILED DESCRIPTION

[0052] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0053] refer to Figure 1 , which is an exemplary flow chart of a method for processing football match data based on big data according to some embodiments of the present application. The method 100 for processing football match data based on big data mainly includes the following steps:

[0054] In step 101, historical processing task data of football match big data is obtained, and a resource management pool for processing football match data is set based on the historical processing task data.

[0055] In specific implementation, historical processing task data of football match big data is obtained through the football match database. The historical processing task data refers to the processing task data for data processing of football match big data in different historical football match cycles, wherein the processing tasks include data cleaning tasks, data integration tasks, and data storage tasks.

[0056] It should be noted that the football match big data refers to the collection and record of a large amount of data generated by football matches. These data cover various match-related information, such as match event data, player data, team data, etc. The football match database is a database that stores football-related information, and contains various data related to football matches, such as match results, team information, player information, league rankings, schedules, etc.

[0057] It should also be noted that in the football match data processing system, each processing task requires a certain amount of computing resources. When large-scale data appears, the competition for computing resources among different processing tasks increases, and the computing resources consumed by different processing tasks increase, making the distribution of computing resources unbalanced. Therefore, different resource levels can be pre-set based on historical processing task data to eliminate competition for computing resources.

[0058] In some embodiments, the resource management pool for football match data processing based on the historical processing task data may be set up by the following steps:

[0059] Obtaining a set of computing resource consumption corresponding to each processing task in the historical processing task data;

[0060] For each processing task, determine the processing resource segment set corresponding to the processing task according to the corresponding computing resource consumption set, and then obtain the processing resource segment set corresponding to each processing task;

[0061] Connect the processing resource segment set of each processing task with the corresponding resource segment to obtain all resource management blocks;

[0062] All resource management blocks are combined into a resource management pool for football match data processing.

[0063] In specific implementation, the computing resource consumption set corresponding to each processing task in the historical processing task data is obtained, that is: the computing resource consumption of each processing task in different football match cycles is obtained from the historical processing task data, thereby obtaining the computing resource consumption set of each processing task.

[0064] It should be noted that the computing resource consumption of different football match cycles of each processing task can be obtained through the football match database, where the computing resource consumption set includes multiple computing resource consumptions. The computing resource consumption in this application represents the amount of computing resources used in the process of performing data processing, analysis, calculation and other tasks.

[0065] In some embodiments, determining the processing resource segment set corresponding to the processing task according to the corresponding computing resource consumption set may specifically be performed by the following steps, namely:

[0066] Arrange the computing resource consumptions in the computing resource consumption set in ascending order to obtain a computing resource consumption sequence;

[0067] determining a plurality of division points of the computing resource consumption sequence;

[0068] The computing resource consumption sequence is divided by all division points to obtain a set of processing resource segments corresponding to the processing tasks.

[0069] In specific implementation, multiple division points of the computing resource consumption sequence are determined, that is: obtain the computing resource consumption sequence, take the computing resource consumption at one-quarter of the computing resource consumption sequence from the front to the back as the first division point, take the computing resource consumption at one-half of the computing resource consumption sequence from the front to the back as the second division point, and take the computing resource consumption at three-quarters of the computing resource consumption sequence from the front to the back as the third division point.

[0070] In specific implementation, the computing resource consumption sequence is divided by all partitioning points to obtain a set of processing resource segments corresponding to the processing task, that is: first, all partitioning points are obtained, and the interval from the smallest computing resource consumption in the computing resource consumption sequence to the first partitioning point is used as the first processing resource segment, the interval from the first partitioning point to the second partitioning point is used as the second processing resource segment, the interval from the second partitioning point to the third partitioning point is used as the third processing resource segment, and the interval from the third resource segment to the maximum computing resource consumption after arrangement is used as the fourth processing resource segment. Finally, all processing resource segments are combined to obtain a set of resource segments corresponding to the processing task.

[0071] It should be noted that the processing resource segment set in the present application includes multiple processing resource segments, wherein the processing resource segments represent processing resource segments divided by computing resource consumption, and are used to divide the computing resource consumption into levels.

[0072] In specific implementation, the processing resource segment set of each processing task is connected with the corresponding resource segments to obtain all resource management blocks, that is, the processing resource segments of the same level in the processing resource segment set of each processing task are connected accordingly to obtain all resource management blocks. For example, the first processing resource segment in the processing resource segment set of each processing task is connected to form a processing data block, the second processing resource segment in the processing resource segment set of each processing task is connected to form a processing data block, the third processing resource segment in the processing resource segment set of each processing task is connected to form a processing data block, and the fourth processing resource segment in the processing resource segment set of each processing task is connected to form a processing data block to obtain all resource management blocks. The connection can be performed by sorting connection, that is, hierarchical sorting is performed first, and then the processing resource segments of the same level are connected. In addition, in other embodiments, range connection, hash connection and other methods can also be used to connect the corresponding resource segments of the processing resource segment set of each processing task, which is not limited here.

[0073] It should be noted that the resource management pool represents a collection of multiple resource management blocks, that is, the resource management pool contains multiple resource management blocks. In this application, a resource management block represents a block connected by processing resource segments of the same level of different processing tasks. In the processing of big data of football matches, the computing resources consumed when processing data from different periods are inconsistent, and the different processing tasks also make the consumed computing resources different. Each time data is processed, different processing tasks are experienced, and there is also a resource competition relationship between different processing tasks. Therefore, this application divides the computing resources into levels by analyzing the consumption of computing resources by different historical processing tasks, and connects the same-level processing resource segments of different processing tasks, which can eliminate competition for computing resources, improve the precise allocation of computing resources, and thus enhance the processing performance of data processing.

[0074] In step 102, the transmission response data of the current football match data acquisition terminal is obtained, and the downward capture response amount and the upward transmission response amount of the current football match data acquisition terminal are extracted through the transmission response data. The current data backlog loss is obtained based on the analysis of the downward capture response amount and the upward transmission response amount.

[0075] In a specific implementation, the transmission response data of the current football match data acquisition terminal can be obtained through the real-time data display function of the football match data processing system. The transmission response data includes transmission response values at different time points, wherein the transmission response value represents the data response amount at different times during the data transmission process of the football match data acquisition terminal, that is, the data response amount at any time point in the time period from the start time point of collection to the end time point of caching of the football match data acquisition terminal. In this application, the transmission response value is characterized by data amount. The time period includes the data collection stage, the data caching stage, and the data processing stage, wherein the data caching stage and the data processing stage can be combined into the data upload stage. In addition, the transmission response value can also be the result value of the request, status information value, error information value or other related content, used to inform the sender or client of the processing status of the request, which is not limited here.

[0076] In some embodiments, the following steps may be used to extract the transmission response data to obtain the downward capture response amount and the upward transmission response amount of the current football match data collection terminal:

[0077] Obtaining the transmission response data;

[0078] Determining a response split point for the transmission response data;

[0079] Splitting the transmission response data according to the response splitting point to obtain first transmission response data and second transmission response data;

[0080] determining a downward capture response amount of the current football match data collection terminal based on the first transmission response data;

[0081] The upward transmission response amount of the current football match data collection terminal is determined by the second transmission response data.

[0082] In specific implementation, the response splitting point of the transmission response data is determined. The response splitting point can be determined by the time point when the current football match data acquisition terminal completes the acquisition task, that is, the time point when the current football match data acquisition terminal completes the acquisition task is used as the response splitting point. It should be noted that the response splitting point in this application represents the time point used to split the transmission response data. By determining the response splitting point, feature extraction of different transmission stages can be effectively performed.

[0083] In specific implementation, the transmission response data is divided according to the response split point to obtain first transmission response data and second transmission response data, that is: with the response split point as the dividing point, the transmission response data is divided into two segments, and the first half of the transmission response data is used as the first transmission response data, and the second half of the transmission response data is used as the second transmission response data. It should be noted that the first transmission response data in this application represents the first half of the transmission response data with the response split point as the dividing point, and the second transmission response data in this application represents the second half of the transmission response data with the response split point as the dividing point.

[0084] In specific implementation, the downward capture response amount of the current football match data acquisition terminal is determined by the first transmission response data, that is: the standard deviation of all transmission response values in the first transmission response data is calculated, and the standard deviation calculation result is used as the downward capture response amount of the current football match data acquisition terminal. In addition, in other embodiments, other calculation methods can also be used to calculate the downward capture response amount of the current football match data acquisition terminal, which is not limited here. It should be noted that the downward capture response amount in this application represents the change in the amount of data collected by the current football match data acquisition terminal during the time period of executing the collection task.

[0085] In specific implementation, the upward transmission response amount of the current football match data acquisition terminal is determined by the second transmission response data, that is: the standard deviation of all transmission response values in the second transmission response data is calculated, and the standard deviation calculation result is used as the upward transmission response amount of the current football match data acquisition terminal. In addition, in other embodiments, other calculation methods can also be used to calculate the upward transmission response amount of the current football match data acquisition terminal, which is not limited here. It should be noted that the upward transmission response amount in this application represents the change in the amount of data collected during the time period when the current football match data acquisition terminal performs data upload.

[0086] In some embodiments, reference Figure 2 As shown in FIG, this figure is an exemplary flow chart of determining data backlog loss according to some embodiments of the present application. In this embodiment, the current data backlog loss is obtained by analyzing the downward capture response volume and the upward transmission response volume, which can be implemented by the following steps:

[0087] First, in step 1021, the capture interference rejection factor of the current football match data collection terminal when capturing football match big data is determined;

[0088] Next, in step 1022, the transmission anti-interference factor of the current football match data collection terminal when transmitting the football match big data is determined;

[0089] Further, in step 1023, a downward capture response confidence value is determined by multiplying the downward capture response value by the capture rejection factor;

[0090] Then, in step 1024, an upward transmission response confidence value is determined by multiplying the upward transmission response value by the transmission interference rejection factor;

[0091] Finally, in step 1025, the ratio of the downward capture response confidence to the upward transmission response confidence is used as the current data backlog loss.

[0092] During specific implementation, the capture anti-interference factor of the current football match data acquisition terminal when capturing football match big data is determined. The capture anti-interference factor can be set according to the acquisition performance of the football match data acquisition terminal, and is usually set to a number between 0 and 1. It can be set according to actual needs. The stronger the acquisition performance, the larger the capture anti-interference factor is set. The weaker the acquisition performance, the smaller the capture anti-interference factor is set. In this application, the capture anti-interference factor is set to 0.65. The capture anti-interference factor in this application represents the ability of data to resist various interferences and difficulties when capturing football match big data, that is, the stronger the resistance, the larger the anti-interference factor is set.

[0093] During specific implementation, the transmission anti-interference factor of the current football match data acquisition terminal when capturing and transmitting football match big data is determined. The transmission anti-interference factor can be set according to the transmission performance of the football match data acquisition terminal, and is usually set to a number between 0 and 1. It can be set according to actual needs. The stronger the transmission performance, the larger the transmission anti-interference factor is set. The weaker the transmission performance, the smaller the transmission anti-interference factor is set. In this application, the transmission anti-interference factor is set to 0.65. The transmission anti-interference factor in this application represents the ability of data to resist various interferences and difficulties when transmitting football match big data, that is, the stronger the resistance, the larger the transmission anti-interference factor is set.

[0094] It should be noted that the downward capture response confidence in this application represents the value of the downward capture response after adjustment by the capture anti-interference factor, which is used to truly represent the change in the amount of data collected by the current football match data acquisition terminal during the time period when the collection task is performed. The upward transmission response confidence in this application represents the value of the upward transmission response after adjustment by the transmission anti-interference factor, which is used to truly represent the change in the amount of data collected during the time period when the current football match data acquisition terminal performs data upload.

[0095] It should also be noted that the data backlog loss in this application refers to the amount of information lost due to data backlog during the data upload process, which is used to measure the degree of loss in the current football match big data transmission process, that is, the greater the data backlog loss, the lower the network transmission efficiency, the greater the loss in the current football match big data transmission process, and the poorer the network quality. The smaller the data backlog loss, the higher the network transmission efficiency, the lower the loss in the current football match big data transmission process, and the better the network quality. In the football match big data processing, the data processing needs to be carried out in a process sequence, that is, from capturing data, caching data to data processing, etc. However, when there is a difference between the amount of data transmitted to the cache area by the football match big data and the actual amount of data captured, if the data capture amount is greater than the data transmission amount, it means that the capture speed exceeds the transmission speed, which may cause the cache area to be blocked, causing the loss in the current football match big data transmission process to increase. Therefore, the degree of loss in the current football match big data transmission process can be measured by the ratio of the downward capture response confidence to the upward transmission response confidence.

[0096] In step 103, the capture time span and the cache time span of the current football match big data are determined, and then the data freshness of the data cache when storing the football match big data is determined based on the capture time span and the cache time span.

[0097] In some embodiments, the following steps may be used to determine the capture time span and cache time span of the current football match big data:

[0098] Get the capture arrival time and capture sending time of the current football match big data capture;

[0099] Determining a capture time span by the difference between the capture sending time point and the capture time arrival point;

[0100] Get the cache arrival time and cache sending time of the current football match big data cache;

[0101] The cache time span is determined by the difference between the cache sending time point and the cache arrival time point.

[0102] It should be noted that the capture arrival time point in this application represents the time point when the capture data request is sent, marking the moment when the capture data request starts to be processed; the capture sending time point in this application represents the time point when the capture data result is sent to the system, marking the completion of the entire capture process; the cache arrival time point in this application represents the time point when the cache request is sent, marking the moment when the cache request starts to be processed; the cache sending time point in this application represents the time point when the cache result is sent to the system, marking the completion of the entire cache process; wherein, the capture arrival time point, the capture sending time point, the cache arrival time point and the cache sending time point can all be obtained through the football event database, which will not be repeated here.

[0103] It should also be noted that the capture time span in this application represents the time span of the system executing the data capture process, and the cache time span in this application represents the time span of the system executing the data caching process. In the process of processing big data of football matches, the time spans of different stages affect the allocation and management of resources. The longer the time span, the more computing resources are required, and the shorter the time span, the fewer computing resources are required. Different time spans may also show different trends, patterns and laws. By analyzing the time spans of different stages, we can understand the characteristics of the data more accurately and make reasonable decisions and strategies.

[0104] In some embodiments, the following steps may be used to determine the data freshness of the data cache when storing the football match big data based on the capture time span and the cache time span, namely:

[0105] Obtaining the capture time span and the cache time span;

[0106] Determine the cache validity limit of football match data collection terminals when caching data;

[0107] The timeliness of the data cache area when storing the football match big data is analyzed based on the capture time span, the cache time span and the cache time limit, thereby obtaining the data freshness of the data cache area when storing the football match big data.

[0108] In specific implementation, the cache expiration limit of the football match data collection terminal when caching data can be set according to the importance and frequency of use of the data. Since the football match big data in this application is real-time data, the cache expiration limit is set to 3 seconds. It should be noted that the cache expiration limit in this embodiment represents the time limit for the data to be considered "valid" or "fresh" in the cache.

[0109] In a specific implementation, the capture time span, the cache time span and the cache time limit are combined to analyze the timeliness of the data cache area when storing the football match big data, and then obtain the data freshness of the data cache area when storing the football match big data, that is: a negative exponential function is constructed by the cache time limit, the absolute difference between the capture time span and the cache time span is calculated, and the absolute difference calculation result is input as an input parameter into the negative exponential function, and the negative exponential function outputs the data freshness of the data cache area when storing the football match big data. In addition, in other embodiments, other calculation methods can also be used to calculate the data freshness of the data cache area when storing the football match big data.

[0110] In a specific implementation, a negative exponential function is constructed from the cache expiration limit, that is, the cache expiration limit is used as the base of the negative exponential function to thereby construct the negative exponential function.

[0111] It should be noted that the data freshness in this application represents an indicator for measuring the timeliness of real-time processing of football match big data, that is, the greater the data freshness, the better the timeliness of real-time processing of football match big data, and the smaller the data freshness, the smaller the timeliness of real-time processing of football match big data. In the processing of football match data, data processing needs to go through different stages before processing. The processing time of different stages is different, and the computing resources consumed are also different. The processing time of the previous stage may be greater than or less than the processing time of the current stage, that is, there is a time difference between the processing of different stages. Due to the existence of the time difference, the computing resources within the system will be consumed in the time gap, resulting in some computing resources being wasted. Therefore, by determining the data freshness of the data cache when storing football match big data, the processing resources required for the current football match big data can be better analyzed, thereby avoiding waste and idleness of processing resources.

[0112] In step 104, a resource adaptation coefficient for processing the current football match big data is determined according to the data backlog loss and the data freshness, and a resource processing adaptation block for the current football match big data is extracted from the resource management pool based on the resource adaptation coefficient.

[0113] In some embodiments, the resource adaptation coefficient for processing the current football match big data is determined based on the data backlog loss and the data freshness, specifically by the following steps, namely:

[0114] Obtaining the data backlog loss and the data freshness;

[0115] Determine the data backlog impact coefficient of the football match data processing system;

[0116] Determining a data timeliness impact coefficient of the football match data processing system;

[0117] The resource adaptation coefficient when processing the current football match big data is determined by the data backlog loss, the data freshness, the data backlog impact coefficient, and the data timeliness impact coefficient.

[0118] During specific implementation, the data backlog impact coefficient of the football match data processing system is determined. The data backlog impact coefficient can be set according to actual needs. For example, it can be set according to the degree of impact of data backlog loss on the football match data processing system. Since the data backlog loss has a greater impact on the football match data processing system in this application, the data backlog impact coefficient is set to 0.67. In addition, in other embodiments, the data backlog impact coefficient can also be set according to actual needs, wherein the data backlog impact coefficient in this example represents a weight coefficient for adjusting the data backlog loss.

[0119] During specific implementation, the data timeliness impact coefficient of the football match data processing system is determined. The data timeliness impact coefficient can be set according to actual needs. For example, it can be set according to the degree of influence of data freshness on the football match data processing system. Since the impact of data freshness on the football match data processing system in this application is relatively small, the data timeliness impact coefficient is set to 0.33. In addition, in other embodiments, the data timeliness impact coefficient can also be set according to actual needs, wherein the data timeliness impact coefficient in this example represents a weight coefficient for adjusting data freshness.

[0120] In specific implementation, the resource adaptation coefficient when processing the current football match big data is determined by the data backlog loss, the data freshness, the data backlog impact coefficient and the data timeliness impact coefficient, that is: the data backlog loss is multiplied by the data backlog impact coefficient to obtain a first product result, the data freshness is multiplied by the data timeliness impact coefficient to obtain a second product result, and the sum of the first product result and the second product result is used as the resource adaptation coefficient when processing the current football match big data. In addition, in other embodiments, other calculation methods can also be used to calculate the resource adaptation coefficient when processing the current football match big data, which is not limited here.

[0121] It should be noted that the resource adaptation coefficient in this application represents an indicator for measuring the degree of matching between data processing resources and data requirements. In this application, the resource adaptation coefficient is used to select adapted resource blocks. Different adaptation coefficients adapt to different resource blocks. The larger the resource adaptation coefficient, the higher the level of the adapted resource block. By adjusting the resource adaptation coefficient, the data processing resources required for the current football match big data can be effectively extracted, thereby improving the utilization rate of data processing resources.

[0122] It should be noted that in the process of processing football match data, the amount of data at different stages and the time span of different stages will have an impact on the consumption of computing resources. That is, the increase in the amount of data will lead to data inflow congestion, resulting in increased losses in the transmission process, and thus increased consumption of computing resources. A longer time span also requires more computing resources. A long time span means that more data points or more time series data need to be processed, which will also increase the complexity of the calculation and resource consumption. Therefore, the data amount and time span at different stages before data processing can be analyzed to pre-determine the resource adaptation coefficient of the current football match big data, and then adapt the resource block according to the resource adaptation coefficient.

[0123] In some embodiments, reference Figure 3 As shown in FIG. 1 , this figure is an exemplary flow chart of determining a resource processing adaptation block according to some embodiments of the present application. In this embodiment, the resource processing adaptation block that extracts the current football match big data from the resource management pool based on the resource adaptation coefficient can be implemented using the following steps:

[0124] In step 1041, all resource management blocks in the resource management pool are obtained;

[0125] In step 1042, a mapping identification section of each resource management block is determined;

[0126] In step 1043, the resource adaptation coefficient is mapped to the mapping identification segment of each resource management block. If the mapping is successful, the resource management block corresponding to the mapping identification segment is used as the resource processing adaptation block for the current football match big data.

[0127] In specific implementation, the mapping identification segment of each resource management block is determined, that is: the number of resource management blocks in the resource management pool is obtained, the resource management blocks in the resource management pool are arranged hierarchically from small to large, and each level resource management block is segment-compressed in sequence according to the number of resource management blocks to obtain the mapping identification segment corresponding to each level resource management block. For example, after the resource management blocks in the resource management pool are hierarchically arranged, there are four levels: first-level resource management block, second-level resource management block, third-level resource management block and fourth-level resource management block, then the four levels of resource management blocks are compressed evenly to the range of 0 to 1, and the mapping identification segments corresponding to each level resource management block are: [0, 0.25), [025, 0.5), [05, 0.75), [0.75, 1], that is, one level resource management block corresponds to one mapping identification segment.

[0128] In specific implementation, the resource adaptation coefficient is mapped to the mapping identification segment of each resource management block, that is: for the mapping identification segment of each resource management block, it is determined whether the value of the resource adaptation coefficient is within the value of the mapping identification segment. If so, the mapping is successful, otherwise the mapping fails.

[0129] It should be noted that the mapping identification segment in this application represents the mapping of different levels of resource management blocks to identification segments between 0 and 1. The determination of the mapping identification segment is to speed up the adaptation of resource allocation during data processing, reduce the complexity of adaptation, and improve the efficiency of adaptation.

[0130] It should also be noted that the resource processing adaptation block in this application represents the resource management block adapted for the current football match big data processing. By determining the resource processing adaptation block, the processing resources of the current football match big data can be effectively allocated, thereby improving the processing efficiency of the football match big data.

[0131] In step 105, corresponding data resource configuration is performed on the current football match big data based on the resource configuration information of the resource processing adapter block.

[0132] In some embodiments, the following steps may be used to perform corresponding data resource configuration on the current football match big data based on the resource configuration information of the resource processing adapter block, namely:

[0133] Acquiring resource configuration information of the resource processing adapter block, wherein the resource configuration information includes processing resource segments for different processing tasks;

[0134] Obtain different processing tasks for current football match big data;

[0135] Configure corresponding processing resource segments for different processing tasks.

[0136] It should be noted that different processing tasks include data cleaning tasks, data integration tasks, and data storage tasks, which are not limited here.

[0137] It should also be noted that the data resource configuration in this application refers to the process of configuring corresponding processing resource segments for the processing task of the current football match big data.

[0138] In specific implementation, corresponding processing resource segments are configured for different processing tasks, namely: data cleaning tasks correspond to processing resource segments configured for data cleaning, data integration tasks correspond to processing resource segments configured for data integration, and data storage tasks correspond to processing resource segments configured for data storage. Details will not be given here.

[0139] It should also be noted that the data resource configuration in this application represents the allocation of corresponding processing resource segments for different data processing tasks of football match big data, namely: the data cleaning task corresponds to the processing resource segment for configuring data cleaning, the data integration task corresponds to the processing resource segment for configuring data integration, and the data storage task corresponds to the processing resource segment for configuring data storage. In the processing of football match data, when large-scale data appears, different processing tasks require different computing resources. For football match big data that is subject to more serious interference, more computing resources may be required in the data cleaning task. For football match big data obtained from different data sources, more computing resources are required in data integration. For the need to predict football match big data, more computing resources are required in data storage. Therefore, this application pre-rationally configures computing resources. For each processing of football match big data, a preliminary data loss analysis and time span analysis can be performed on the current football match big data, so as to comprehensively characterize the resource configuration required in the current situation based on the data loss and time span, thereby achieving effective processing of football match big data and avoiding waste of computing resources.

[0140] In addition, in another aspect of the present application, in some embodiments, the present application provides a football match data processing system based on big data, referring to Figure 4 This figure is a schematic diagram of the structure of a football match data processing system based on big data according to some embodiments of the present application. The football match data processing system 200 based on big data includes: an acquisition module 201, a processing module 202 and an execution module 203, which are described as follows:

[0141] Acquisition module 201, in this application, acquisition module 201 is mainly used to obtain historical processing task data of football match big data, and set the resource management pool for football match data processing based on the historical processing task data;

[0142] Processing module 202, in this application, is primarily used to obtain transmission response data from the current football match data collection terminal, extract the downward capture response volume and the upward transmission response volume of the current football match data collection terminal from the transmission response data, and analyze the downward capture response volume and the upward transmission response volume to obtain the current data backlog loss;

[0143] It should be noted that the processing module 202 in the present application is also used to determine the capture time span and cache time span of the current football match big data, and then determine the data freshness of the data cache when storing the football match big data based on the capture time span and the cache time span;

[0144] In addition, the processing module 202 is further configured to determine a resource adaptation coefficient for processing the current football match big data based on the data backlog loss and the data freshness, and extract a resource processing adaptation block for the current football match big data from the resource management pool based on the resource adaptation coefficient;

[0145] The execution module 203 in this application is mainly used to perform corresponding data resource configuration on the current football match big data based on the resource configuration information of the resource processing adaptation block.

[0146] In addition, the present application also provides a computer device, which includes a memory and a processor, the memory storing a code, and the processor being configured to obtain the code and execute the above-mentioned big data-based football match data processing method.

[0147] In some embodiments, reference Figure 5 , which is a schematic diagram of the structure of a computer device that applies a football match data processing method based on big data according to some embodiments of the present application. The football match data processing method based on big data in the above embodiment can be Figure 5 The computer device 300 shown in FIG. 1 is implemented as shown in FIG. 1 , and the computer device 300 includes at least one processor 301 , a communication bus 302 , a memory 303 , and at least one communication interface 304 .

[0148] The processor 301 can be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more processors for controlling the execution of the big data-based football match data processing method in this application.

[0149] The communication bus 302 may include a pathway for transmitting information between the aforementioned components.

[0150] Memory 303 may be, but is not limited to, a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, a random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, an optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer. Memory 303 may be independent and connected to processor 301 via communication bus 302. Memory 303 may also be integrated with processor 301.

[0151] Memory 303 is used to store program code for executing the solution of the present application, and is controlled by processor 301 for execution. Processor 301 is used to execute the program code stored in memory 303. The program code may include one or more software modules. The determination of the big data-based football match data processing method in the above embodiment can be implemented by processor 301 and one or more software modules in the program code in memory 303.

[0152] The communication interface 304 uses any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0153] In a specific implementation, as an example, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0154] The aforementioned computer device can be a general-purpose computer device or a dedicated computer device. In a specific implementation, the computer device can be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. The embodiments of this application do not limit the type of computer device.

[0155] In addition, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned football match data processing method based on big data.

[0156] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0157] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A football match data processing method based on big data, characterized in that: The steps include: Acquire historical processing task data of football match big data, and set a resource management pool for football match data processing based on the historical processing task data; Acquiring transmission response data of a current football match data collection terminal, extracting a downward capture response amount and an upward transmission response amount of the current football match data collection terminal through the transmission response data, and analyzing the downward capture response amount and the upward transmission response amount to obtain a current data backlog loss; Determining a capture time span and a cache time span of the current football match big data, and then determining the data freshness of the data cache when storing the football match big data based on the capture time span and the cache time span; determining a resource adaptation coefficient for processing the current football match big data according to the data backlog loss and the data freshness, and extracting a resource processing adaptation block for the current football match big data from the resource management pool based on the resource adaptation coefficient; Performing corresponding data resource configuration on the current football match big data based on the resource configuration information of the resource processing adapter block; The resource management pool for football match data processing based on the historical processing task data is specifically set up as follows: Obtaining a set of computing resource consumption corresponding to each processing task in the historical processing task data; For each processing task, determine the processing resource segment set corresponding to the processing task according to the corresponding computing resource consumption set, and then obtain the processing resource segment set corresponding to each processing task; Connect the processing resource segment set of each processing task with the corresponding resource segment to obtain all resource management blocks; Combine all resource management blocks into a resource management pool for football match data processing; Determining the processing resource segment set corresponding to the processing task according to the corresponding computing resource consumption set specifically includes: Arrange the computing resource consumptions in the computing resource consumption set in ascending order to obtain a computing resource consumption sequence; determining a plurality of division points of the computing resource consumption sequence; Dividing the computing resource consumption sequence by all the division points to obtain a set of processing resource segments corresponding to the processing tasks; The following steps are specifically used to extract the transmission response data to obtain the downward capture response amount and the upward transmission response amount of the current football match data acquisition terminal: Obtaining the transmission response data; Determining a response split point for the transmission response data; Splitting the transmission response data according to the response splitting point to obtain first transmission response data and second transmission response data; determining a downward capture response amount of the current football match data collection terminal based on the first transmission response data; determining the upward transmission response amount of the current football match data collection terminal based on the second transmission response data; Determining the downward capture response amount of the current football match data collection terminal from the first transmission response data specifically includes: calculating a standard deviation of all transmission response values in the first transmission response data, and using the standard deviation calculation result as the downward capture response amount of the current football match data collection terminal, wherein the downward capture response amount represents a change in the amount of data collected by the current football match data collection terminal during the time period of executing the collection task; The determining of the upward transmission response amount of the current football match data collection terminal from the second transmission response data specifically includes: calculating a standard deviation of all transmission response values in the second transmission response data, and using the standard deviation calculation result as the upward transmission response amount of the current football match data collection terminal, wherein the upward transmission response amount represents a change in the amount of data collected during a time period in which the current football match data collection terminal performs data upload; The current data backlog loss obtained by analyzing the downward capture response volume and the upward transmission response volume specifically includes: Determine the capture interference rejection factor of the current football match data collection terminal when capturing football match big data; Determine the transmission anti-interference factor when the current football match data collection terminal transmits football match big data; Determining a downward capture response confidence value by multiplying the downward capture response value by the capture rejection factor; determining an upward transmission response confidence value by multiplying the upward transmission response value by the transmission interference rejection factor; Taking the ratio of the downward capture response confidence to the upward transmission response confidence as the current data backlog loss; The resource adaptation coefficient for processing the current football match big data is determined based on the data backlog loss and the data freshness by specifically adopting the following steps, namely: Obtaining the data backlog loss and the data freshness; Determine the data backlog impact coefficient of the football match data processing system; Determining a data timeliness impact coefficient of the football match data processing system; Determining a resource adaptation coefficient when processing the current football match big data based on the data backlog loss, the data freshness, the data backlog impact coefficient, and the data timeliness impact coefficient; The resource processing adaptation block for extracting the current football match big data from the resource management pool based on the resource adaptation coefficient specifically includes: Obtain all resource management blocks in the resource management pool; Determine a mapping identification section for each resource management block; The resource adaptation coefficient is mapped to the mapping identification segment of each resource management block. If the mapping is successful, the resource management block corresponding to the mapping identification segment is used as the resource processing adaptation block of the current football match big data.

2. The method according to claim 1, wherein Determining the capture time span and cache time span of the current football match big data specifically includes: Get the capture arrival time and capture sending time of the current football match big data capture; Determining a capture time span by the difference between the capture sending time point and the capture time arrival point; Get the cache arrival time and cache sending time of the current football match big data cache; The cache time span is determined by the difference between the cache sending time point and the cache arrival time point.

3. The method according to claim 1, wherein The resource management pool includes multiple resource management blocks.

4. A football match data processing system based on big data, which processes football match data using the method according to any one of claims 1 to 3, characterized in that: The system includes: An acquisition module is used to acquire historical processing task data of football match big data, and to set a resource management pool for football match data processing based on the historical processing task data; a processing module configured to obtain transmission response data from a current football match data collection terminal, extract a downward capture response volume and an upward transmission response volume of the current football match data collection terminal from the transmission response data, and analyze the downward capture response volume and the upward transmission response volume to obtain a current data backlog loss; The processing module is further configured to determine a capture time span and a cache time span of the current football match big data, and further determine the data freshness of the data cache when storing the football match big data based on the capture time span and the cache time span; The processing module is further configured to determine a resource adaptation coefficient for processing the current football match big data based on the data backlog loss and the data freshness, and extract a resource processing adaptation block for the current football match big data from the resource management pool based on the resource adaptation coefficient; The execution module is used to perform corresponding data resource configuration on the current football match big data based on the resource configuration information of the resource processing adaptation block.

5. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a code, and the processor is configured to obtain the code and execute the football match data processing method based on big data as described in any one of claims 1 to 3.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for processing football match data based on big data as described in any one of claims 1 to 3 is implemented.

Images