Football match data processing system and method based on big data

Abstract

The invention provides a football match data processing system and method based on big data. A resource management pool of data is set based on historical processing task data; the downward capture response amount and the upward transmission response amount of the current football match data acquisition terminal are obtained through transmission response data extraction, and then the current data backlog loss is obtained; determining a capture time span and a cache time span of the current football match big data, and determining data freshness of the data cache region according to the capture time span and the cache time span; and determining a resource adaptation coefficient of the current football match big data according to the data backlog loss and the data freshness, and performing corresponding data resource allocation on the current football match big data through a resource processing adaptation block extracted from a resource management pool according to the resource adaptation coefficient. The current football match big data is processed on the basis of the resource processing adaptation block, balance of computing resources in data processing can be achieved, and then the data processing quality is improved.

Description

Technical Field

[0001] The present application relates to the technical field of event data processing, and more specifically, to a football event data processing system and method based on big data. Background Art

[0002] With the development of big data, the amount of data is growing exponentially, including structured data, semi-structured data and unstructured data. In event data processing, especially in data analysis and machine learning competitions, participants are usually faced with the task of processing and analyzing large-scale data sets, which requires a lot of computing resources. With the development of data processing and machine learning algorithms, more complex algorithms are needed to improve the performance of the model. These algorithms may include deep learning models, ensemble learning algorithms, etc., which often require more computing resources to train and optimize. In addition, some events may have higher requirements for real-time performance, such as real-time data analysis, real-time prediction, etc. In order to meet these requirements, it is more necessary to use high-performance computing resources for real-time data processing and analysis. In addition, the development of big data and cloud computing technology has also provided strong support for event data processing. Big data technology can process huge sets of event data and extract valuable information and patterns from them, providing a reliable infrastructure for event data processing.

[0003] Event data processing usually adopts steps such as data collection, cleaning, storage, analysis and visualization, and utilizes various technologies and tools, including data mining, machine learning and big data technology. However, in the process of data processing, we may face situations where the sources are wide and the data types are different. Some data sources may provide a large amount of data, while others may provide less data, resulting in uneven distribution of computing resources in the data processing process. In addition, in important competitions or critical moments, the number of data requests may surge, which also leads to uneven distribution of computing resources in the data processing process. The problem of uneven distribution of computing resources will lead to increased processing delays and processing response time, affecting the quality of data processing. Therefore, how to balance the computing resources in data processing and improve the quality of data processing has become a difficult problem faced by the industry. Summary of the invention

[0004] The present application provides a football match data processing system and method based on big data, which can achieve a balance of computing resources in data processing and thus improve the quality of data processing.

[0005] In a first aspect, the present application provides a method for processing football match data based on big data, comprising the following steps: Acquire historical processing task data of football match big data, and set a resource management pool for football match data processing based on the historical processing task data; Acquire the transmission response data of the current football match data collection terminal, extract the downward capture response amount and the upward transmission response amount of the current football match data collection terminal through the transmission response data, and analyze the current data backlog loss according to the downward capture response amount and the upward transmission response amount; Determine the capture time span and cache time span of the current football match big data, and then determine the data freshness of the data cache when storing the football match big data according to the capture time span and the cache time span; Determine a resource adaptation coefficient when processing the current football match big data according to the data backlog loss and the data freshness, and extract a resource processing adaptation block of the current football match big data from the resource management pool based on the resource adaptation coefficient; Based on the resource configuration information of the resource processing adapter block, corresponding data resource configuration is performed on the current football match big data.

[0006] In some embodiments, setting a resource management pool for football match data processing based on the historical processing task data specifically includes: Obtaining a set of computing resource consumption corresponding to each processing task in the historical processing task data; For each processing task, determine the processing resource segment set corresponding to the processing task according to the corresponding computing resource consumption set, and then obtain the processing resource segment set corresponding to each processing task; Connect the processing resource segment set of each processing task with the corresponding resource segment to obtain all resource management blocks; All resource management blocks are combined into a resource management pool for football match data processing.

[0007] In some embodiments, determining a processing resource segment set corresponding to a processing task according to a corresponding computing resource consumption set specifically includes: Arrange the computing resource consumption in the computing resource consumption set in ascending order to obtain a computing resource consumption sequence; Determining a plurality of division points of the computing resource consumption sequence; The computing resource consumption sequence is divided by all the division points to obtain a set of processing resource segments corresponding to the processing task.

[0008] In some embodiments, analyzing the current data backlog loss according to the downward capture response amount and the upward transmission response amount specifically includes: Determine the capture anti-interference factor when the current football match data collection terminal captures football match big data; Determine the transmission anti-interference factor when the current football match data collection terminal transmits football match big data; Determining a downward capture response confidence amount by multiplying the downward capture response amount by the capture anti-disturbance factor; Determining an upward transmission response confidence amount by multiplying the upward transmission response amount by the transmission anti-interference factor; The ratio of the downward capture response confidence to the upward transmission response confidence is taken as the current data backlog loss.

[0009] In some embodiments, determining the capture time span and cache time span of the current football match big data specifically includes: Get the capture arrival time point and capture sending time point when capturing the current football match big data; Determine the capture time span by the difference between the capture sending time point and the capture time arrival point; Get the cache arrival time and cache sending time of the current football match big data cache; The cache time span is determined by the difference between the cache sending time point and the cache arrival time point.

[0010] In some embodiments, the resource processing adaptation block for extracting the current football match big data from the resource management pool based on the resource adaptation coefficient specifically includes: Obtain all resource management blocks in the resource management pool; Determine a mapping identification section for each resource management block; The resource adaptation coefficient is mapped to the mapping identification section of each resource management block. If the mapping is successful, the resource management block corresponding to the mapping identification section is used as the resource processing adaptation block of the current football match big data.

[0011] In some embodiments, the resource management pool includes a plurality of resource management blocks.

[0012] In a second aspect, the present application provides a football match data processing system based on big data, comprising: An acquisition module, used to acquire historical processing task data of football match big data, and to set a resource management pool for football match data processing based on the historical processing task data; A processing module is used to obtain the transmission response data of the current football match data collection terminal, extract the downward capture response amount and the upward transmission response amount of the current football match data collection terminal through the transmission response data, and analyze the current data backlog loss according to the downward capture response amount and the upward transmission response amount; The processing module is further used to determine the capture time span and cache time span of the current football match big data, and then determine the data freshness of the data cache area when storing the football match big data according to the capture time span and the cache time span; The processing module is further used to determine the resource adaptation coefficient when processing the current football match big data according to the data backlog loss and the data freshness, and extract the resource processing adaptation block of the current football match big data from the resource management pool based on the resource adaptation coefficient; The execution module is used to perform corresponding data resource configuration on the current football match big data based on the resource configuration information of the resource processing adaptation block.

[0013] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores codes, and the processor is configured to obtain the codes and execute the above-mentioned football match data processing method based on big data.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned football match data processing method based on big data is implemented.

[0015] The technical solution provided by the embodiments disclosed in this application has the following beneficial effects: In the football match data processing system and method based on big data provided by the present application, firstly, historical processing task data of football match big data is obtained, and a resource management pool for football match data processing is set based on the historical processing task data; secondly, the transmission response data of the current football match data acquisition terminal is obtained, and the downward capture response amount and the upward transmission response amount of the current football match data acquisition terminal are extracted through the transmission response data, and the current data backlog loss is obtained according to the downward capture response amount and the upward transmission response amount; then, the capture time span and the cache time span of the current football match big data are determined, and then the data freshness of the data cache area when the football match big data is stored is determined according to the capture time span and the cache time span; the resource adaptation coefficient when the current football match big data is processed is determined according to the data backlog loss and the data freshness, and the resource processing adaptation block of the current football match big data is extracted from the resource management pool based on the resource adaptation coefficient; finally, the current football match big data is configured with corresponding data resources based on the resource configuration information of the resource processing adaptation block.

[0016] It can be seen that the present application can achieve the balance of computing resources in data processing, thereby improving the quality of data processing; firstly, by analyzing the computing resource consumption of historical processing task data, a data resource management pool is set, which can centrally manage and schedule computing resources, and dynamically allocate resources according to actual needs, thereby maximizing resource utilization and resource balance; secondly, the data backlog loss of the data cache area is determined, and the data backlog loss can be used to measure the degree of loss in the current football match big data transmission and processing process, thereby avoiding the imbalance of computing resources caused by large changes in data volume; further, the data freshness of the data cache area when storing football match big data is determined, and the data backlog loss is ... The timeliness of real-time processing of football match big data can be measured by data freshness, so as to better analyze the processing resources required for the current football match big data, thereby avoiding waste and idleness of processing resources; then, the resource adaptation coefficient when processing the current football match big data is determined according to the data backlog loss and the data freshness, and the resource processing adaptation block of the current football match big data is extracted from the resource management pool according to the resource adaptation coefficient; finally, the current football match big data is configured with corresponding data resources based on the resource configuration information of the resource processing adaptation block; in summary, the technical solution provided by the present application can achieve the balance of computing resources in data processing, thereby improving the quality of data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is an exemplary flow chart of a method for processing football match data based on big data according to some embodiments of the present application; Figure 2 is an exemplary flow chart of determining data backlog loss according to some embodiments of the present application; Figure 3 is an exemplary flow chart of determining a resource processing adaptation block according to some embodiments of the present application; Figure 4 It is a structural schematic diagram of a football match data processing system based on big data according to some embodiments of the present application; Figure 5 It is a structural diagram of a computer device for implementing a football match data processing method based on big data as shown in some embodiments of the present application. DETAILED DESCRIPTION

[0018] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0019] refer to Figure 1, which is an exemplary flow chart of a football match data processing method based on big data according to some embodiments of the present application. The football match data processing method 100 based on big data mainly includes the following steps: In step 101, historical processing task data of football match big data is obtained, and a resource management pool for processing football match data is set based on the historical processing task data.

[0020] In specific implementation, historical processing task data of football match big data is obtained through the football match database. The historical processing task data refers to processing task data for data processing of football match big data in different historical football match cycles, wherein the processing tasks include data cleaning tasks, data integration tasks, and data storage tasks.

[0021] It should be noted that the football match big data refers to the collection and record of a large amount of data generated by football matches, which covers various match-related information, such as match event data, player data, team data, etc. The football match database is a database that stores football-related information, and contains various football match-related data, such as match results, team information, player information, league rankings, schedules, etc.

[0022] It should also be noted that in the football match data processing system, each processing task requires a certain amount of computing resources. When large-scale data appears, the competition for computing resources among different processing tasks increases, and the computing resources consumed by different processing tasks increase, making the distribution of computing resources unbalanced. Therefore, different resource levels can be pre-set according to historical processing task data to eliminate competition for computing resources.

[0023] In some embodiments, the resource management pool for football match data processing is set based on the historical processing task data, and the following steps may be specifically adopted, namely: Obtaining a set of computing resource consumption corresponding to each processing task in the historical processing task data; For each processing task, determine the processing resource segment set corresponding to the processing task according to the corresponding computing resource consumption set, and then obtain the processing resource segment set corresponding to each processing task; Connect the processing resource segment set of each processing task with the corresponding resource segment to obtain all resource management blocks; All resource management blocks are combined into a resource management pool for football match data processing.

[0024] In specific implementation, the computing resource consumption set corresponding to each processing task in the historical processing task data is obtained, that is: the computing resource consumption of each processing task in different football match cycles is obtained from the historical processing task data, so as to obtain the computing resource consumption set of each processing task.

[0025] It should be noted that the computing resource consumption of different football match cycles of each processing task can be obtained through the football match database, where the computing resource consumption set includes multiple computing resource consumptions. The computing resource consumption in this application represents the amount of computing resources consumed in the process of performing data processing, analysis, calculation and other tasks.

[0026] In some embodiments, the following steps may be used to determine the processing resource segment set corresponding to the processing task according to the corresponding computing resource consumption set, namely: Arrange the computing resource consumption in the computing resource consumption set in ascending order to obtain a computing resource consumption sequence; Determining a plurality of division points of the computing resource consumption sequence; The computing resource consumption sequence is divided by all the division points to obtain a set of processing resource segments corresponding to the processing task.

[0027] In the specific implementation, multiple division points of the computing resource consumption sequence are determined, that is: the computing resource consumption sequence is obtained, the computing resource consumption at one-quarter of the computing resource consumption sequence from the front to the back is used as the first division point, the computing resource consumption at one-half of the computing resource consumption sequence from the front to the back is used as the second division point, and the computing resource consumption at three-quarters of the computing resource consumption sequence from the front to the back is used as the third division point.

[0028] In a specific implementation, the computing resource consumption sequence is divided by all the partitioning points to obtain a set of processing resource segments corresponding to the processing task, that is: first, all the partitioning points are obtained, and the interval from the smallest computing resource consumption in the computing resource consumption sequence to the first partitioning point is taken as the first processing resource segment, the interval from the first partitioning point to the second partitioning point is taken as the second processing resource segment, the interval from the second partitioning point to the third partitioning point is taken as the third processing resource segment, and the interval from the third resource segment to the maximum computing resource consumption after arrangement is taken as the fourth processing resource segment; finally, all the processing resource segments are combined to obtain a set of resource segments corresponding to the processing task.

[0029] It should be noted that the processing resource segment set in the present application includes multiple processing resource segments, wherein the processing resource segments represent processing resource segments divided by computing resource consumption, and are used to divide the levels of computing resource consumption.

[0030] In specific implementation, the processing resource segment set of each processing task is connected with the corresponding resource segments to obtain all resource management blocks, that is: the processing resource segments of the same level in the processing resource segment set of each processing task are connected correspondingly to obtain all resource management blocks. For example, the first processing resource segment in the processing resource segment set of each processing task is connected to a processing data block, the second processing resource segment in the processing resource segment set of each processing task is connected to a processing data block, the third processing resource segment in the processing resource segment set of each processing task is connected to a processing data block, and the fourth processing resource segment in the processing resource segment set of each processing task is connected to a processing data block to obtain all resource management blocks. The connection can be performed by sorting connection, that is, hierarchical sorting is performed first, and then the processing resource segments of the same level are connected. In addition, in other embodiments, the processing resource segment set of each processing task can also be connected to the corresponding resource segments by using range connection, hash connection and other methods, which are not limited here.

[0031] It should be noted that the resource management pool represents a collection of multiple resource management blocks, that is, the resource management pool includes multiple resource management blocks. In the present application, a resource management block represents a block connected by processing resource segments of the same level of different processing tasks. In the processing of big data of football matches, the computing resources consumed when processing data from different periods are inconsistent, and the different processing tasks also make the consumed computing resources different. Each data processing goes through different processing tasks, and there is also a resource competition relationship between different processing tasks. Therefore, the present application divides the computing resources into levels by analyzing the consumption of computing resources by different historical processing tasks, and connects the same-level processing resource segments of different processing tasks, which can eliminate the competition for computing resources, improve the precise allocation of computing resources, and thus enhance the processing performance of data processing.

[0032] In step 102, the transmission response data of the current football match data acquisition terminal is obtained, and the downward capture response amount and the upward transmission response amount of the current football match data acquisition terminal are extracted through the transmission response data. The current data backlog loss is obtained based on the analysis of the downward capture response amount and the upward transmission response amount.

[0033] In specific implementation, the transmission response data of the current football match data acquisition terminal can be obtained through the real-time data display function of the football match data processing system, and the transmission response data includes transmission response values ​​at different time points, wherein the transmission response value represents the data response amount at different times during the data transmission process of the football match data acquisition terminal, that is, the data response amount of the football match data acquisition terminal at any time point in the time period from the collection start time point to the cache end time point. In this application, the transmission response value is characterized by the data amount, and the time period includes the data collection stage, the data caching stage, and the data processing stage, wherein the data caching stage and the data processing stage can be combined into the data upload stage. In addition, the transmission response value can also be the result value of the request, the status information value, the error information value or other related content, which is used to inform the sender or the client of the processing status of the request, which is not limited here.

[0034] In some embodiments, the following steps may be used to extract the downward capture response amount and the upward transmission response amount of the current football match data collection terminal through the transmission response data: Acquiring the transmission response data; Determining a response split point for the transmission response data; Splitting the transmission response data according to the response splitting point to obtain first transmission response data and second transmission response data; Determining the downward capture response amount of the current football match data collection terminal according to the first transmission response data; The upward transmission response amount of the current football match data collection terminal is determined by the second transmission response data.

[0035] In specific implementation, the response split point of the transmission response data is determined. The response split point can be determined by the time point when the current football match data acquisition terminal completes the acquisition task, that is, the time point when the current football match data acquisition terminal completes the acquisition task is used as the response split point. It should be noted that the response split point in the present application represents the time point used to split the transmission response data. By determining the response split point, feature extraction of different transmission stages can be effectively performed.

[0036] In specific implementation, the transmission response data is divided according to the response division point to obtain first transmission response data and second transmission response data, that is: taking the response division point as the dividing point, the transmission response data is divided into two sections, and the first half of the transmission response data is used as the first transmission response data, and the second half of the transmission response data is used as the second transmission response data. It should be noted that the first transmission response data in this application represents the first half of the transmission response data of the transmission response data with the response division point as the dividing point, and the second transmission response data in this application represents the second half of the transmission response data of the transmission response data with the response division point as the dividing point.

[0037] In specific implementation, the downward capture response amount of the current football match data acquisition terminal is determined by the first transmission response data, that is: the standard deviation of all transmission response values ​​in the first transmission response data is calculated, and the standard deviation calculation result is used as the downward capture response amount of the current football match data acquisition terminal. In addition, in other embodiments, other calculation methods can also be used to calculate the downward capture response amount of the current football match data acquisition terminal, which is not limited here. It should be noted that the downward capture response amount in the present application represents the change in the amount of data collected by the current football match data acquisition terminal during the time period of executing the collection task.

[0038] In specific implementation, the upward transmission response amount of the current football match data acquisition terminal is determined by the second transmission response data, that is: the standard deviation of all transmission response values ​​in the second transmission response data is calculated, and the standard deviation calculation result is used as the upward transmission response amount of the current football match data acquisition terminal. In addition, in other embodiments, other calculation methods can also be used to calculate the upward transmission response amount of the current football match data acquisition terminal, which is not limited here. It should be noted that the upward transmission response amount in this application represents the change in the amount of data collected during the time period when the current football match data acquisition terminal executes data upload.

[0039] In some embodiments, reference Figure 2 As shown, this figure is an exemplary flow chart of determining data backlog loss according to some embodiments of the present application. In this embodiment, the current data backlog loss is obtained by analyzing the downward capture response amount and the upward transmission response amount, which can be implemented by the following steps: First, in step 1021, the capture anti-interference factor when the current football match data collection terminal captures football match big data is determined; Secondly, in step 1022, the transmission anti-interference factor when the current football match data collection terminal transmits the football match big data is determined; Further, in step 1023, a downward capture response confidence amount is determined by multiplying the downward capture response amount by the capture anti-disturbance factor; Then, in step 1024, an upward transmission response confidence amount is determined by multiplying the upward transmission response amount by the transmission anti-interference factor; Finally, in step 1025, the ratio of the downward capture response confidence to the upward transmission response confidence is used as the current data backlog loss.

[0040] In specific implementation, determine the capture anti-interference factor of the current football match data acquisition terminal when capturing football match big data. The capture anti-interference factor can be set according to the acquisition performance of the football match data acquisition terminal, and is usually set to a number between 0 and 1. It can be set according to actual needs. The stronger the acquisition performance, the larger the capture anti-interference factor is set. The weaker the acquisition performance, the smaller the capture anti-interference factor is set. In this application, the capture anti-interference factor is set to 0.65. The capture anti-interference factor in this application represents the ability of the data to resist various interferences and difficulties when capturing football match big data, that is, the stronger the resistance, the larger the anti-interference factor is set.

[0041] In specific implementation, the transmission anti-interference factor of the current football match data acquisition terminal when capturing and transmitting football match big data is determined. The transmission anti-interference factor can be set according to the transmission performance of the football match data acquisition terminal, and is usually set to a number between 0 and 1. It can be set according to actual needs. The stronger the transmission performance, the larger the transmission anti-interference factor is set. The weaker the transmission performance, the smaller the transmission anti-interference factor is set. In this application, the transmission anti-interference factor is set to 0.65. The transmission anti-interference factor in this application represents the ability of data to resist various interferences and difficulties when transmitting football match big data, that is, the stronger the resistance, the larger the transmission anti-interference factor is set.

[0042] It should be noted that the downward capture response confidence in the present application represents the value of the downward capture response after being adjusted by the capture anti-interference factor, which is used to truly represent the change in the amount of data collected by the current football match data acquisition terminal during the time period when the collection task is executed. The upward transmission response confidence in the present application represents the value of the upward transmission response after being adjusted by the transmission anti-interference factor, which is used to truly represent the change in the amount of data collected during the time period when the current football match data acquisition terminal executes data upload.

[0043] It should also be noted that the data backlog loss in the present application refers to the amount of information lost due to data backlog during the data upload process, which is used to measure the degree of loss in the current football match big data transmission process, that is, the greater the data backlog loss, the lower the network transmission efficiency, the greater the loss in the current football match big data transmission process, and the poorer the network quality. The smaller the data backlog loss, the higher the network transmission efficiency, the lower the loss in the current football match big data transmission process, and the better the network quality. In the football match big data processing, the data needs to be processed in accordance with the process sequence, that is, from capturing data, caching data to data processing, etc. However, when there is a difference between the amount of data transmitted to the cache area by the football match big data and the actual amount of data captured, if the data capture amount is greater than the data transmission amount, it means that the capture speed exceeds the transmission speed, which may cause the cache area to be blocked, resulting in an increase in the loss in the current football match big data transmission process. Therefore, the degree of loss in the current football match big data transmission process can be measured by the ratio of the downward capture response confidence to the upward transmission response confidence.

[0044] In step 103, the capture time span and cache time span of the current football match big data are determined, and then the data freshness of the data cache when storing the football match big data is determined according to the capture time span and the cache time span.

[0045] In some embodiments, the following steps may be used to determine the capture time span and cache time span of the current football match big data, namely: Get the capture arrival time point and capture sending time point when capturing the current football match big data; Determine the capture time span by the difference between the capture sending time point and the capture time arrival point; Get the cache arrival time and cache sending time of the current football match big data cache; The cache time span is determined by the difference between the cache sending time point and the cache arrival time point.

[0046] It should be noted that the capture arrival time point in the present application indicates the time point when the capture data request is sent, marking the moment when the capture data request starts to be processed; the capture sending time point in the present application indicates the time point when the capture data result is sent to the system, marking the completion of the entire capture process; the cache arrival time point in the present application indicates the time point when the cache request is sent, marking the moment when the cache request starts to be processed; the cache sending time point in the present application indicates the time point when the cache result is sent to the system, marking the completion of the entire cache process; wherein, the capture arrival time point, the capture sending time point, the cache arrival time point and the cache sending time point can all be obtained through the football event database, which will not be repeated here.

[0047] It should also be noted that the capture time span in this application represents the time span of the system executing the data capture process, and the cache time span in this application represents the time span of the system executing the data caching process. In the process of processing big data of football matches, the time spans of different stages affect the allocation and management of resources. The longer the time span, the more computing resources are required, and the shorter the time span, the fewer computing resources are required. Different time spans may also show different trends, patterns and laws. By analyzing the time spans of different stages, we can more accurately understand the characteristics of the data and make reasonable decisions and strategies.

[0048] In some embodiments, the data freshness of the data cache area when storing the football match big data can be determined according to the capture time span and the cache time span by the following steps, namely: Obtaining the capture time span and the cache time span; Determine the cache validity limit of the football match data collection terminal when caching data; The timeliness of the data cache area when storing the football match big data is analyzed by combining the capture time span, the cache time span and the cache time limit, so as to obtain the data freshness of the data cache area when storing the football match big data.

[0049] In specific implementation, the cache time limit of the football match data collection terminal when caching data can be set according to the importance and frequency of use of the data. Since the football match big data in this application is real-time data, the cache time limit is set to 3 seconds. It should be noted that the cache time limit in this embodiment represents the time limit for the data to be considered "valid" or "fresh" in the cache.

[0050] In specific implementation, the capture time span, the cache time span and the cache time limit are combined to analyze the timeliness of the data cache area when storing the football match big data, and then the data freshness of the data cache area when storing the football match big data is obtained, that is: a negative exponential function is constructed by the cache time limit, the absolute difference between the capture time span and the cache time span is calculated, and the absolute difference calculation result is input as an input parameter into the negative exponential function, and the negative exponential function outputs the data freshness of the data cache area when storing the football match big data. In addition, in other embodiments, other calculation methods can also be used to calculate the data freshness of the data cache area when storing the football match big data.

[0051] In a specific implementation, a negative exponential function is constructed by the cache time limit, that is, the cache time limit is used as the base of the negative exponential function to thereby construct the negative exponential function.

[0052] It should be noted that the data freshness in the present application represents an indicator for measuring the timeliness of real-time processing of big data of football matches, that is, the greater the data freshness, the better the timeliness of real-time processing of big data of football matches, and the smaller the data freshness, the smaller the timeliness of real-time processing of big data of football matches. In the processing of football match data, different stages need to be passed before data processing, and the processing time of different stages is different, and the computing resources consumed are also different. The processing time of the previous stage may be greater than or less than the processing time of the current stage, that is, there is a time difference between the processing of different stages. Due to the existence of the time difference, the computing resources within the system will be consumed at the time gap, resulting in part of the computing resources being wasted. Therefore, by determining the data freshness of the data cache when storing the big data of football matches, the processing resources required for the current big data of football matches can be better analyzed, thereby avoiding waste and idleness of processing resources.

[0053] In step 104, the resource adaptation coefficient for processing the current football match big data is determined according to the data backlog loss and the data freshness, and the resource processing adaptation block for the current football match big data is extracted from the resource management pool based on the resource adaptation coefficient.

[0054] In some embodiments, the resource adaptation coefficient for processing the current football match big data can be determined according to the data backlog loss and the data freshness by using the following steps, namely: Obtaining the data backlog loss and the data freshness; Determine the data backlog impact coefficient of the football match data processing system; Determine the data timeliness impact coefficient of the football match data processing system; The resource adaptation coefficient when processing the current football match big data is determined by the data backlog loss, the data freshness, the data backlog impact coefficient and the data timeliness impact coefficient.

[0055] During specific implementation, the data backlog impact coefficient of the football match data processing system is determined. The data backlog impact coefficient can be set according to actual needs. For example, it can be set according to the degree of influence of data backlog loss on the football match data processing system. Since the data backlog loss in this application has a greater impact on the football match data processing system, the data backlog impact coefficient is set to 0.67. In addition, in other embodiments, the data backlog impact coefficient can also be set according to actual needs, wherein the data backlog impact coefficient in this example represents a weight coefficient for adjusting the data backlog loss.

[0056] During specific implementation, the data timeliness impact coefficient of the football match data processing system is determined. The data timeliness impact coefficient can be set according to actual needs. For example, it can be set according to the degree of influence of data freshness on the football match data processing system. Since the impact of data freshness on the football match data processing system in this application is relatively small, the data timeliness impact coefficient is set to 0.33. In addition, in other embodiments, the data timeliness impact coefficient can also be set according to actual needs, wherein the data timeliness impact coefficient in this example represents a weight coefficient for adjusting the data freshness.

[0057] In specific implementation, the resource adaptation coefficient when processing the current football match big data is determined by the data backlog loss, the data freshness, the data backlog influence coefficient and the data timeliness influence coefficient, that is: the data backlog loss is multiplied by the data backlog influence coefficient to obtain a first product result, the data freshness is multiplied by the data timeliness influence coefficient to obtain a second product result, and the sum of the first product result and the second product result is used as the resource adaptation coefficient when processing the current football match big data. In addition, in other embodiments, other calculation methods can also be used to calculate the resource adaptation coefficient when processing the current football match big data, which is not limited here.

[0058] It should be noted that the resource adaptation coefficient in the present application represents an indicator for measuring the degree of match between data processing resources and data requirements. In the present application, the resource adaptation coefficient is used to select adapted resource blocks. Different adaptation coefficients adapt different resource blocks. The larger the resource adaptation coefficient, the higher the level of the adapted resource block. By adjusting the resource adaptation coefficient, the data processing resources required for the current football match big data can be effectively extracted, thereby improving the utilization rate of data processing resources.

[0059] It should be noted that in the process of processing football match data, the amount of data at different stages and the time span of different stages will have an impact on the consumption of computing resources, that is, the increase in the amount of data will lead to data inflow congestion, resulting in increased losses in the transmission process, and thus increased consumption of computing resources. A longer time span also requires more computing resources. A long time span means that more data points or more time series data need to be processed, which will also increase the complexity of the calculation and resource consumption. Therefore, the amount of data and the time span at different stages before data processing can be analyzed to pre-determine the resource adaptation coefficient of the current football match big data, and then select the resource block according to the resource adaptation coefficient.

[0060] In some embodiments, reference Figure 3As shown in FIG. 1 , this figure is an exemplary flow chart of determining a resource processing adaptation block according to some embodiments of the present application. In this embodiment, the resource processing adaptation block for extracting the current football match big data from the resource management pool based on the resource adaptation coefficient can be implemented by the following steps: In step 1041, all resource management blocks in the resource management pool are obtained; In step 1042, a mapping identification section of each resource management block is determined; In step 1043, the resource adaptation coefficient is mapped to the mapping identification segment of each resource management block. If the mapping is successful, the resource management block corresponding to the mapping identification segment is used as the resource processing adaptation block of the current football match big data.

[0061] In specific implementation, the mapping identification segment of each resource management block is determined, that is, the number of resource management blocks in the resource management pool is obtained, the resource management blocks in the resource management pool are arranged in ascending order by level, and each level resource management block is segment-compressed in sequence according to the number of resource management blocks to obtain a mapping identification segment corresponding to each level resource management block. For example, after the resource management blocks in the resource management pool are arranged in a hierarchical manner, there are four levels: a first-level resource management block, a second-level resource management block, a third-level resource management block and a fourth-level resource management block. The four levels of resource management blocks are evenly compressed to the range of 0 to 1 to obtain a mapping identification segment corresponding to each level of resource management block: [0, 0.25), [025, 0.5), [05, 0.75), [0.75, 1], that is, one level resource management block corresponds to one mapping identification segment.

[0062] In specific implementation, the resource adaptation coefficient is mapped to the mapping identification segment of each resource management block, that is: for each mapping identification segment of the resource management block, it is determined whether the value of the resource adaptation coefficient is within the value of the mapping identification segment. If so, the mapping is successful, otherwise the mapping fails.

[0063] It should be noted that the mapping identification segment in the present application represents the identification segment between 0 and 1 in which resource management blocks of different levels are mapped. The purpose of determining the mapping identification segment is to speed up the adaptation of resource allocation in the data processing process, reduce the complexity of adaptation, and improve the efficiency of adaptation.

[0064] It should also be noted that the resource processing adaptation block in this application represents the resource management block adapted for the current football match big data processing. By determining the resource processing adaptation block, the processing resources of the current football match big data can be effectively allocated, thereby improving the processing efficiency of the football match big data.

[0065] In step 105, corresponding data resource configuration is performed on the current football match big data based on the resource configuration information of the resource processing adaptation block.

[0066] In some embodiments, the following steps may be specifically adopted to configure the corresponding data resources for the current football match big data based on the resource configuration information of the resource processing adapter block, namely: Acquire resource configuration information of the resource processing adapter block, wherein the resource configuration information includes processing resource sections of different processing tasks; Get different processing tasks for current football match big data; Configure corresponding processing resource segments for different processing tasks.

[0067] It should be noted that the different processing tasks are data cleaning tasks, data integration tasks, and data storage tasks, which are not limited here.

[0068] It should also be noted that the data resource configuration in this application represents the process of configuring corresponding processing resource segments for the processing task of the current football match big data.

[0069] In the specific implementation, corresponding processing resource segments are configured for different processing tasks, namely: the data cleaning task corresponds to the processing resource segment configured for data cleaning, the data integration task corresponds to the processing resource segment configured for data integration, and the data storage task corresponds to the processing resource segment configured for data storage, which will not be elaborated here.

[0070] It should also be noted that the data resource configuration in the present application represents the allocation of corresponding processing resource segments for different data processing tasks of football match big data, that is: the data cleaning task corresponds to the processing resource segment for configuring data cleaning, the data integration task corresponds to the processing resource segment for configuring data integration, and the data storage task corresponds to the processing resource segment for configuring data storage. In the processing of football match data, when large-scale data appears, different processing tasks need to consume different computing resources. For football match big data that is more seriously interfered, more computing resources may be required in the data cleaning task. For football match big data obtained from different data sources, more computing resources are required in data integration. For the need to predict football match big data, more computing resources are required in data storage. Therefore, the present application pre-reasonably configures the computing resources. For each processing of football match big data, the current football match big data can be pre-analyzed for data loss and time span, thereby comprehensively representing the resource configuration required in the current situation according to the data loss and time span, thereby achieving effective processing of football match big data and avoiding waste of computing resources.

[0071] In addition, in another aspect of the present application, in some embodiments, the present application provides a football match data processing system based on big data, referring to Figure 4 , which is a schematic diagram of the structure of a football match data processing system based on big data according to some embodiments of the present application. The football match data processing system 200 based on big data includes: an acquisition module 201, a processing module 202 and an execution module 203, which are described as follows: Acquisition module 201, in this application, acquisition module 201 is mainly used to acquire historical processing task data of football match big data, and set a resource management pool for football match data processing based on the historical processing task data; Processing module 202, in the present application, the processing module 202 is mainly used to obtain the transmission response data of the current football match data collection terminal, extract the downward capture response amount and the upward transmission response amount of the current football match data collection terminal through the transmission response data, and analyze the current data backlog loss according to the downward capture response amount and the upward transmission response amount; It should be noted that the processing module 202 in the present application is also used to determine the capture time span and cache time span of the current football match big data, and then determine the data freshness of the data cache when storing the football match big data according to the capture time span and the cache time span; In addition, the processing module 202 is also used to determine the resource adaptation coefficient when processing the current football match big data according to the data backlog loss and the data freshness, and extract the resource processing adaptation block of the current football match big data from the resource management pool based on the resource adaptation coefficient; Execution module 203, in this application, execution module 203 is mainly used to perform corresponding data resource configuration on the current football match big data based on the resource configuration information of the resource processing adaptation block.

[0072] In addition, the present application also provides a computer device, which includes a memory and a processor, the memory stores code, and the processor is configured to obtain the code and execute the above-mentioned football match data processing method based on big data.

[0073] In some embodiments, reference Figure 5 , which is a schematic diagram of the structure of a computer device for applying a football match data processing method based on big data according to some embodiments of the present application. The football match data processing method based on big data in the above embodiment can be Figure 5 The computer device 300 shown in the figure is implemented, and the computer device 300 includes at least one processor 301, a communication bus 302, a memory 303 and at least one communication interface 304.

[0074] The processor 301 may be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more processors for controlling the execution of the football match data processing method based on big data in the present application.

[0075] Communication bus 302 may include a pathway for transmitting information between the above-mentioned components.

[0076] The memory 303 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 303 may exist independently and be connected to the processor 301 through the signal bus 302. The memory 303 may also be integrated with the processor 301.

[0077] The memory 303 is used to store the program code for executing the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the program code stored in the memory 303. The program code may include one or more software modules. The determination of the football match data processing method based on big data in the above embodiment can be implemented by the processor 301 and one or more software modules in the program code in the memory 303.

[0078] The communication interface 304 uses any transceiver or other device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0079] In a specific implementation, as an embodiment, a computer device may include multiple processors, each of which may be a single-CPU processor or a multi-CPU processor. The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0080] The above-mentioned computer device may be a general-purpose computer device or a special-purpose computer device. In a specific implementation, the computer device may be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device or an embedded device. The embodiment of the present application does not limit the type of computer device.

[0081] In addition, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned football match data processing method based on big data.

[0082] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0083] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A football match data processing method based on big data, characterized in that: The steps include: Acquire historical processing task data of football match big data, and set a resource management pool for football match data processing based on the historical processing task data; Acquire the transmission response data of the current football match data collection terminal, extract the downward capture response amount and the upward transmission response amount of the current football match data collection terminal through the transmission response data, and analyze the current data backlog loss according to the downward capture response amount and the upward transmission response amount; Determine the capture time span and cache time span of the current football match big data, and then determine the data freshness of the data cache area when storing the football match big data according to the capture time span and the cache time span; Determine a resource adaptation coefficient when processing the current football match big data according to the data backlog loss and the data freshness, and extract a resource processing adaptation block of the current football match big data from the resource management pool based on the resource adaptation coefficient; Based on the resource configuration information of the resource processing adapter block, corresponding data resource configuration is performed on the current football match big data.

2. The method according to claim 1, characterized in that The resource management pool for football match data processing based on the historical processing task data specifically includes: Obtaining a set of computing resource consumption corresponding to each processing task in the historical processing task data; For each processing task, determine the processing resource segment set corresponding to the processing task according to the corresponding computing resource consumption set, and then obtain the processing resource segment set corresponding to each processing task; Connect the processing resource segment set of each processing task with the corresponding resource segment to obtain all resource management blocks; All resource management blocks are combined into a resource management pool for football match data processing.

3. The method according to claim 2, characterized in that Determining the processing resource segment set corresponding to the processing task according to the corresponding computing resource consumption set specifically includes: Arrange the computing resource consumption in the computing resource consumption set in ascending order to obtain a computing resource consumption sequence; Determining a plurality of division points of the computing resource consumption sequence; The computing resource consumption sequence is divided by all the division points to obtain a set of processing resource segments corresponding to the processing task.

4. The method according to claim 1, characterized in that The current data backlog loss obtained by analyzing the downward capture response amount and the upward transmission response amount specifically includes: Determine the capture anti-interference factor when the current football match data collection terminal captures football match big data; Determine the transmission anti-interference factor when the current football match data collection terminal transmits football match big data; Determining a downward capture response confidence amount by multiplying the downward capture response amount by the capture anti-disturbance factor; Determining an upward transmission response confidence amount by multiplying the upward transmission response amount by the transmission anti-interference factor; The ratio of the downward capture response confidence to the upward transmission response confidence is taken as the current data backlog loss.

5. The method according to claim 1, characterized in that Determining the capture time span and cache time span of the current football match big data specifically includes: Get the capture arrival time point and capture sending time point when capturing the current football match big data; Determine the capture time span by the difference between the capture sending time point and the capture time arrival point; Get the cache arrival time and cache sending time of the current football match big data cache; The cache time span is determined by the difference between the cache sending time point and the cache arrival time point.

6. The method according to claim 1, characterized in that The resource processing adaptation block for extracting the current football match big data from the resource management pool based on the resource adaptation coefficient specifically includes: Obtain all resource management blocks in the resource management pool; Determine a mapping identification section for each resource management block; The resource adaptation coefficient is mapped to the mapping identification section of each resource management block. If the mapping is successful, the resource management block corresponding to the mapping identification section is used as the resource processing adaptation block of the current football match big data.

7. The method according to claim 1, characterized in that The resource management pool includes multiple resource management blocks.

8. A football match data processing system based on big data, characterized in that: include: An acquisition module, used to acquire historical processing task data of football match big data, and to set a resource management pool for football match data processing based on the historical processing task data; A processing module is used to obtain the transmission response data of the current football match data collection terminal, extract the downward capture response amount and the upward transmission response amount of the current football match data collection terminal through the transmission response data, and analyze the current data backlog loss according to the downward capture response amount and the upward transmission response amount; The processing module is further used to determine the capture time span and cache time span of the current football match big data, and then determine the data freshness of the data cache area when storing the football match big data according to the capture time span and the cache time span; The processing module is further used to determine the resource adaptation coefficient when processing the current football match big data according to the data backlog loss and the data freshness, and extract the resource processing adaptation block of the current football match big data from the resource management pool based on the resource adaptation coefficient; The execution module is used to perform corresponding data resource configuration on the current football match big data based on the resource configuration information of the resource processing adaptation block.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores codes, and the processor is configured to obtain the codes and execute the football match data processing method based on big data as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for processing football match data based on big data as described in any one of claims 1 to 7 is implemented.

Images