Competition information management system and method

Abstract

The invention discloses a competition information management system and method, and relates to the technical field of information management, and the system comprises a competition information collection processing unit, a node fragmentation management unit, a node adaptive management unit, an information storage management unit and a management information output unit. The technical problems that node fragmentation and performance management cannot adapt to competition dynamic characteristics, hierarchical management is not performed during data storage, and storage resources are wasted are solved, the capacity expansion response time is shortened through prediction based on an LSTM + attention mechanism model and competition historical cycle data, and the capacity expansion efficiency is improved through competition type-resource template matching. The resource waste rate is reduced; metadata and business data are separately stored; a centralized shared pool and a local cache are combined; the repeated storage of the metadata is reduced; and the storage cost and the hot data read-write delay are reduced.

Description

Technical Field

[0001] This invention relates to the field of information management technology, specifically to a competition information management system and method. Background Technology

[0002] University student academic competitions are a special form of examination and training that goes beyond the scope of textbooks. They are important activities to improve university students' ability to apply textbook theories, enhance their practical skills, stimulate their potential, select outstanding talents, and broaden their knowledge.

[0003] According to patent application number CN202310961874.2, an information management platform for college students' academic competitions is disclosed, including: a team management subsystem, a competition activity management subsystem, an operation and maintenance team management subsystem, a big data analysis subsystem, and a back-end database management subsystem.

[0004] However, existing competition information management systems often use static node sharding rules, which cannot adapt to the dynamic characteristics of competitions. Performance analysis relies solely on hardware metrics without considering the correlation between competition business operations, leading to inaccurate node classification, low task scheduling efficiency, failure to separate metadata from business data, and repeated storage of the same competition information across multiple shard nodes, resulting in severe storage redundancy. Duplicate data identification relies solely on general hashes without considering competition affiliation, team association, and other business rules, leading to a higher false positive rate. Furthermore, storage media are not layered according to data lifecycle, resulting in significant waste of high-performance storage resources. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a competition information management system and method that solves the problems of node sharding and performance management failing to adapt to the dynamic characteristics of competitions, and the lack of hierarchical management during data storage, resulting in wasted storage resources.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a competition information management system, comprising: The competition information collection and processing unit is used to collect the competition information of the participants, perform data cleaning on the competition information to obtain preprocessed competition information, perform data fragmentation processing on the preprocessed competition information according to the user area to generate data fragment information, and transmit it to the node fragmentation management unit. The node sharding management unit is used to perform node allocation management based on data sharding information, obtain and quantify the CPU utilization, memory usage, query response time and data write frequency of each processing node, and calculate a comprehensive evaluation value to classify the processing nodes into high-performance nodes and low-performance nodes according to the comprehensive evaluation value. Obtain all high-performance nodes and their corresponding loads, compare the node loads with load thresholds, classify them into high-load nodes and low-load nodes, obtain the processing tasks of high-load nodes, divide the processing tasks into high-priority tasks and low-priority tasks according to the timeliness and impact of the competition, migrate high-priority tasks to low-load nodes, generate sharding management information and transmit it to the node adaptive management unit. The node adaptive management unit is used to process sharding management information, monitor the real-time load of high-performance nodes after sharding, and if the real-time load is greater than the load threshold, perform elastic scaling processing, calculate the number of nodes to be expanded, select target nodes that meet the conditions from the idle node pool to perform scaling, generate node scaling information and transmit it to the information storage management unit and management information output unit.

[0007] As a further embodiment of the present invention, it also includes an information storage management unit, which is used to receive node expansion information transmitted by the node adaptive management unit, split the information into metadata and business data, divide the duplicate data in the two into structured business data, semi-structured data and unstructured data according to the competition data type, perform deduplication storage management and generate storage management information respectively. The management information output unit is used to obtain storage information from the information storage management unit, display the information according to user needs, and perform elastic expansion processing according to the node expansion information.

[0008] As a further embodiment of the present invention, the data collection method can be obtained through student terminals, third-party system integration, or data entry by organizers. Data cleaning includes data collection format standardization, heterogeneous information format conversion, field format verification, and intelligent deduplication and cleaning.

[0009] As a further aspect of the present invention, the method for calculating the comprehensive evaluation value is as follows: Quantify and assign values ​​to CPU utilization to obtain the actual CPU utilization U and the normal threshold A. CPU and warning threshold B CPU If U≤A CPU Then S CPU =1-(U / A CPU If A × 0.3, CPU <U≤B CPU S CPU =0.7-[(UA CPU ) / (B CPU -A CPU )]×0.5, if U>B CPU S CPU =0.2-[(UB CPU ) / (100-B CPU )]×0.2; Quantify and assign values ​​to memory usage to obtain the actual memory usage M and the normal threshold A. 内存 and warning threshold B 内存 If M≤A 内存 S M =1-(M / A 内存） ×0.2, if A 内存 <M≤B 内存 S M =0.8-[(MA 内存） / (B 内存 -A 内存 ] × 0.6, if M > B 内存 S M =0.2-[(MB 内存） / (100-B 内存） ×0.2; The query response time is quantified and assigned values. Using time t as a period, the corresponding query response time is obtained, and its mean T is calculated as a standard. Simultaneously, a normal threshold A is obtained. 时间 Warning threshold B 时间 and the abnormal threshold C 时间 If T≤A 时间 S T =1-(T / A 时间） ×0.2, if A 时间 <T≤B 时间 S T =0.8-[(TA 时间） / (B 时间 -A 时间） ]×0.6, if T>B 时间 S T =0.2-[(TB 时间） / (C 时间 -B 时间） ×0.2; The data write frequency is quantized and assigned a value to obtain the actual write frequency value W. Simultaneously, the lower limit threshold L, the normal range N, and the upper limit threshold H corresponding to the processing node are obtained. If L≤W≤N 上限 S W =1-|WN median| / (N) 上限 -L), if N 上限 <W≤H,S W =0.8-[(WN 上限） / (HN 上限） ]×0.6, if W <L:S W =0.8-[(LW) / L]×0.6; The performance metrics obtained from the quantitative assignment are weighted and summed, and the comprehensive evaluation value is calculated according to the formula: S CPU ×a1+SM ×a2+S T ×a3+S W ×a4, where a1, a2, a3 and a4 are the corresponding weight coefficients.

[0010] As a further aspect of the present invention, the method for classifying high-load nodes and low-load nodes is as follows: Next, processing nodes with a comprehensive evaluation value greater than the performance index threshold are classified as high-performance nodes, and processing nodes with a comprehensive evaluation value less than the performance index threshold are classified as low-performance nodes. Simultaneously, all high-performance nodes and their corresponding node loads are obtained, and the node loads are compared with the corresponding load thresholds. If the node load is greater than the load threshold, it is classified as a high-load node; otherwise, it is classified as a low-load node.

[0011] As a further aspect of the present invention, the method for generating fragment management information is as follows: For the high-load nodes identified by classification, obtain their corresponding processing tasks, classify the processing tasks into high-priority tasks and low-priority tasks, and transfer high-priority tasks to low-load nodes. Specifically, select target nodes from the low-load node pool that are in the same region, have the same configuration, and have idle resources ≥ 120% of the task requirements. Pre-start the task dependency environment on the target node, synchronize the task status through a dual-write buffer mechanism, and switch the route to the target node when the task execution reaches the atomic breakpoint. The original node stops processing. After the migration is completed, verify the consistency of the task status and generate shard management information.

[0012] As a further aspect of the present invention, the method for calculating the number of expansion nodes is as follows: Historical periodic data is acquired and input into an LSTM+attention mechanism model. The model outputs a predicted load increment. Based on the predicted load increment and the single node's carrying capacity, the number of expansion nodes is dynamically calculated. The expansion node count is calculated using the formula: Where 1.2 is the redundancy coefficient, the single node capacity represents the maximum load that an existing node can bear without affecting service performance, and ceil represents the rounding up function to obtain the number of nodes to be expanded.

[0013] As a further aspect of the present invention, the method for generating node expansion information is as follows: Prioritize selecting nodes from the idle node pool that meet the expansion requirements. The nodes must be in the same region and have the same configuration as the current high-performance nodes, and the idle resources must be at least 120% of the resources required for expansion. After selecting the target node, pre-start the task dependency environment on the node to ensure that the environment configuration is consistent with the original node.

[0014] As a further aspect of the present invention, the method for performing deduplication storage management and generating storage management information is as follows: The deduplication management method for structured business data is as follows: within the same shard, if the event ID + user ID / team ID are absolutely duplicated, or the core information hash is the same and the event ID is the same, when writing data, first query the deduplication index table within the shard. If an absolute duplicate is matched, directly return that the data already exists and refuse to write it again; if a suspected duplicate is matched, trigger manual verification, and the verification result will be updated synchronously in the index table. The deduplication management method for semi-structured and unstructured data is as follows: within the same shard, if the content hash is completely identical and the associated subject + event ID is the same, it is judged as duplicate, and duplicate data is not stored repeatedly, but only a reference pointer is added to the current business record; if the content hash is identical but the associated subject is different, it is judged as legally similar and is allowed to be stored independently.

[0015] A method for managing competition information, which specifically includes the following steps: Step 1: Collect competition information from participants through three methods: participant-side, third-party system integration, or organizer input. After data cleaning, preprocessed competition information is obtained. Data cleaning includes standardization of collection format, conversion of heterogeneous information format, field format validation, and intelligent deduplication. Data fragmentation is performed on the preprocessed competition information according to user region to generate data fragment information. Step 2: Perform node allocation management based on data sharding information, obtain and quantify the CPU utilization, memory usage, query response time and data write frequency of each processing node, and calculate the comprehensive evaluation value. Based on the comprehensive evaluation value, classify the processing nodes into high-performance nodes and low-performance nodes. Step 3: Obtain all high-performance nodes and their corresponding loads. Compare the node loads with the load thresholds to classify them into high-load and low-load nodes. Obtain the processing tasks of high-load nodes. Divide the processing tasks into high-priority and low-priority tasks according to the timeliness and impact of the competition. Migrate the high-priority tasks to low-load nodes and generate shard management information. Step 4: Monitor the real-time load of high-performance nodes after sharding. If the real-time load is greater than the load threshold, perform elastic scaling, calculate the number of nodes to be scaled up, select target nodes that meet the conditions from the idle node pool to perform scaling up, and generate node scaling up information. Step 5: Split the information into metadata and business data. According to the data type of the competition, divide the duplicate data in the two into structured business data, semi-structured data, and unstructured data. Perform deduplication storage management on each and generate storage management information.

[0016] This invention provides a competition information management system and method. Compared with the prior art, it has the following advantages: This invention solves the problem of incomplete information from a single collection channel by supporting multi-source heterogeneous data collection. Through standardized collection formats, field format validation, and intelligent deduplication, it reduces data error rates and redundancy, laying a high-quality data foundation for subsequent business processing.

[0017] This invention employs dynamic region-level granularity sharding to reduce the data volume differences among shards and improve load balancing. Through quantitative assignment and weighted analysis of CPU, memory, query response time, and data write frequency, it achieves accurate classification of node performance and improves the accuracy of high-performance node identification. High-priority task migration adopts a dual-write buffer + atomic breakpoint switching mechanism to shorten business interruption time.

[0018] This invention shortens the expansion response time based on the LSTM+attention mechanism model and prediction of historical competition data. It reduces resource waste by matching competition type with resource templates, stores metadata and business data separately, reduces duplicate metadata storage by combining a centralized shared pool and local caching, improves the deduplication rate of structured and unstructured data by deduplication logic bound to competition business rules, and reduces storage costs and hot data read / write latency by storing data in layers according to data lifecycle. Attached Figure Description

[0019] Figure 1 This is a block diagram of the competition information management system of the present invention; Figure 2 This is a flowchart of the competition information management method of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example

[0021] Please see Figure 1 This application provides a competition information management system, including: a competition information acquisition and processing unit, a node fragmentation management unit, a node adaptive management unit, an information storage management unit, and a management information output unit, and in conjunction with... Figure 1 It can be seen that the information between the above functional units is transmitted in one direction only.

[0022] The competition information collection and processing unit collects competition information from participants. This information can be collected through participant terminals, third-party system integration, or input by organizers. The unit also performs data cleaning on the competition information, including standardizing the collection format, converting heterogeneous information to new formats, validating field formats, and performing intelligent deduplication. This process yields pre-processed competition information, which is then segmented according to user regions to generate data segment information. Finally, this data is transmitted to the node segment management unit.

[0023] The node sharding management unit is used to allocate and manage corresponding nodes according to data sharding information. It obtains all processing nodes and assigns them unique labels denoted as i, where i = 1, 2, ..., j, and j represents the number of processing nodes. Next, it obtains the performance metrics corresponding to processing node i, including CPU utilization, memory usage, query response time, and data write frequency. The performance of the processing node is then analyzed based on these metrics, using the following specific analysis method: Quantify and assign values ​​to CPU utilization to obtain the actual CPU utilization U and the normal threshold A. CPU and warning threshold B CPU If U≤A CPU Then S CPU =1-(U / A CPU If A × 0.3, CPU <U≤B CPU S CPU =0.7-[(UA CPU ) / (B CPU -A CPU )]×0.5, if U>B CPU S CPU =0.2-[(UB CPU ) / (100-B CPU )]×0.2; Quantify and assign values ​​to memory usage to obtain the actual memory usage M and the normal threshold A. 内存 and warning threshold B 内存 If M≤A 内存 S M =1-(M / A 内存） ×0.2, if A 内存 <M≤B 内存 S M =0.8-[(MA 内存） / (B 内存 -A 内存 ] × 0.6, if M > B 内存 S M =0.2-[(MB 内存） / (100-B 内存）×0.2; The query response time is quantified and assigned values. Using time t as a period, the corresponding query response time is obtained, and its mean T is calculated as a standard. Simultaneously, a normal threshold A is obtained. 时间 Warning threshold B 时间 and the abnormal threshold C 时间 If T≤A 时间 S T =1-(T / A 时间） ×0.2, if A 时间 <T≤B 时间 S T =0.8-[(TA 时间） / (B 时间 -A 时间） ]×0.6, if T>B 时间 S T =0.2-[(TB 时间） / (C 时间 -B 时间） ×0.2; The data write frequency is quantized and assigned a value to obtain the actual write frequency value W. Simultaneously, the lower limit threshold L, the normal range N, and the upper limit threshold H corresponding to the processing node are obtained. If L≤W≤N 上限 S W =1-|WN median| / (N) 上限 -L), if N 上限 <W≤H,S W =0.8-[(WN 上限） / (HN 上限） ]×0.6, if W <L:S W =0.8-[(LW) / L]×0.6; The performance metrics obtained from the quantitative assignment are weighted and summed, and the comprehensive evaluation value is calculated according to the formula: S CPU ×a1+S M ×a2+S T ×a3+S W ×a4, where a1, a2, a3 and a4 are the corresponding weight coefficients, and the specific values ​​are set by the operator. Then, the processing nodes with a comprehensive evaluation value greater than (including equal to) the performance index threshold are classified as high-performance nodes, and the processing nodes with a comprehensive evaluation value less than the performance index threshold are classified as low-performance nodes. Simultaneously, all high-performance nodes and their corresponding loads are acquired, and the node loads are compared with the corresponding load thresholds. If the node load is greater than the load threshold, it is classified as a high-load node; otherwise, it is classified as a low-load node. For the high-load nodes, their corresponding processing tasks are acquired, and priorities are defined according to the timeliness and impact of the competition. Specifically, tasks with timeliness greater than the timeliness threshold and a wide impact are classified as high-priority, and vice versa. The processing tasks are classified into high-priority and low-priority tasks, and high-priority tasks are transferred to low-load nodes. Specifically, target nodes with the same region, same configuration, and idle resources ≥ 120% of the task requirements are selected from the low-load node pool. The task dependency environment is pre-started on the target node, and the task status is synchronized through a dual-write buffer mechanism. When the task execution reaches the atomic breakpoint, the route is switched to the target node, and the original node stops processing. After the migration is completed, the consistency of the task status is verified, and the processing of low-priority tasks is paused. When the load returns to normal, the processing of low-priority tasks continues, and shard management information is generated and then transmitted to the node adaptive management unit.

[0024] The node adaptive management unit processes the acquired sharding management information, monitors the high-performance nodes after sharding, and obtains their corresponding real-time load. If the real-time load still exceeds the load threshold, elastic scaling is performed, and the specific processing method is as follows: Historical periodic data is acquired, including load curves for the 72 hours before the registration deadline and peak time on the first day of judging for the past three competitions. This data is then input into an LSTM+attention mechanism model to output a predicted load increment. Based on the predicted load increment and the single-node capacity, the number of expansion nodes is dynamically calculated using the formula: Number of expansion nodes = Where 1.2 is the redundancy coefficient, the single node capacity represents the maximum load that the existing node can bear without affecting business performance, and ceil represents the rounding function. The number of expansion nodes is calculated and used as the standard for expansion processing. During the expansion process, nodes that meet the expansion requirements are selected from the idle node pool first. The nodes must meet the conditions of being in the same region and having the same configuration as the current high-performance nodes, and the idle resources must be at least 120% of the resources required for expansion. After the target node is selected, the task dependency environment is pre-started on the node to ensure that the environment configuration is consistent with the original node, so as to reduce the adaptation time after task migration. If the real-time load is less than or equal to the load threshold, but the future load is predicted to exceed the threshold based on historical data, pre-expansion processing is performed to add a certain number of nodes in advance to cope with possible future load peaks. The number of nodes to be pre-expanded is calculated based on the predicted load increment and the carrying capacity of a single node, while taking into account a certain redundancy coefficient to ensure the stability and reliability of the system. At the same time, node expansion information is generated and transmitted to the management information output unit.

[0025] The management information output unit is used to perform elastic scaling based on the acquired node scaling information. Example

[0026] As a second embodiment of the present invention, it is implemented based on the first embodiment, and the difference from the first embodiment is as follows: The node adaptive management unit transmits the generated node expansion information to the information storage management unit and stores and manages the information within different nodes. It obtains the information within the nodes and splits it into metadata and business data. The metadata includes event ID, name, organizer, rule text, time nodes, award settings, etc., while the business data includes registration records, work data, review results, etc. The metadata is stored in a centralized shared pool, such as a single node or master-slave architecture, while the business data is stored in each shard node. The shard node only associates metadata with the event ID. At the same time, it identifies duplicate data and divides the data according to the competition data type to obtain structured business data, semi-structured data, and unstructured data, which are then stored and managed separately. Structured business data indicates that within the same shard, the event ID + user ID / team ID are absolutely duplicated, or the core information hashes are the same and the event ID is the same. When writing data, first query the deduplication index table within the shard. If an absolute duplicate is matched, return that the data already exists and reject duplicate writing. If a suspected duplicate is matched, trigger manual verification. The verification result synchronously updates the index table and generates storage management information. Semi-structured and unstructured data refer to data within the same shard where the content hash is completely identical and the associated subject and event ID are the same. When uploading data, the content hash is calculated, and the content hash index library within the shard is queried. If a matching record exists and the associated subject and event ID are the same, it is determined to be duplicated. Duplicate data is not stored repeatedly; only a reference pointer is added to the current business record. At the same time, for cases where the content hash is the same but the associated subject is different, it is determined to be legally similar, allowed to be stored independently, and storage management information is generated. The management information output unit is used to retrieve stored information from the information storage management unit and display and output the information according to the user's needs. At the same time, it supports the information export function, allowing users to export the required information to common file formats such as Excel and PDF, so that users can carry out further processing and analysis. Example

[0027] As a third embodiment of the present invention, the focus is on combining the implementation processes of the first and second embodiments. Example

[0028] Please see Figure 2 This application provides a method for managing competition information, which specifically includes the following steps: Step 1: Collect competition information from participants through three methods: participant-side, third-party system integration, or organizer input. After data cleaning, preprocessed competition information is obtained. Data cleaning includes standardization of collection format, conversion of heterogeneous information format, field format verification, and intelligent deduplication. Data fragmentation is performed on the preprocessed competition information according to user region to generate data fragment information. Step 2: Perform node allocation management based on data sharding information, obtain and quantify the CPU utilization, memory usage, query response time and data write frequency of each processing node, and calculate the comprehensive evaluation value. Based on the comprehensive evaluation value, classify the processing nodes into high-performance nodes and low-performance nodes. Step 3: Obtain all high-performance nodes and their corresponding loads. Compare the node loads with the load thresholds to classify them into high-load and low-load nodes. Obtain the processing tasks of high-load nodes. Divide the processing tasks into high-priority and low-priority tasks according to the timeliness and impact of the competition. Migrate the high-priority tasks to low-load nodes and generate shard management information. Step 4: Monitor the real-time load of high-performance nodes after sharding. If the real-time load is greater than the load threshold, perform elastic scaling, calculate the number of nodes to be scaled up, select target nodes that meet the conditions from the idle node pool to perform scaling up, and generate node scaling up information. Step 5: Split the information into metadata and business data. According to the data type of the competition, divide the duplicate data in the two into structured business data, semi-structured data, and unstructured data. Perform deduplication storage management on each and generate storage management information.

[0029] Some of the data in the above formulas are numerical calculations with dimensions removed, and the contents not described in detail in this specification are all prior art known to those skilled in the art.

[0030] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A competition information management system, characterized in that, include: The competition information collection and processing unit is used to collect the competition information of the participants, perform data cleaning on the competition information to obtain preprocessed competition information, perform data fragmentation processing on the preprocessed competition information according to the user area to generate data fragment information, and transmit it to the node fragmentation management unit. The node sharding management unit is used to perform node allocation management based on data sharding information, obtain and quantify the CPU utilization, memory usage, query response time and data write frequency of each processing node, and calculate a comprehensive evaluation value to classify the processing nodes into high-performance nodes and low-performance nodes according to the comprehensive evaluation value. Obtain all high-performance nodes and their corresponding loads, compare the node loads with load thresholds, classify them into high-load nodes and low-load nodes, obtain the processing tasks of high-load nodes, divide the processing tasks into high-priority tasks and low-priority tasks according to the timeliness and impact of the competition, migrate high-priority tasks to low-load nodes, generate sharding management information and transmit it to the node adaptive management unit. The node adaptive management unit is used to process sharding management information, monitor the real-time load of high-performance nodes after sharding, and if the real-time load is greater than the load threshold, perform elastic scaling processing, calculate the number of nodes to be expanded, select target nodes that meet the conditions from the idle node pool to perform scaling, generate node scaling information and transmit it to the information storage management unit and management information output unit.

2. The competition information management system according to claim 1, characterized in that, It also includes an information storage management unit, which receives node expansion information transmitted by the node adaptive management unit, splits the information into metadata and business data, divides the duplicate data in the two into structured business data, semi-structured data and unstructured data according to the competition data type, performs deduplication storage management and generates storage management information respectively. The management information output unit is used to obtain storage information from the information storage management unit, display the information according to user needs, and perform elastic expansion processing according to the node expansion information.

3. The competition information management system according to claim 1, characterized in that, Data can be collected through student terminals, third-party system integration, or data entry by organizers. Data cleaning includes standardization of collection formats, conversion of heterogeneous information formats, field format validation, and intelligent deduplication and cleaning.

4. A competition information management system according to claim 1, characterized in that, The method for calculating the comprehensive evaluation value is as follows: Quantify and assign values ​​to CPU utilization to obtain the actual CPU utilization U and the normal threshold A. CPU and warning threshold B CPU If U≤A CPU Then S CPU =1-(U / A CPU If A × 0.3, CPU <U≤B CPU S CPU =0.7-[(UA CPU ) / (B CPU -A CPU )]×0.5, if U>B CPU S CPU =0.2-[(UB CPU ) / (100-B CPU )]×0.2; Quantify memory usage and assign values ​​to obtain the actual memory usage M and the normal threshold A. 内存 and warning threshold B 内存 If M≤A 内存 S M =1-(M / A 内存） ×0.2, if A 内存 <M≤B 内存 S M =0.8-[(MA 内存） / (B 内存 -A 内存 ] × 0.6, if M > B 内存 S M =0.2-[(MB 内存） / (100-B 内存） ×0.2; The query response time is quantified and assigned values. Using time t as a period, the corresponding query response time is obtained, and its mean T is calculated as a standard. Simultaneously, a normal threshold A is obtained. 时间 Warning threshold B 时间 and the abnormal threshold C 时间 If T≤A 时间 S T =1-(T / A 时间） ×0.2, if A 时间 <T≤B 时间 S T =0.8-[(TA 时间） / (B 时间 -A 时间） ]×0.6, if T>B 时间 S T =0.2-[(TB 时间） / (C 时间 -B 时间） ×0.2; The data write frequency is quantized and assigned a value to obtain the actual write frequency value W. Simultaneously, the lower limit threshold L, the normal range N, and the upper limit threshold H corresponding to the processing node are obtained. If L≤W≤N 上限 S W =1-|WN median| / (N) 上限 -L), if N 上限 <W≤H,S W =0.8-[(WN 上限） / (HN 上限） ]×0.6, if W <L:S W =0.8-[(LW) / L]×0.6; The performance metrics obtained from the quantitative assignment are weighted and summed, and the comprehensive evaluation value is calculated according to the formula: S CPU ×a1+S M ×a2+S T ×a3+S W ×a4, where a1, a2, a3 and a4 are the corresponding weight coefficients.

5. A competition information management system according to claim 1, characterized in that, The method for classifying high-load nodes and low-load nodes is as follows: Next, processing nodes with a comprehensive evaluation value greater than the performance index threshold are classified as high-performance nodes, and processing nodes with a comprehensive evaluation value less than the performance index threshold are classified as low-performance nodes. Simultaneously, all high-performance nodes and their corresponding node loads are obtained, and the node loads are compared with the corresponding load thresholds. If the node load is greater than the load threshold, it is classified as a high-load node; otherwise, it is classified as a low-load node.

6. A competition information management system according to claim 1, characterized in that, The method for generating fragment management information is as follows: For the high-load nodes identified by classification, obtain their corresponding processing tasks, classify the processing tasks into high-priority tasks and low-priority tasks, and transfer high-priority tasks to low-load nodes. Specifically, select target nodes from the low-load node pool that are in the same region, have the same configuration, and have idle resources ≥ 120% of the task requirements. Pre-start the task dependency environment on the target node, synchronize the task status through a dual-write buffer mechanism, and switch the route to the target node when the task execution reaches the atomic breakpoint. The original node stops processing. After the migration is completed, verify the consistency of the task status and generate shard management information.

7. A competition information management system according to claim 1, characterized in that, The method for calculating the number of nodes to be expanded is as follows: Historical periodic data is acquired and input into an LSTM+attention mechanism model. The model outputs a predicted load increment. Based on the predicted load increment and the single node's carrying capacity, the number of expansion nodes is dynamically calculated. The expansion node count is calculated using the formula: Where 1.2 is the redundancy coefficient, the single node capacity represents the maximum load that an existing node can bear without affecting service performance, and ceil represents the rounding up function to obtain the number of nodes to be expanded.

8. A competition information management system according to claim 1, characterized in that, The method for generating node expansion information is as follows: Prioritize selecting nodes from the idle node pool that meet the expansion requirements. The nodes must be in the same region and have the same configuration as the current high-performance nodes, and the idle resources must be at least 120% of the resources required for expansion. After selecting the target node, pre-start the task dependency environment on the node to ensure that the environment configuration is consistent with the original node.

9. A competition information management system according to claim 2, characterized in that, The methods for performing deduplication storage management and generating storage management information are as follows: The deduplication management method for structured business data is as follows: within the same shard, if the event ID + user ID / team ID are absolutely duplicated, or the core information hash is the same and the event ID is the same, when writing data, first query the deduplication index table within the shard. If an absolute duplicate is matched, directly return that the data already exists and refuse to write it again; if a suspected duplicate is matched, trigger manual verification, and the verification result will be updated synchronously in the index table. The deduplication management method for semi-structured and unstructured data is as follows: within the same shard, if the content hash is completely identical and the associated subject + event ID is the same, it is judged as duplicate, and duplicate data is not stored repeatedly, but only a reference pointer is added to the current business record; if the content hash is identical but the associated subject is different, it is judged as legally similar and is allowed to be stored independently.

10. A competition information management method, executed by the competition information management system according to any one of claims 1-9, characterized in that, The method specifically includes the following steps: Step 1: Collect competition information from participants through three methods: participant-side, third-party system integration, or organizer input. After data cleaning, preprocessed competition information is obtained. Data cleaning includes standardization of collection format, conversion of heterogeneous information format, field format validation, and intelligent deduplication. Data fragmentation is performed on the preprocessed competition information according to user region to generate data fragment information. Step 2: Perform node allocation management based on data sharding information, obtain and quantify the CPU utilization, memory usage, query response time and data write frequency of each processing node, and calculate the comprehensive evaluation value. Based on the comprehensive evaluation value, classify the processing nodes into high-performance nodes and low-performance nodes. Step 3: Obtain all high-performance nodes and their corresponding loads. Compare the node loads with the load thresholds to classify them into high-load and low-load nodes. Obtain the processing tasks of high-load nodes. Divide the processing tasks into high-priority and low-priority tasks according to the timeliness and impact of the competition. Migrate the high-priority tasks to low-load nodes and generate shard management information. Step 4: Monitor the real-time load of high-performance nodes after sharding. If the real-time load is greater than the load threshold, perform elastic scaling, calculate the number of nodes to be scaled up, select target nodes that meet the conditions from the idle node pool to perform scaling up, and generate node scaling up information. Step 5: Split the information into metadata and business data. According to the data type of the competition, divide the duplicate data in the two into structured business data, semi-structured data, and unstructured data. Perform deduplication storage management on each and generate storage management information.

Images