Big data-based review optimization system

Abstract

The present application relates to big data analysis technical field, specifically be based on big data's comment optimization system, system includes answer analysis module, comment weight adjustment module, result matching module, trend prediction module, storage optimization module.In the present application, through the logical structure and key information point of depth analysis answer content, distinguish and filter core, extension and redundant information, realize the dynamic optimization to the scoring weight, enhance the deep level consideration to the logical correlation, exposition completeness and information contribution of answer, according to the feature, the internal connection of different answers is more accurately identified, the future trend is predicted and the related answer is filtered, the process not only improves the accuracy and fairness of scoring, but also realizes the optimal allocation of comment resources, finally, according to the comment data distribution characteristics, the storage partition layout is dynamically adjusted, the data retrieval efficiency and storage security are improved, effectively solve the problem of traditional comment to the insufficient mining of answer deep value.

Description

Technical Field

[0001] This invention relates to the field of big data analytics, and in particular to a big data-based review and optimization system. Background Technology

[0002] Big data analytics technology falls under the interdisciplinary application of information processing and intelligent computing. It primarily researches the technological system for extracting valuable information from multi-source heterogeneous data through collection, storage, cleaning, integration, and analysis. Core aspects of this field include data acquisition technology, data preprocessing technology, data warehouse construction technology, data mining and statistical analysis technology, machine learning modeling technology, and result visualization and decision support technology. Its overall technological system processes massive amounts of structured and unstructured data in a distributed computing environment to achieve in-depth data value mining and application expansion, and is widely used in scenarios such as education assessment, medical management, financial analysis, and intelligent manufacturing. Among these, traditional grading optimization systems refer to computer-aided systems used in education or examination scenarios to grade and evaluate subjective question answers. These systems are primarily designed to address the low efficiency and inconsistent accuracy of manual grading. Traditional grading optimization systems use character recognition and natural language matching to extract and compare student answers, determine answer correctness through keyword matching or similarity calculation methods, and combine a rule base to determine scores and statistically analyze results.

[0003] Traditional grading optimization systems primarily rely on character recognition and keyword matching. This approach has significant limitations when dealing with complex subjective questions. Because it only focuses on the surface textual similarity, it struggles to deeply understand and evaluate the logical structure, depth of argumentation, and information organization of the answers. For example, when two students use different expressions to illustrate the same core concept, the system may assign drastically different scores due to keyword mismatch, leading to biased grading results. This mechanical comparison method lacks a hierarchical distinction of the value of the answer content, failing to identify which are core scoring points and which are supplementary explanations. Consequently, the grading criteria become too simplistic, affecting the fairness and accuracy of the grading process. Summary of the Invention

[0004] To address the technical problems existing in the prior art, embodiments of the present invention provide a review optimization system based on big data. The technical solution is as follows:

[0005] On the one hand, a big data-based grading optimization system is provided, which includes:

[0006] The answer parsing module is based on the graded dataset. It extracts the logical structure and key information points from the graded answers, divides the content features into three categories: basic information, extended information, and redundant information, analyzes the information coverage, filters the core information areas, and obtains the answer content features and core information area set.

[0007] The review weight adjustment module, based on the answer content features and the core information area set, sets the review priority according to the content features, applies weighted scoring to the core information area, adjusts the weighted scoring of the extended information area, and processes the redundant information area with low weight, thereby obtaining a review weight distribution dataset.

[0008] The result matching module calls the grading weight distribution dataset, extracts the logical correlation, completeness of explanation and information contribution of the answers, classifies them according to the matching features, filters the answers within the same feature unit, and obtains the grading result comparison relationship dataset.

[0009] Based on the grading result comparison dataset, the trend prediction module analyzes the changing trends and consistency of grading needs, determines the correlation of grading needs in the future time period, sets correlation judgment conditions, filters related answers, and obtains an overall trend optimization plan for grading needs.

[0010] As a further aspect of the present invention, the set of answer content features and core information regions includes basic information, extended information, and redundant information; the grading weight distribution dataset includes a weighted scoring region, an adjusted weighted scoring region, and a low-weight processing region; the grading result comparison dataset includes answer units with consistent logical correlation and answer units with similar information contribution; and the overall trend optimization scheme for grading needs includes the changing trend of grading needs, consistency of grading needs, correlation determination conditions, and related answers.

[0011] As a further aspect of the present invention, the answer parsing module includes:

[0012] The logical structure extraction submodule, based on the grading dataset, measures and records the logical hierarchy in the answers, analyzes the logical structure and key information points in each region, and obtains content feature values ​​based on the logical and information analysis.

[0013] The feature classification submodule calls the content feature value, identifies the category based on the changing trend of content features in adjacent areas, determines the feature range to which the area belongs, sets the feature classification standard, divides the area into three types of features: basic, extended, and redundant, analyzes the distribution within the area, and generates a set of answer content features and core information areas.

[0014] The core information filtering submodule calls the answer content features and core information region set, identifies the distribution density of feature regions, filters high-density regions, and obtains the core information region location results.

[0015] As a further aspect of the present invention, the review weight adjustment module includes:

[0016] The priority setting submodule calls the core information area location result, sets the review priority based on content characteristics, sorts them according to the keyness of the characteristics, and obtains the area review priority data.

[0017] The weight adjustment submodule calls the regional review priority data, adjusts the review weight according to the priority, calculates the core weight feature value, sets the weight score for the core information area, adjusts the weight score for the extended information area, and processes the redundant information area with low weight. It analyzes the changes in weight distribution, adjusts the size of the review dataset, and obtains the review weight distribution dataset.

[0018] As a further aspect of the present invention, the result matching module includes:

[0019] The logical correlation extraction submodule extracts the logical correlation, completeness of explanation, and information contribution of the answers based on the grading weight distribution dataset, and obtains the grading feature set;

[0020] The feature classification submodule calls the review feature set, analyzes feature differences based on logical correlation, completeness of explanation and information contribution, filters answers with similar features, determines the category to which the answer belongs, divides the same feature units, optimizes the classification boundary, and obtains feature unit distribution data.

[0021] The result comparison generation submodule calls the feature unit distribution data, performs local region segmentation on the answer data in each feature unit and extracts answer features, encodes the extracted features and measures similarity, filters answer data within the same unit, and obtains the grading result comparison relationship dataset.

[0022] As a further aspect of the present invention, the extraction of the logical relevance, completeness of explanation, and information contribution of the answer refers to using a preset natural language processing model to perform syntactic and semantic analysis on each answer text in the grading weight distribution dataset. The syntactic analysis refers to identifying the sentence structure, dependency relations, and parts of speech in the answer text, and the semantic analysis refers to extracting keywords, topic vectors, and contextual information.

[0023] As a further aspect of the present invention, the trend prediction module includes:

[0024] The demand trend analysis submodule calls the review result comparison relationship dataset to analyze the trend and consistency of review demand changes, extract time series data, and obtain the characteristics of demand trend changes;

[0025] The correlation judgment submodule calls the aforementioned demand trend change characteristics, analyzes the consistency between the changes and distribution of the review demands, identifies the correlation of demands, sets correlation judgment conditions, filters the correlation of demands, eliminates low correlation demands, optimizes data matching relationships, and obtains the demand correlation filtering results.

[0026] The overall trend filtering submodule calls the required association filtering results, uses the distribution information between answer units to compare and analyze the positional correspondence, identifies the direction of demand change and evolution path, extracts the demand evolution stage through time series data, filters demand combination units with prominent trend changes, and obtains the overall trend optimization plan for the reviewed demands.

[0027] As a further aspect of the present invention, the distribution information between the answer units refers to the information obtained after analyzing and quantifying the spatial or structural layout of the answer content.

[0028] The aforementioned demand evolution stage refers to the period in which demand changes are identified by analyzing time series data. Based on the turning points of demand changes, the entire evolution process is divided into several independent stages with differentiated characteristics.

[0029] As a further aspect of the present invention, the system also includes a storage optimization module:

[0030] Based on the overall trend optimization scheme of the review requirements, the storage optimization module extracts the distribution information of the review data, identifies the corresponding positions between data, divides the storage area according to the distribution characteristics, adjusts the storage partition layout, and obtains the result of secure storage of the review data.

[0031] The secure storage results of the approval data include storage area division, storage partition layout adjustment, extraction of approval data distribution information, and identification of corresponding data locations.

[0032] As a further aspect of the present invention, the storage optimization module includes:

[0033] The data distribution extraction submodule calls the overall trend optimization scheme of the review requirements to extract the review data distribution data, identify the corresponding position of the data, and obtain the review data distribution information;

[0034] The storage area partitioning submodule calls the batch data distribution information, identifies distribution characteristics based on the corresponding data location, filters data-dense areas, partitions storage areas, sets differentiated storage area boundaries, adjusts the data distribution balance, and obtains the storage area partitioning results.

[0035] The partition layout adjustment submodule calls the storage area division results, adjusts the storage partition layout, optimizes the batch data storage mapping, and obtains the secure storage results of the batch data.

[0036] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0037] By deeply analyzing the logical structure and key information points of the answer content, and distinguishing and filtering core, extended, and redundant information, dynamic optimization of scoring weights is achieved. This enhances the in-depth consideration of the logical relevance, completeness of explanation, and information contribution of the answers. Answers are classified and matched based on features to more accurately identify the inherent connections between different answers. Furthermore, the changes and consistency of grading needs are analyzed, future trends are predicted, and related answers are filtered. This process not only improves the accuracy and fairness of scoring but also optimizes the allocation of grading resources. Finally, the storage partition layout is dynamically adjusted based on the distribution characteristics of grading data, improving data retrieval efficiency and storage security, and effectively solving the problem of insufficient deep value mining of answers in traditional grading. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic diagram of the big data-based review optimization system provided in an embodiment of the present invention;

[0040] Figure 2 This is a schematic diagram of the system framework of the present invention;

[0041] Figure 3 This is a flowchart of the answer parsing module in this invention;

[0042] Figure 4 This is a flowchart of the review weight adjustment module in this invention;

[0043] Figure 5 This is a flowchart of the result matching module in this invention;

[0044] Figure 6 This is a flowchart of the trend prediction module in this invention;

[0045] Figure 7 This is a flowchart of the storage optimization module in this invention. Detailed Implementation

[0046] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0047] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0048] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0049] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0050] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0051] This invention provides a big data-based grading optimization system, such as... Figure 1-2 The diagram shown illustrates a big data-based grading optimization system, which includes:

[0052] The answer parsing module is based on the graded dataset. It extracts the logical structure and key information points from the graded answers, divides the content features into three categories: basic information, extended information, and redundant information, analyzes the information coverage, filters the core information areas, and obtains the answer content features and core information area set.

[0053] The review weight adjustment module is based on the answer content features and the core information area set. It sets the review priority according to the content features, applies weighted scoring to the core information area, adjusts the weighted scoring of the extended information area, and processes the redundant information area with low weight to obtain the review weight distribution dataset.

[0054] The result matching module calls the grading weight distribution dataset, extracts the logical correlation, completeness of explanation and information contribution of the answers, classifies them according to the matching features, filters the answers within the same feature unit, and obtains the grading result comparison relationship dataset.

[0055] The trend prediction module analyzes the changing trends and consistency of grading needs based on the comparison dataset of grading results, judges the correlation of grading needs in the future time period, sets correlation judgment conditions, filters related answers, and obtains an overall trend optimization plan for grading needs.

[0056] The storage optimization module optimizes the overall trend of review demand, extracts the distribution information of review data, identifies the corresponding positions between data, divides the storage area according to the distribution characteristics, adjusts the storage partition layout, and obtains the result of secure storage of review data.

[0057] The answer content features and core information area set includes basic information, extended information, and redundant information; the grading weight distribution dataset includes weighted scoring areas, adjusted weighted scoring areas, and low-weight processing areas; the grading result comparison dataset includes answer units with consistent logical correlation and answer units with similar information contribution; the overall trend optimization scheme for grading needs includes the changing trend of grading needs, consistency of grading needs, correlation judgment conditions, and related answers; and the secure storage results of grading data include storage area division, storage partition layout adjustment, extraction of grading data distribution information, and identification of corresponding data locations.

[0058] Specifically, such as Figure 2 , 3 As shown, the answer parsing module includes:

[0059] The logical structure extraction submodule, based on the grading dataset, measures and records the logical hierarchy in the answers, analyzes the logical structure and key information points in each region, and obtains content feature values ​​based on the logical and information analysis.

[0060] Based on the graded dataset, a sample answer to the essay question "Discuss the impact of artificial intelligence on social development" was retrieved from the dataset. Its logical hierarchy was measured and recorded. By identifying logical connectors such as "firstly," "secondly," and "in conclusion," the answer was divided into four logical levels, L1 to L4, and assigned a unique level ID. Next, the logical structure and key information points within each region were analyzed. In region L1, dependency parsing was used to identify the core phrase "promote the development of productivity," extracting the key information points "productivity" and "development," and assigning the information point ID KP1. In region L2, "change in employment structure" was identified. The "employment structure" and "change" were extracted and assigned the ID KP2. In the L3 region, "bringing ethical challenges" was identified, "ethical challenges" was extracted and assigned the ID KP3. In the L4 region, "needs to be viewed dialectically" was identified, "viewed dialectically" was extracted and assigned the ID KP4. Then, based on the logical and information analysis of the content features, the logical level depth 4 and the number of key information points 4 were quantitatively combined to form the initial content feature vector V=[4,4]. This vector and the preset standard answer feature vector V_std=[4,5] were subjected to cosine similarity calculation, and the calculated value was 0.98. This value was used as the content feature value.

[0061] The feature classification submodule calls the content feature values, identifies the category based on the changing trend of content features in adjacent areas, determines the feature range to which the area belongs, sets the feature classification criteria, divides the area into three types of features: basic, extended, and redundant, analyzes the distribution within the area, and generates a set of answer content features and core information areas.

[0062] The system retrieves a series of content feature values ​​from different answer regions, such as 0.98 for region A, 0.75 for region B, and 0.32 for region C. Based on the changing trends of content features in adjacent regions, it identifies the category, calculating the content feature change rate from region A to B as Δ_AB = |0.75 - 0.98| / 0.98, which is 23.5%. It also calculates the change rate from region B to C as Δ_BC = |0.32 - 0.75| / 0.75, which is 57.3%. Next, it determines the feature range to which the region belongs. This determination is based on a preset feature classification standard: content feature values ​​in the range [0.8, 1.0] are basic features, those in the range [0.6, 0.8) are extended features, and values ​​below 0.6 are redundant features. This standard is based on the 1... The content feature values ​​of 0000 historical high-scoring answers were statistically analyzed, and the 80th and 60th percentiles were used as the dividing lines. Based on this, region A (0.98) was classified as a basic feature, region B (0.75) as an extended feature, and region C (0.32) as a redundant feature. Subsequently, the regions were divided into three types of features, and the distribution within the regions was analyzed. It was found that in a single answer, the basic feature region appeared 3 times, the extended feature region appeared 2 times, and the redundant feature region appeared 1 time, forming a distribution vector D=[3, 2, 1]. This generated a set of answer content features and core information regions, specifically {[region A, basic], [region B, extended], [region C, redundant], ...}, in which the feature classification of each region was clearly marked.

[0063] The core information filtering submodule calls the answer content features and core information region set, identifies the distribution density of feature regions, filters high-density regions, and obtains the core information region location results;

[0064] The system retrieves the answer content features and core information region sets, such as 0.98 for region A, 0.75 for region B, and 0.32 for region C. Based on the changing trends of content features in adjacent regions, it identifies the category, calculating the content feature change rate from region A to B as Δ_AB = |0.75 - 0.98| / 0.98, which is 23.5%. It also calculates the change rate from region B to C as Δ_BC = |0.32 - 0.75| / 0.75, which is 57.3%. Next, it determines the feature range to which the region belongs, based on a preset feature classification standard: content feature values ​​in the range [0.8, 1.0] are considered basic features, and values ​​in the range [0.6, 0.8] are considered basic features. The range is considered an extended feature, and values ​​below 0.6 are considered redundant features. This standard is based on the statistical distribution of content feature values ​​of 10,000 historical high-scoring answers, using the 80th and 60th quantiles as the dividing lines. Accordingly, region A (0.98) is classified as a basic feature, region B (0.75) as an extended feature, and region C (0.32) as a redundant feature. Subsequently, the regions are divided into three types of features, and the distribution within each region is analyzed. In a single answer, the basic feature region appears 3 times, the extended feature region appears 2 times, and the redundant feature region appears 1 time, forming a distribution vector D=[3, 2, 1], which generates the core information region location result.

[0065] Specifically, such as Figure 2 , 4 As shown, the review weight adjustment module includes:

[0066] The priority setting submodule calls the core information area location results, sets the review priority based on content characteristics, sorts them according to the keyness of the characteristics, and obtains the area review priority data;

[0067] The core information area location results are retrieved. This setting is accomplished through a preset mapping rule: the priority of the basic feature area is set to 3, the extended feature area to 2, and the redundant feature area to 1. This rule is established based on regression analysis of the scoring habits of senior review experts, where the feature category is the independent variable and the scoring weight is the dependent variable. The obtained regression coefficients are rounded to form this mapping rule. Therefore, the priority PA of area A is assigned a value of 3, the priority PB of area B is 2, and the priority PC of area C is 1. Then, the areas are sorted according to the keyness of the features and arranged in descending order according to the priority values ​​to obtain the sorted area sequence [area A, area B, area C]. The area review priority data is obtained. This data is a set containing each area and its corresponding priority value, such as {area A: 3, area B: 2, area C: 1}.

[0068] The weight adjustment submodule calls the regional review priority data, adjusts the review weight according to the priority, calculates the core weight feature value, sets the weight score for the core information area, adjusts the weight score for the extended information area, and processes the redundant information area with low weight. It analyzes the changes in weight distribution, adjusts the size of the review dataset, and obtains the review weight distribution dataset.

[0069] The core weight feature values ​​are calculated using the following formula: ;

[0070] in, Represents the core weight feature value. Representing the The approval priority data values ​​for each core information area Representing the The scoring weight values ​​for each core information area This represents the average of the score weights for the core information area. The total number of areas representing core information;

[0071] The system retrieves the region review priority data {Region A: 3, Region B: 2, Region C: 1}, adjusts the review weights according to the priority, and assigns weighted scores to core information regions (i.e., basic feature regions). For example, it assigns a weight to Region A. The scoring weight for another core information region, D, is set to 0.5. The scoring weight for another core information region E is set to 0.4. The weight is set to 0.6. For extended information regions (such as region B), the weight score is adjusted to 0.2. Redundant information regions (such as region C) are given a low weight, with a score of 0.05. Next, the core weight feature values ​​are calculated. This calculation is only performed for core information regions. Assume there are three core information regions A, D, and E in the current answer, with priority data values ​​of... , , The corresponding scoring weight values ​​are as follows: , , The total number of core information areas First, calculate the average value of the scoring weights for the core information area. , Then substitute the value into the core weight feature value formula. ,in, Representing the core weight feature value, it is a comprehensive indicator that measures the dispersion of weight allocation within the core information region and the impact of priority weighting. Representing the The review priority data value for each core information area is quantified based on the area's content characteristics (basic, extended, redundant), reflecting the area's criticality within the scoring system. Representing the The scoring weight value for each core information area is the original score weight assigned to that area. This represents the arithmetic mean of the score weights for all core information areas. The total number of core information areas, symbol This indicates taking the absolute value. This represents the summation of all core information regions from 1 to n;

[0072] The calculation logic of the entire formula is as follows: First, calculate the absolute value of the deviation between the score weight of each core area and its average value. Then, weight it with the priority of the area. This product reflects that the weight deviation of high priority areas has a greater impact on the whole. Next, sum the weighted deviations of all core areas and divide by the number of core areas to get an average weighted deviation. Finally, take its square root to get an eigenvalue in the form of a standard deviation.

[0073] The advantage of the formula lies in the introduction of priority data. As a weighting factor, the core weight eigenvalues It can not only measure the volatility of weight allocation, but also highlight whether the weight settings for high-priority areas are reasonable. If the weight values ​​of high-priority areas deviate significantly from the average, it will lead to... A significant increase indicates that the weighting allocation needs to be adjusted;

[0074] The calculation process of this formula is as follows:

[0075] ;

[0076] A threshold range for core weight features is set within the range of (0.3, 0.5), based on the analysis of 1000 high-scoring sample answers from the past. Statistical analysis of the values ​​revealed that 95% of the high-scoring samples... The value falling within this range indicates that its weight allocation is both discriminative and not too extreme. The value 0.447 falls within this interval, indicating that the current weight allocation of the core region is reasonable. Next, the changes in weight distribution are analyzed, comparing the weight distribution before and after adjustment. For example, before adjustment, all regions had a weight of 0.25; after adjustment, the weight vector is (0.5, 0.2, 0.05, 0.4, 0.6]. And based on... The calculation results determine whether the size of the grading dataset needs to be adjusted, especially if there are multiple answers. If the values ​​are generally below 0.3, then add answer samples containing more diverse weight allocation cases to the grading dataset to obtain the grading weight distribution dataset. This dataset contains the text of each answer, the division of each region, the feature classification, the priority, and the final adjusted weight value.

[0077] Specifically, such as Figure 2 , 5 As shown, the result matching module includes:

[0078] The logical relevance extraction submodule extracts the logical relevance, completeness of explanation, and information contribution of the answers based on the grading weight distribution dataset, and obtains the grading feature set;

[0079] Extracting the logical coherence, completeness of explanation, and information contribution of the answers refers to using a pre-defined natural language processing model to perform syntactic and semantic analysis on each answer text in the grading weight distribution dataset. Syntactic analysis refers to identifying sentence structure, dependency relations, and parts of speech in the answer text, while semantic analysis refers to extracting keywords, topic vectors, and contextual information.

[0080] Based on the weighted distribution dataset, one answer text was processed. The weight vector of this text is [0.5, 0.2, 0.05, 0.4, 0.6]. A pre-defined natural language processing model was used to perform syntactic analysis, identifying the subject-verb-object structure of the first sentence, "Artificial intelligence is the core driving force of the new round of technological revolution." The dependency relationship between "artificial intelligence" and "driving force" was analyzed, and parts of speech were tagged. Then, semantic analysis was performed to extract the keywords "artificial intelligence," "productivity," "employment," and "ethics," calculating their TF-IDF values ​​to be 0.25, 0.18, 0.15, and 0.12, respectively. A 300-dimensional topic vector [0.02, -0.15, ..., 0.08] was generated, and the context of "productivity" was analyzed. The text information is used to determine its association with "improvement". Then, logical relevance, completeness of expression and information contribution are extracted. Logical relevance is quantified by calculating the cosine similarity of the topic vectors of adjacent sentences. If the similarity is greater than 0.7 (this threshold is the average similarity of sentence pairs labeled "strongly related" by 50 linguistic experts who have labeled 1000 sentence pairs), it is considered to be a strong relevance. Completeness of expression is determined by checking 5 key information points of the standard answer. The current answer covers 4 of them, and the completeness score is 0.8. Information contribution is determined according to the scoring weight value of each region. The grading feature set is obtained. This set contains a logical relevance score of 0.75, a completeness of expression score of 0.8 and an information contribution vector [0.5, 0.2, 0.05, 0.4, 0.6].

[0081] The feature classification submodule calls the review feature set, analyzes feature differences based on logical correlation, completeness of explanation and information contribution, filters answers with similar features, determines the category to which the answer belongs, divides the same feature units, optimizes the classification boundary, and obtains feature unit distribution data.

[0082] The feature set of answer A ({logical relevance: 0.75, completeness of expression: 0.8, ...}) and the feature set of answer B ({logical relevance: 0.72, completeness of expression: 0.81, ...}) are used. Based on logical relevance, completeness of expression, and information contribution, the feature differences are analyzed. The Euclidean distance D(A, B) between the two answer features is calculated to be 0.032. Then, answers with similar features are selected by setting a distance threshold Th_d = 0.1, which is obtained by taking the 80th percentile of the feature distance of 500 pairs of answer pairs rated as "of the same type" by the same expert. Since 0.032 < 0.1, it is determined that answers A and B belong to the same category. Then, the same feature units are divided, and answers A and B are assigned to the same feature unit C1. Then, the classification boundary is optimized. When a new answer C is added, the average distance between it and all answers in C1 is calculated. If the average distance is less than Th_d, then C is assigned to C1. Otherwise, a new feature unit C2 is created, and the feature unit distribution data is obtained. This data records the list of answer IDs contained in each feature unit, such as {C1: [Answer A_ID, Answer B_ID], C2: [Answer C_ID]}.

[0083] The result comparison generation submodule calls the feature unit distribution data, performs local region segmentation on the answer data in each feature unit and extracts answer features, encodes the extracted features and measures similarity, filters answer data within the same unit, and obtains the grading result comparison relationship dataset.

[0084] The data {C1: [Answer A_ID, Answer B_ID], C2: [Answer C_ID]} is retrieved. For each feature unit, local region segmentation is performed on the answer data, and features are extracted. Taking C1 as an example, answers A and B are extracted, and their first paragraphs are processed to extract keyword vectors. For example, the vector for the first paragraph of answer A is V_A1=[0.8, 0.2, 0.1], and the vector for the first paragraph of answer B is V_B1=[0.78, 0.22, 0.11]. Then, the extracted features are encoded and similarity is measured. The keyword vectors are used as feature encodings. The cosine similarity Sim(V_A1, V_B1) is calculated to be 0.99. Then, answer data within the same unit is filtered, and similarity is calculated for all corresponding local regions of all answer pairs in unit C1. All regions with similarity greater than 0.95 (high similarity threshold, which is determined by ROC curve analysis on the test set to balance precision and recall) are recorded. The grading result comparison dataset is obtained. This dataset is an association table that records the high similarity between "the first paragraph of answer A" and "the first paragraph of answer B", as well as their original scores, weights, and other information.

[0085] Specifically, such as Figure 2 , 6 As shown, the trend prediction module includes:

[0086] The demand trend analysis submodule calls the review result comparison relationship dataset to analyze the trend and consistency of review demand changes, extracts time series data, and obtains the characteristics of demand trend changes.

[0087] The dataset is used to retrieve timestamp information, such as answer A submitted on 2023-10-01 and answer B on 2023-10-02. The trend and consistency of grading requirements are analyzed, and the consistency of grading scores for answers within feature unit C1 at different time points is tracked. For example, in October 2023, the standard deviation of the average score for answers within unit C1 was 1.2 points, which dropped to 0.8 points in November, indicating an improvement in grading consistency. Next, time series data is extracted and counted weekly to determine the number of answers submitted to unit C1, forming a time series [25, 30, 28, 35]. The trend of demand changes is then analyzed, and a moving average is applied to this time series data, showing a positive growth trend with an average weekly increase of 2.5 submissions. The time series of score standard deviations [1.2, 1.1, 0.9, 0.8] is analyzed to identify the characteristic of increased consistency requirements.

[0088] The correlation judgment submodule calls the characteristics of demand trend changes, analyzes the consistency between the changes and distribution of approval demands, identifies the correlation of demands, sets correlation judgment conditions, filters demand correlation, eliminates low correlation demands, optimizes data matching relationships, and obtains demand correlation filtering results.

[0089] Based on the characteristics of demand trend changes, specifically, the quantity growth trend feature (+2.5 articles per week) and the consistency enhancement feature (standard deviation -0.1 per week) were used to analyze the consistency between the changes and distribution of review demand. It was found that quantity growth was concentrated in feature units C1 and C3, while the phenomenon of enhanced consistency demand was reflected in all feature units. Next, demand correlation was identified by calculating the Pearson correlation coefficient between the time series of quantity growth rate and the time series of consistency demand intensity (expressed as the reciprocal of the standard deviation), yielding a value of 0.85. A correlation criterion was set, stipulating that a high correlation was defined when the absolute value of the Pearson correlation coefficient was greater than 0.75. This value of 0.75 was determined through historical... Historical data backtesting confirmed that the prediction model had the highest accuracy at this threshold. Since 0.85 > 0.75, a high correlation was found between "increased number of answers" and "increased need for consistency in grading". Subsequently, the correlation of needs was screened, and low-correlation needs were eliminated. For example, the correlation coefficient between "change in average answer length" and "need for consistency in grading" was 0.2, so the correlation was eliminated. The data matching relationship was optimized. In subsequent analysis, the two highly correlated needs of quantity and consistency were given priority. The results of the demand correlation screening were obtained. The results are a list containing all highly correlated demand pairs, such as [(increased quantity, increased consistency), (frequency of knowledge point examination, score difference)].

[0090] The overall trend screening submodule calls the requirements association screening results, uses the distribution information between answer units to compare and analyze the position correspondence, identifies the direction of demand change and evolution path, extracts the demand evolution stage through time series data, screens demand combination units with prominent trend changes, and obtains the overall trend optimization plan for the reviewed demands.

[0091] Distribution information between answer units refers to the information obtained after analyzing and quantifying the spatial or structural layout of answer content;

[0092] The demand evolution stage refers to the period in which demand changes are identified by analyzing time series data. Based on the turning points of demand changes, the entire evolution process is divided into several independent stages with differentiated characteristics.

[0093] The system retrieves the results of the demand correlation filtering. This distribution information refers to the quantitative analysis of the structural layout of the answer content. For example, the "core argument" unit appears in the first two paragraphs in 80% of high-scoring answers. Retrieving this information and combining it with highly correlated demand pairs, the system identifies the direction and evolution path of demand changes. It finds that the frequency of testing for "specific knowledge point A" is increasing. Simultaneously, discussions related to knowledge point A are migrating from the extended area (paragraphs three and four) to the core area (paragraphs one and two), identifying the evolution path of "knowledge point centralization." Then, the system extracts the demand evolution stages through time series data. This stage refers to the period defined by analyzing the turning points of demand changes. The system further analyzes the knowledge... The time series data on the frequency of point A in the core area identified the following stages: before June 2023, the frequency was below 10%, which was the "embryonic stage"; from June to September, the frequency was between 10% and 40%, which was the "development stage"; and after September, the frequency stabilized above 40%, which was the "mature stage". Then, demand combination units with prominent trend changes were selected, and the growth rate of each demand combination (such as "knowledge point A" + "core area") in different evolution stages was calculated. Combinations with a growth rate of over 50% were selected to obtain an overall trend optimization plan for the review requirements. This plan clearly pointed out that "the discussion of knowledge point A in the first and second paragraphs should be given special attention, and its weight in the scoring criteria should be increased."

[0094] Specifically, such as Figure 2 , 7 As shown, the storage optimization module includes:

[0095] The data distribution extraction submodule calls the overall trend optimization scheme of the review requirements, extracts the review data distribution data, identifies the corresponding location of the data, and obtains the review data distribution information;

[0096] The overall trend optimization plan for review requirements is invoked, specifically the conclusion in the plan that "focus on the discussion of knowledge point A in the first and second paragraphs." Review data distribution data is extracted, and all data records related to knowledge point A and appearing in the first two paragraphs of the answer are filtered from the overall review data database. This includes text fragments, scores, review times, etc. Next, the corresponding data positions are identified, recording the start and end positions (row and column numbers) of each data entry in the original answer text and its logical address in the storage system. Review data distribution information is obtained as a mapping table describing the distribution of data items conforming to the optimization plan on the physical storage medium, such as {Data ID_001: [Disk A, Sector 1024], Data ID_002: [Disk B, Sector 5678], ...}, showing a snapshot of the current physical location of the target data.

[0097] The storage area partitioning submodule calls the batch data distribution information, identifies distribution characteristics based on the corresponding data location, filters data-dense areas, partitions storage areas, sets differentiated storage area boundaries, adjusts the data distribution balance, and obtains the storage area partitioning results.

[0098] Based on the distribution information of the reviewed data, the mapping table {Data ID_001: [Disk A, Sector 1024], ...} is invoked. Distribution characteristics are identified based on the corresponding data locations. Analysis reveals that over 70% of the data related to knowledge point A is stored on disk A, forming a storage hotspot. Next, data-intensive areas are filtered, and disk A with a storage utilization rate exceeding 80% and an I / O request frequency ranking in the top 10% over the past hour is identified as a data-intensive area. Storage areas are then divided, and based on the identification results, the physical storage space is divided into a "hot data area" (composed of high-speed SSDs) and a "warm data area". The system defines "hot data area" and "cold data area" and sets differentiated storage area boundaries based on data access frequency and modification time. For example, data accessed more than 100 times in the past week is automatically classified into the hot data area, with its logical boundary set as "hot". Then, the data distribution balance is adjusted, and a data migration task is initiated to migrate 30% of the data about knowledge point A on disk A to another low-load high-speed SSD disk C. The storage area division result is obtained, which is an updated storage policy document that defines the physical range, capacity, and data admission and migration rules of the hot, warm, and cold areas.

[0099] The partition layout adjustment submodule calls the storage area partitioning results, adjusts the storage partition layout, optimizes the batch data storage mapping, and obtains the secure storage results of the batch data.

[0100] The updated storage policy document defines three storage zones: hot, warm, and cold. The storage partition layout is adjusted, and the original single logical partition ` / data` is reorganized into three new logical partitions: ` / data / hot`, ` / data / warm`, and ` / data / cold`. These new partitions are then mounted onto storage pools composed of different physical media (SSDs and HDDs). The batch data storage mapping is optimized, and the storage routing rules for batch data are updated. All newly generated data matching the characteristic "core knowledge point A is in the first two paragraphs" will have its write requests directly routed to the ` / data / hot` partition. Simultaneously, an index is created to map the data's logical address (e.g., answer ID + region ID) to its new physical storage location (e.g., ` / data / hot / block_123`). The secure storage result of the batch data is obtained and recorded in the logs, indicating successful partition layout adjustment and the new data mapping relationship taking effect. Furthermore, by configuring RAID1 (mirroring) on ​​the ` / data / hot` partition, redundant backup and high availability of core batch data are ensured.

[0101] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A big data-based grading optimization system, characterized in that: The system includes: The answer parsing module is based on the graded dataset. It extracts the logical structure and key information points from the graded answers, divides the content features into three categories: basic information, extended information, and redundant information, analyzes the information coverage, filters the core information areas, and obtains the answer content features and core information area set. The review weight adjustment module, based on the answer content features and the core information area set, sets the review priority according to the content features, applies weighted scoring to the core information area, adjusts the weighted scoring of the extended information area, and processes the redundant information area with low weight, thereby obtaining a review weight distribution dataset. The result matching module calls the grading weight distribution dataset, extracts the logical correlation, completeness of explanation and information contribution of the answers, classifies them according to the matching features, filters the answers within the same feature unit, and obtains the grading result comparison relationship dataset. Based on the grading result comparison dataset, the trend prediction module analyzes the changing trends and consistency of grading needs, determines the correlation of grading needs in the future time period, sets correlation judgment conditions, filters related answers, and obtains an overall trend optimization plan for grading needs.

2. The big data-based review optimization system according to claim 1, characterized in that: The set of answer content features and core information regions includes basic information, extended information, and redundant information. The grading weight distribution dataset includes weighted scoring regions, adjusted weighted scoring regions, and low-weight processing regions. The grading result comparison dataset includes answer units with consistent logical correlation and answer units with similar information contribution. The overall trend optimization scheme for grading needs includes the changing trend of grading needs, consistency of grading needs, correlation judgment conditions, and related answers.

3. The big data-based review optimization system according to claim 1, characterized in that: The answer parsing module includes: The logical structure extraction submodule, based on the grading dataset, measures and records the logical hierarchy in the answers, analyzes the logical structure and key information points in each region, and obtains content feature values ​​based on the logical and information analysis. The feature classification submodule calls the content feature value, identifies the category based on the changing trend of content features in adjacent areas, determines the feature range to which the area belongs, sets the feature classification standard, divides the area into three types of features: basic, extended, and redundant, analyzes the distribution within the area, and generates a set of answer content features and core information areas. The core information filtering submodule calls the answer content features and core information region set, identifies the distribution density of feature regions, filters high-density regions, and obtains the core information region location results.

4. The big data-based review optimization system according to claim 3, characterized in that: The review weight adjustment module includes: The priority setting submodule calls the core information area location result, sets the review priority based on content characteristics, sorts them according to the keyness of the characteristics, and obtains the area review priority data. The weight adjustment submodule calls the regional review priority data, adjusts the review weight according to the priority, calculates the core weight feature value, sets the weight score for the core information area, adjusts the weight score for the extended information area, and processes the redundant information area with low weight. It analyzes the changes in weight distribution, adjusts the size of the review dataset, and obtains the review weight distribution dataset.

5. The big data-based review optimization system according to claim 4, characterized in that: The result matching module includes: The logical correlation extraction submodule extracts the logical correlation, completeness of explanation, and information contribution of the answers based on the grading weight distribution dataset, and obtains the grading feature set; The feature classification submodule calls the review feature set, analyzes feature differences based on logical correlation, completeness of explanation and information contribution, filters answers with similar features, determines the category to which the answer belongs, divides the same feature units, optimizes the classification boundary, and obtains feature unit distribution data. The result comparison generation submodule calls the feature unit distribution data, performs local region segmentation on the answer data in each feature unit and extracts answer features, encodes the extracted features and measures similarity, filters answer data within the same unit, and obtains the grading result comparison relationship dataset.

6. The big data-based review optimization system according to claim 5, characterized in that: The extraction of logical relevance, completeness of explanation, and information contribution of the answers refers to the use of a preset natural language processing model to perform syntactic and semantic analysis on each answer text in the grading weight distribution dataset. The syntactic analysis refers to identifying sentence structure, dependency relations, and parts of speech in the answer text, while the semantic analysis refers to extracting keywords, topic vectors, and contextual information.

7. The big data-based review optimization system according to claim 5, characterized in that: The trend prediction module includes: The demand trend analysis submodule calls the review result comparison relationship dataset to analyze the trend and consistency of review demand changes, extract time series data, and obtain the characteristics of demand trend changes; The correlation judgment submodule calls the aforementioned demand trend change characteristics, analyzes the consistency between the changes and distribution of the review demands, identifies the correlation of demands, sets correlation judgment conditions, filters the correlation of demands, eliminates low correlation demands, optimizes data matching relationships, and obtains the demand correlation filtering results. The overall trend filtering submodule calls the required association filtering results, uses the distribution information between answer units to compare and analyze the positional correspondence, identifies the direction of demand change and evolution path, extracts the demand evolution stage through time series data, filters demand combination units with prominent trend changes, and obtains the overall trend optimization plan for the reviewed demands.

8. The big data-based review optimization system according to claim 7, characterized in that: The distribution information between answer units refers to the information obtained after analyzing and quantifying the spatial or structural layout of the answer content; The aforementioned demand evolution stage refers to the period in which demand changes are identified by analyzing time series data. Based on the turning points of demand changes, the entire evolution process is divided into several independent stages with differentiated characteristics.

9. The big data-based review optimization system according to claim 1, characterized in that: The system also includes a storage optimization module: Based on the overall trend optimization scheme of the review requirements, the storage optimization module extracts the distribution information of the review data, identifies the corresponding positions between data, divides the storage area according to the distribution characteristics, adjusts the storage partition layout, and obtains the result of secure storage of the review data. The secure storage results of the approval data include storage area division, storage partition layout adjustment, extraction of approval data distribution information, and identification of corresponding data locations.

10. The big data-based grading optimization system according to claim 9, characterized in that: The storage optimization module includes: The data distribution extraction submodule calls the overall trend optimization scheme of the review requirements to extract the review data distribution data, identify the corresponding position of the data, and obtain the review data distribution information; The storage area partitioning submodule calls the batch data distribution information, identifies distribution characteristics based on the corresponding data location, filters data-dense areas, partitions storage areas, sets differentiated storage area boundaries, adjusts the data distribution balance, and obtains the storage area partitioning results. The partition layout adjustment submodule calls the storage area division results, adjusts the storage partition layout, optimizes the batch data storage mapping, and obtains the secure storage results of the batch data.

Images