An online work scoring method, system and medium

Abstract

An online work scoring method, system and medium, which comprises: calculating a first dimension score based on the reward value obtained by each contestant and the maximum value in the reward value; calculating a second dimension score by using a preset first Wilson score interval algorithm based on the number of qualified manuscripts of each contestant and the total number of submitted manuscripts; obtaining the scoring information of the qualified manuscripts of each contestant, the scoring information including the judge scores and the judge eye movement data corresponding to the same qualified manuscript; adjusting the judge scores and the manuscript quality scores according to the judge eye movement data, calculating a third dimension score based on the manuscript quality scores of each contestant after the adjustment, and calculating a fourth dimension score based on the ranking position of the contestant in all contestants according to the manuscript qualified rate and the total number of contestants; and generating a scoring result according to the first dimension score, the second dimension score, the third dimension score and the fourth dimension score.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to an online work scoring method, system, and medium. Background Technology

[0002] With the popularization of digital technology, online competitions have been widely used in education, culture, science and technology and other fields. The number of entries has exploded, making it an urgent need for the industry to promote the intelligent transformation of online competition scoring.

[0003] Online entries are becoming increasingly diverse, encompassing text, images, code, and other formats, placing higher demands on the comprehensiveness and accuracy of scoring. Existing online competitions lack a comprehensive scoring mechanism, making it difficult to provide stable, comprehensive scores that reflect actual skill distribution.

[0004] In traditional online competitions, scoring often relies on a single metric, such as determining the participant's score solely based on the total score or the order of completion.

[0005] Existing online competitions lack a comprehensive scoring mechanism, making it difficult to provide stable, comprehensive, and realistic scoring results that reflect the actual distribution of abilities. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides an online work scoring method, which includes the following steps: S10. Calculate the first dimension score based on the reward value of each participant and the maximum reward value among all participants; S20. Based on the number of qualified manuscripts and the total number of manuscripts submitted by each participant, the second dimension score is calculated using the preset first Wilson score interval algorithm. S30. Obtain the scoring information of each participant's qualified manuscripts. The scoring information includes the judges' scores and judges' eye-tracking data for the same qualified manuscript. The judges' scores and manuscript quality scores were adjusted based on the judges' eye-tracking data. Based on the manuscript quality scores of each participant after the score adjustment, the third dimension score was calculated using the preset second Wilson score interval algorithm. S40. Based on the participant's current ranking in terms of manuscript pass rate among all participants and the total number of participants, calculate the fourth dimension score, and generate the score result based on the first dimension score, second dimension score, third dimension score and fourth dimension score; The scoring of judges and manuscript quality was adjusted based on the judges' eye-tracking data, including: Obtain the eye movement coordinates from the judges' eye movement data, and generate eye movement trajectories based on the eye movement coordinates; Calculate the trajectory similarity between the eye-tracking trajectory and the preset trajectory. When the trajectory similarity is greater than the trajectory threshold, adjust the judge's score corresponding to the eye-tracking trajectory, and update the manuscript quality score based on the adjusted judge's score. Obtain the fixation duration corresponding to each eye movement coordinate, and determine the region of interest in qualified manuscripts based on the fixation duration; Obtain the number of judges' attention for each region of interest, and assign weights to each region of interest based on the number of judges' attention to obtain the interest weights; Based on the interest weight and viewing time of each region of interest, the corresponding judges' scores are adjusted, and the manuscript quality score is updated based on the adjusted judges' scores.

[0007] Optionally, the algorithm for the first Wilson score interval preset in S20 is as follows: ;in, For the second dimension of scoring, To determine the pass rate of the corresponding participants' manuscripts, The total number of submissions for the corresponding participant is denoted by z, which is a preset confidence level parameter; the manuscript qualification rate is specifically the ratio of the number of qualified manuscripts for the corresponding participant to the total number of submissions. In S30, the preset algorithm for the second Wilson score interval is as follows: ;in, For the third dimension of scoring, This represents the ratio of the average quality score for each participant's submission to the maximum quality score. This corresponds to the number of qualified submissions from each participant.

[0008] Optionally, after generating the rating results based on the first dimension rating, the second dimension rating, the third dimension rating, and the fourth dimension rating, the following may also be included: At least the scores from the first, second, third, and fourth dimensions are weighted and summed to obtain the comprehensive scores of each participant, which are then sorted to obtain the initial score sequence.

[0009] Optionally, after obtaining and ranking the overall scores of all participants, the following may also be included: Determine if there are participants with the same overall score in the initial scoring sequence; if so, compare the participants with the same overall score according to the preset priority order of auxiliary indicators to determine a unique competition scoring sequence; if not, use the initial scoring sequence as the competition scoring sequence.

[0010] Optionally, participants with the same overall score can be compared according to a preset priority order of auxiliary indicators, including: Identify whether this online competition is related to award allocation; If related to award allocation, the following criteria will be compared in order: the legal status of participants with the same overall score, the reward value they have received, the manuscript qualification rate, the manuscript quality score, and the total number of manuscripts submitted. If no awards are associated, the following criteria will be compared in order of importance: the reward value, manuscript pass rate, manuscript quality score, and total number of manuscripts submitted by participants with the same overall score.

[0011] Optionally, before performing the weighted summation, the fifth dimension score can also be calculated based on the submission data of each participant within a preset historical time window. The scores from the first, second, third, fourth, and fifth dimensions are weighted and summed to obtain the comprehensive scores of each participant, which are then sorted to obtain the initial score sequence.

[0012] Optionally, participants with the same overall score can be compared according to a preset priority order of auxiliary indicators, including: Obtain the attention information of each participant's qualified submissions, and calculate the attention score of each work based on the attention information; For the same participant, the attention to their work is analyzed to obtain an attention sparsity value, and participants with the same overall score are compared based on the attention sparsity value.

[0013] Optionally, interest sparsity can be performed on the work's attention to obtain attention sparsity values, including: Feature extraction is performed on attention information to obtain behavioral features, interest features, semantic features and temporal features, and an interest feature matrix is ​​constructed based on the behavioral features, interest features, semantic features and temporal features; For the same participant, the interest overlap and distribution migration similarity between different qualified manuscripts are calculated based on the interest feature matrix. A heterogeneous graph is constructed based on the interest overlap and distribution migration similarity, and an interest overlap matrix is ​​generated based on the heterogeneous graph. Sparse coefficients are generated based on the interest overlap matrix, and the attention of works is sparsed based on the sparse coefficients to obtain the attention sparse value.

[0014] Corresponding to the aforementioned online work rating method, the present invention provides an online work rating system, which includes: The calculation module is used to calculate the first dimension score based on the maximum of the reward values ​​of each participant and all participants; to calculate the second dimension score based on the number of qualified manuscripts of each participant and the total number of manuscripts submitted, using a preset first Wilson score interval algorithm; to obtain the score information of qualified manuscripts of each participant, including the judge's score and judge's eye-tracking data for the same qualified manuscript; to adjust the judge's score and manuscript quality score based on the judge's eye-tracking data; to calculate the third dimension score based on the manuscript quality score of each participant after the score adjustment, using a preset second Wilson score interval algorithm; and to calculate the fourth dimension score based on the participant's current ranking of manuscript qualification rate among all participants and the total number of participants. The scoring output module is used to generate scoring results based on the first dimension score, the second dimension score, the third dimension score, and the fourth dimension score; The calculation module is also used to: obtain the eye movement coordinates from the judges' eye movement data, and generate eye movement trajectories based on the eye movement coordinates; Calculate the trajectory similarity between the eye-tracking trajectory and the preset trajectory. When the trajectory similarity is greater than the trajectory threshold, adjust the judge's score corresponding to the eye-tracking trajectory, and update the manuscript quality score based on the adjusted judge's score. Obtain the fixation duration corresponding to each eye movement coordinate, and determine the region of interest in qualified manuscripts based on the fixation duration; Obtain the number of judges' attention for each region of interest, and assign weights to each region of interest based on the number of judges' attention to obtain the interest weights; Based on the interest weight and viewing time of each region of interest, the corresponding judges' scores are adjusted, and the manuscript quality score is updated based on the adjusted judges' scores.

[0015] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing an online work rating program, which, when executed by a processor, implements the steps of the online work rating method described above.

[0016] Compared with the prior art, the present invention has the following beneficial effects: (1) By constructing a multi-dimensional quantitative evaluation system covering economic benefit capability (first dimension), manuscript qualification confidence (second dimension), manuscript quality confidence (third dimension), and relative score ranking (fourth dimension), this technical solution breaks through the limitations of traditional single-indicator scoring. Among them, the first dimension score reflects the economic benefit dimension performance of the participants through normalized reward values; the second and third dimension scores introduce the Wilson score interval algorithm, and when evaluating the manuscript qualification rate and quality score respectively, the confidence correction of the total sample size and the number of qualified works is included, which effectively solves the statistical fluctuation problem under small sample size, and enables a fair comparison between "a small number of high-quality works" and "a large number of medium-quality works" on the basis of statistical significance; the fourth dimension score reflects the relative level of the participants in the group through relative ranking. Based on the first, second, third, and fourth dimension scores, this solution overcomes the shortcomings of the previous technology, which is that a single indicator cannot reflect the essential differences in professional ability and lacks comprehensive consideration of multi-dimensional ability, and provides a more stable, comprehensive, and realistic competition scoring result. By adjusting judge scores and manuscript quality scores using judge eye-tracking data, the accuracy of these scores is effectively improved. By obtaining eye-tracking coordinates from the judges' eye-tracking data, eye-tracking trajectories can be effectively generated. By calculating the similarity between these trajectories and preset trajectories, judge scores corresponding to the eye-tracking trajectories can be effectively adjusted. Based on the adjusted judge scores, manuscript quality scores can be effectively updated, improving their accuracy. By obtaining the fixation duration corresponding to each eye-tracking coordinate, regions of interest (ROIs) in qualified manuscripts can be effectively identified. By obtaining the number of judges' attention points for each ROI, weights can be effectively allocated to each ROI, resulting in interest weights. Based on the interest weights and fixation durations of each ROI, corresponding judge scores can be effectively adjusted. Based on the adjusted judge scores, manuscript quality scores can be effectively updated, improving their accuracy.

[0017] (2) By employing a specific Wilson score interval algorithm to calculate the second and third dimension scores respectively, the influence of sample size on the reliability of the results is naturally incorporated into the score calculation using the confidence interval principle in statistics. Specifically, in the second dimension score calculation, the algorithm uses the total number of submitted manuscripts and the confidence level parameter to perform interval estimation correction on the manuscript pass rate, so that the score tends to be conservatively estimated when the number of submissions is small, effectively suppressing the phenomenon of inflated pass rates caused by accidental factors; similarly, in the third dimension score calculation, the algorithm corrects the quality score ratio based on the number of qualified manuscripts n, ensuring that the quality assessment results are statistically significant. This design significantly improves the statistical robustness and fairness of the scoring system when there are large differences in sample size, enabling participants with different submission sizes to conduct ability assessments at comparable confidence levels, avoiding the problem of small sample participants obtaining high scores due to good luck.

[0018] (3) By weighted integration of the above multi-dimensional indicators, the present invention can objectively distinguish the types of contestants with different skill focuses and realize the comprehensive quantitative representation of the contestants' multi-dimensional professional abilities.

[0019] (4) By setting the priority of auxiliary indicators, participants with the same comprehensive score are further subdivided and judged, which ensures the uniqueness and certainty of the competition score sequence.

[0020] (5) This invention achieves refined adaptation and flexible adjudication for competition scenarios. When the competition is associated with award allocation, the legal status of the participants is verified first to ensure the compliance of the award eligibility, and then the economic benefits, pass rate, quality and total quantity indicators are compared in turn to reflect the strict requirements of compliance in award allocation; when the competition is not associated with award allocation, the reward value obtained is directly used as the primary comparison indicator to highlight the commercial value orientation. This dynamic priority mechanism enables the scoring adjudication to flexibly adjust the evaluation focus according to the nature of the competition, ensuring the uniqueness of the score while taking into account the fairness and rationality requirements under different application scenarios, and improving the scenario adaptability of the scoring system and the configurability of the adjudication logic.

[0021] (6) By introducing a fifth dimension of scoring based on submission data within a preset historical time window, this invention incorporates the participant's historical performance into the comprehensive evaluation system, effectively reflecting the participant's continuous creative ability, long-term stability, and historical contribution. The addition of this dimension enables the scoring system to not only focus on the instantaneous performance in the current competition cycle but also to identify the participant's creative inertia, growth trend, and historical accumulation. This provides more comprehensive data support for evaluating the participant's overall professional competence and career continuity, further enhancing the accuracy of the scoring results in representing the participant's true long-term ability and reducing the impact of the randomness of a single competition performance on the overall evaluation.

[0022] (7) By obtaining the attention information of each participant's qualified manuscripts, the attention of the works can be effectively calculated. By conducting interest sparsity analysis on the attention of the works, the accuracy of the attention of the works is improved, and the problem of deviation in the attention of works caused by the same type of works is prevented.

[0023] (8) By extracting features from attention information, behavioral features, interest features, semantic features and temporal features can be effectively extracted. Based on behavioral features, interest features, semantic features and temporal features, interest feature matrix can be effectively constructed. The matrix similarity of interest feature matrices between different qualified manuscripts can be calculated to obtain interest overlap. The Walsh-Stein distance of interest feature matrices between different qualified manuscripts can be calculated to obtain distribution migration similarity. User and interest tag features of qualified manuscripts can be extracted. The extracted features can be mapped to the same high-dimensional space to obtain an initial graph. Bidirectional directed edges of manuscripts, users, user-tags and manuscript-tags can be constructed. The interest overlap and distribution migration similarity can be weighted to obtain edge weights. The weights of the bidirectional directed edges can be superimposed according to the edge weights to obtain a heterogeneous graph. The weighting coefficients in the weighting process can be set according to the requirements. The embedding vectors of the heterogeneous graph can be extracted. The vector similarity of the embedding vectors between different qualified manuscripts can be calculated. An interest overlap matrix can be generated according to the vector similarity. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a simplified flowchart of an embodiment of the online work scoring method of the present invention; Figure 2 This is a framework diagram of an embodiment of the online work rating system of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] like Figure 1 As shown, an online work scoring method of the present invention includes the following steps: S10. Calculate the first dimension score based on the reward value of each participant and the maximum reward value among all participants; S20. Based on the number of qualified manuscripts and the total number of manuscripts submitted by each participant, the second dimension score is calculated using the preset first Wilson score interval algorithm. S30. Based on the manuscript quality scores of each participant, the third dimension score is calculated using the preset second Wilson score interval algorithm. S40. Calculate the fourth dimension score based on the participant's current ranking of manuscript pass rates among all participants and the total number of participants; S50. At least the first dimension score, the second dimension score, the third dimension score and the fourth dimension score are weighted and summed to obtain the comprehensive score of each participant and sort them to obtain the initial score sequence. S60. Determine whether there are participants with the same overall score in the initial scoring sequence; if so, compare the participants with the same overall score according to the preset priority order of auxiliary indicators to determine a unique competition scoring sequence; if not, use the initial scoring sequence as the competition scoring sequence.

[0027] This invention constructs a multi-dimensional quantitative evaluation system encompassing economic benefit capability (first dimension), manuscript qualification confidence level (second dimension), manuscript quality confidence level (third dimension), and relative ranking (fourth dimension). This technical solution overcomes the limitations of traditional single-indicator ranking. Specifically, the first-dimensional score reflects the participant's economic benefit performance through normalized reward values; the second and third-dimensional scores incorporate the Wilson score interval algorithm, incorporating confidence corrections based on the total sample size and the number of qualified manuscripts when evaluating the manuscript qualification rate and quality score respectively. This effectively solves the statistical fluctuation problem under small sample sizes, allowing for a fair comparison between "a small number of high-quality works" and "a large number of medium-quality works" with comparable statistical significance; the fourth-dimensional score reflects the participant's relative level within the group through relative ranking. Furthermore, by weightedly integrating the above multi-dimensional indicators, this invention can objectively distinguish participant types with different skill focuses, achieving a comprehensive quantitative representation of participants' multi-dimensional professional abilities. This overcomes the shortcomings of previous technologies where a single indicator could not reflect the essential differences in professional abilities and lacked comprehensive consideration of multi-dimensional abilities, providing more stable, comprehensive, and realistic competition scoring results that reflect the actual distribution of abilities. Furthermore, by prioritizing auxiliary indicators, participants with the same overall score are further subdivided for adjudication, ensuring the uniqueness and certainty of the competition's scoring sequence.

[0028] For ease of understanding, the following explains some key terms in this embodiment: Online competitions refer to various activities involving the submission, review, and scoring of works conducted through online platforms, such as online programming competitions, creative design competitions, or content creation competitions. In this environment, participants submit their works online, judges review them online, and the system scores the participants according to preset rules.

[0029] Reward Value: This refers to the quantifiable representation of virtual or tangible rewards received by participants for the recognition or achievement of their work, such as project bounties, prize points, or honor points. This reward value reflects the market value or level of recognition of the participant's work.

[0030] Number of qualified submissions: This refers to the number of submissions that, after review, meet the requirements of the competition or project. This indicator reflects the quality of the submissions and their compliance with the standards.

[0031] Total number of submissions: This refers to the total number of all works submitted by participants in this competition or project, regardless of whether they are eligible. This metric reflects the participants' level of engagement and workload.

[0032] Manuscript quality score: refers to the score given by judges or the system after evaluating qualified manuscripts submitted by participants according to preset standards. It is used to measure the professional level, creativity or completion of the work.

[0033] Wilson's score interval algorithm is a statistical method for estimating the lower bound of the confidence interval for a binomial distribution proportion. It is particularly suitable for small sample data and provides more robust and reliable evaluation results than simple proportions. In this embodiment, the algorithm is used to process proportion data such as pass rates and quality scores to address the problem of statistical instability in small sample data.

[0034] Manuscript pass rate ranking: This refers to the ranking position of a participant among all participants based on their manuscript pass rate.

[0035] Overall score: This refers to the final score obtained by weighting and summing the scores from multiple dimensions. This score comprehensively reflects the participant's overall ability.

[0036] The priority order of auxiliary indicators refers to the system's comparison of multiple participants with the same overall score according to a preset sequence of indicators (such as reward value, number of qualified manuscripts, etc.) to determine a unique score.

[0037] This embodiment provides an online work scoring method, the specific implementation process of which is as follows: In step S10, the first dimension score is calculated. This score aims to measure the participant's contribution to the rewards. Specifically, it involves obtaining the cumulative reward value earned by each participant in this competition and identifying the highest reward value earned by all participants.

[0038] Preferred, first dimension score = (Participant's total reward value / maximum reward value among all participants) × 100. For example, if a participant wins a reward of 5,000 yuan, and the highest reward in this competition is 10,000 yuan, then their first-dimensional score can be calculated as 50; =(5000 / 10000)×100=50.

[0039] In step S20, the second-dimensional score is calculated, which primarily reflects the pass rate of the participants' works. Specifically, the system can count the number of qualified submissions from each participant and the total number of submissions. Subsequently, the data is processed using a preset first Wilson score interval algorithm to obtain a robust pass rate assessment value.

[0040] Preferably, the algorithm for the first Wilson score interval preset in S20 is as follows: ;in, For the second dimension of scoring, To determine the pass rate of the corresponding participants' manuscripts, The number of submissions for each participant is denoted by z, which is a preset confidence level parameter. The manuscript qualification rate is specifically the ratio of the number of qualified manuscripts for each participant to the total number of submissions.

[0041] In step S30, a third-dimensional score is calculated, which aims to assess the overall quality of the participant's work. Specifically, the system can obtain the average quality score of all qualified submissions for each participant (total qualified quality scores / number of qualified submissions). Then, a preset second Wilson score interval algorithm is used to process this quality score to obtain a reliable quality assessment value. For example, if a participant's average quality score for qualified submissions is 85 points (out of 100), the third-dimensional score calculated by this algorithm will reflect the quality level of their work.

[0042] Preferably, in S30, the preset algorithm for the second Wilson score interval is as follows: ;in, For the third dimension of scoring, This represents the ratio of the average quality score for each participant's submission to the maximum quality score. This corresponds to the number of qualified submissions from each participant.

[0043] This invention employs a specific Wilson score interval algorithm to calculate the second and third dimension scores separately. Utilizing the confidence interval principle in statistics, it naturally incorporates the impact of sample size on the reliability of the results into the scoring calculation. Specifically, in the second dimension scoring calculation, the algorithm uses the total number of submitted manuscripts and the confidence level parameters to perform interval estimation correction on the manuscript acceptance rate. This ensures that when the number of submissions is small, the score tends to be a conservative estimate, effectively suppressing the phenomenon of inflated acceptance rates due to accidental factors. Similarly, in the third dimension scoring calculation, the algorithm corrects the quality score ratio based on the number of acceptable manuscripts (n), ensuring that the quality assessment results are statistically significant. This design significantly improves the statistical robustness and fairness of the scoring system when there are large differences in sample size, allowing participants with different submission sizes to undergo competence assessment at comparable confidence levels, avoiding the problem of small sample participants receiving high scores due to luck.

[0044] In step S40, the fourth-dimensional score is calculated, which measures the participant's relative performance in terms of pass rate. Specifically, the system first calculates the pass rate of all participants' manuscripts and ranks them according to the pass rate to determine each participant's current ranking. Then, by calculating the ratio of this ranking to the total number of participants in the competition and performing appropriate normalization, the fourth-dimensional score is obtained. =(Participant's current ranking in terms of manuscript pass rate / Total number of participants) × 100.

[0045] In step S50, the calculated scores for each dimension are comprehensively processed. Specifically, the system can perform a weighted summation of at least the first, second, third, and fourth dimension scores. When performing the weighted summation, a fixed weight ratio can be preset for each dimension score; for example, the first dimension score has a weight of 45%, the second dimension score has a weight of 30%, the third dimension score has a weight of 20%, and the fourth dimension score has a weight of 5%. In this way, the comprehensive score for each participant can be obtained. The above weight ratio is only an example and can be adjusted according to different competition rules. Subsequently, all participants are ranked according to their comprehensive scores, with participants having higher comprehensive scores ranked higher, thus obtaining the initial score sequence.

[0046] In step S60, the initial scoring sequence is adjudicated to ensure the uniqueness of the final score. The system first determines whether there are participants with the same overall score in the initial scoring sequence. If no participants have the same overall score, the initial scoring sequence is determined as the final competition scoring sequence. If participants have the same overall score, the system will further compare these participants according to a preset priority order of auxiliary indicators. For example, it can compare indicators such as the participant's reward value, the number of qualified manuscripts, and the manuscript quality score. Through this multi-level comparison mechanism, a unique competition scoring sequence can be ultimately determined.

[0047] The online submission scoring method proposed in this embodiment effectively addresses the problems of incomplete evaluation, unstable statistics with small sample data, and lack of comprehensive consideration of different ability dimensions in traditional scoring algorithms by comprehensively considering multiple dimensions such as the participant's reward value, the number of qualified submissions, the submission quality score, and the ranking of submissions based on the pass rate. This method utilizes the Wilson score interval algorithm to process the pass rate and quality score, significantly improving the robustness and accuracy of evaluation with small sample data. Furthermore, through multi-dimensional weighted summation and auxiliary indicator adjudication mechanisms, the comprehensiveness, stability, and uniqueness of the scoring results are ensured, thus providing a scientific, reasonable, and stable scoring result for online competitions and promoting fair competition.

[0048] However, in practical applications, there may be situations where different participants have the same overall score. This results in ties in the initial scoring sequence, making it impossible to determine a unique competition scoring sequence, thus affecting the fairness of the scoring and the certainty of the final result.

[0049] To address this, this application further proposes that when it is determined that there are participants with the same overall score in the initial scoring sequence, the participants with the same overall score are compared according to a preset priority order of auxiliary indicators to determine a unique competition scoring sequence. In this embodiment, S60, comparing participants with the same overall score according to the preset priority order of auxiliary indicators includes: The system identifies whether the online competition is associated with award allocation. Preferably, during the competition creation or configuration phase, the system sets an identifier, such as a boolean value or an enumeration type, to indicate whether the current competition has awards. When executing the scoring and adjudication, the system reads this configuration information to determine the comparison strategy for subsequent auxiliary indicators. If related to award allocation, the following criteria will be compared in order: the legal status of participants with the same overall score, the reward value they have received, the manuscript qualification rate, the manuscript quality score, and the total number of manuscripts submitted. If no awards are associated, the following criteria will be compared in order of importance: the reward value, manuscript pass rate, manuscript quality score, and total number of manuscripts submitted by participants with the same overall score.

[0050] Preferably, "legitimacy status" refers to whether the participant meets the eligibility requirements for receiving the award, such as whether they have completed real-name authentication, and whether they meet age or geographical restrictions. Participants with a "yes" legitimacy status are generally given priority over those with a "no" legitimacy status. "Reward value" refers to the total amount of rewards earned by the participant in this competition; higher values ​​give priority. "Manuscript qualification rate" is the ratio of the number of qualified manuscripts to the total number of manuscripts submitted; higher values ​​give priority. "Manuscript quality score" refers to the average or weighted average quality score of all qualified manuscripts submitted by the participant; higher values ​​give priority. "Total number of manuscripts submitted" refers to the total number of manuscripts submitted by the participant in this competition; higher values ​​give priority. During comparison, once a certain indicator can distinguish the participants' ranking, the comparison of subsequent indicators stops, and the scoring order is determined accordingly.

[0051] This invention enables refined adaptation and flexible adjudication for competition scenarios. When a competition is associated with award allocation, the legal status of participants is verified first to ensure the compliance of award eligibility. Then, economic benefits, pass rate, quality, and total quantity indicators are compared in turn, reflecting the strict compliance requirements of award allocation. When a competition is not associated with award allocation, the reward value is directly used as the primary comparison indicator, highlighting the commercial value orientation. This dynamic priority mechanism allows the scoring adjudication to flexibly adjust the evaluation focus according to the nature of the competition. While ensuring the uniqueness of the score, it also takes into account the fairness and rationality requirements of different application scenarios, improving the scenario adaptability of the scoring system and the configurability of the adjudication logic.

[0052] Traditional online competition scoring methods typically rely on a comprehensive evaluation of participants' cumulative or average historical data (such as awards received, number of qualified submissions, total number of submissions, and submission quality scores). However, this approach may result in scores that fail to reflect participants' current skill levels, activity levels, and market relevance in a timely manner. For example, some long-term inactive participants with excellent historical data may maintain high rankings, while those with recent outstanding performance but less historical experience may struggle to stand out. This contradicts the emphasis placed on "freshness" and "current ability" in the creative design field, affecting the timeliness of the scoring and the accuracy of assessing current capabilities.

[0053] In this regard, this application further proposes that, in this embodiment S50, before performing the weighted summation, the method further includes: calculating the fifth dimension score based on the submission data of each participant within a preset historical time window; preferably, the submission data includes at least the number of submissions and the reward value obtained.

[0054] In S50, the scores of the first dimension, the second dimension, the third dimension, the fourth dimension, and the fifth dimension are weighted and summed to obtain the comprehensive score of each participant, which is then sorted to obtain the initial score sequence.

[0055] Specifically, the fifth dimension score is first calculated based on each participant's submission data within a pre-defined historical time window. This pre-defined historical time window defines a recent time range for evaluating a participant's activity and performance during that period. This time window can be flexibly configured according to the specific competition type, platform characteristics, or industry practices; for example, it can be set to the most recent 3 months, 6 months, or 1 year. By limiting the time window, it ensures that the evaluated data reflects the participant's current activity level and ability, rather than their entire historical accumulation. Submission data refers to information related to the works submitted by the participant within the aforementioned pre-defined historical time window. This data includes at least the number of submissions and the reward value received within that time window. Furthermore, it can include the number of qualified submissions and submission quality scores within that time window to more comprehensively reflect the participant's recent participation, work quality, and economic benefits. This data forms the basis for calculating the fifth dimension score, used to quantify the participant's overall performance in the recent period. The calculation of the fifth dimension score aims to transform the participant's submission data within the pre-defined historical time window into a quantifiable score. Specifically, the score can be calculated by combining weighted averages, normalization, or statistical methods based on indicators such as the number of submissions, the amount of awards received, the number of qualified manuscripts, and the manuscript quality score. This score is typically normalized to a range of 0 to 100 to facilitate comparison with other dimensions and weighted summation.

[0056] Subsequently, when calculating the overall score for each participant, the scores from the first, second, third, and fourth dimensions, as well as the newly added fifth dimension, are weighted and summed. This means that each dimension score is assigned a preset weight, reflecting the importance of that dimension in the overall evaluation. In this way, the participant's performance across different dimensions can be integrated into a single overall score, thus achieving a comprehensive assessment of the participant's overall abilities. With the introduction of the fifth dimension, the weights of the original dimensions need to be appropriately adjusted to ensure that the fifth dimension score effectively reflects the participant's recent performance and, together with the other dimensions, constitutes a balanced and comprehensive evaluation system.

[0057] By introducing a fifth-dimensional score based on each participant's submission data within a preset historical time window and incorporating it into the weighted sum of the overall score, this method effectively addresses the problem of traditional scoring algorithms failing to reflect participants' recent performance in a timely manner. Specifically, the introduction of the fifth-dimensional score allows the system to quantify a participant's activity level, output, and earnings within a specific time period, thereby more accurately capturing their current skill level, market adaptability, and participation. This avoids the potential lag that can result from relying solely on accumulated historical data, making the scoring results more timely and valuable. For example, for newcomers with outstanding recent performance but limited historical data, the fifth-dimensional score provides additional weight and recognition, enabling them to stand out more quickly. Conversely, for participants who have been inactive for a long time but have excellent historical data, a decrease in their fifth-dimensional score effectively prevents them from maintaining a high ranking due to historical advantages, thus ensuring their continued competitiveness. This mechanism makes the final overall score not only comprehensive and stable but also more "real-time," providing a more scientific, reasonable, and dynamic competition scoring sequence.

[0058] This invention introduces a fifth dimension of scoring based on submission data within a preset historical time window. This incorporates the participant's historical performance into a comprehensive evaluation system, effectively reflecting their sustained creative ability, long-term stability, and historical contributions. The addition of this dimension allows the scoring system to not only focus on the immediate performance of the current competition cycle but also to identify the participant's creative inertia, growth trends, and historical accumulation. This provides more comprehensive data support for evaluating the participant's overall professional competence and career sustainability, further enhancing the accuracy of the scoring results in representing the participant's true long-term ability and reducing the impact of the randomness of a single competition performance on the overall evaluation.

[0059] In some of the above implementations, although a fifth dimension of scoring is introduced to consider the participant's submission data within a preset historical time window, in practical applications, simply weighting and summing the fifth dimension score with other dimension scores with fixed weights may not fully reflect the participant's current actual ability and activity level. Especially for participants with a small number of submissions and insufficient historical data accumulation, the importance of their recent performance is often diluted, resulting in scoring results that fail to reflect their potential in a timely and accurate manner.

[0060] Therefore, in this embodiment, the scores of the first dimension, second dimension, third dimension, fourth dimension, and fifth dimension are weighted and summed, specifically as follows: The weighting of the fifth dimension score in the weighted sum is dynamically adjusted based on the total number of submissions made by the participants; preferably, the fewer the total number of submissions made by the participants, the higher the weighting of the fifth dimension score in the weighted sum.

[0061] Specifically, the total number of submissions refers to the number of all submissions a participant has made to the platform or system since joining the competition. This metric reflects the participant's historical participation and data accumulation. During implementation, the system continuously tracks and records this data for each participant, using it as a key basis for judging the richness of their historical data. Dynamically adjusting the weighting of the fifth dimension score in the weighted sum means that the importance of the fifth dimension score in calculating the overall score is not fixed but changes in real-time or periodically according to specific rules. This adjustment can be achieved through a preset function or lookup table, where the input is the total number of submissions and the output is the weighting of the fifth dimension score. For example, a threshold can be set; when the total number of submissions is below this threshold, the weighting decreases; when it is above the threshold, the weighting remains unchanged or decreases slowly. The fewer submissions a participant makes, the higher the weighting of the fifth dimension score in the weighted sum—this is a specific dynamic adjustment strategy. The core idea is that for participants with limited historical data, their recent performance (reflected by the fifth dimension score) is a more significant indicator of their current abilities and potential, and therefore should be given higher weight. Conversely, for participants with abundant historical data, their long-term performance is relatively stable, and fluctuations in recent performance have a relatively smaller impact on the overall evaluation, so the weight of the fifth dimension score can be appropriately reduced.

[0062] The submission data of this invention includes at least the number of submissions and the reward value obtained, ensuring that the fifth dimension score can simultaneously reflect the participant's historical creative activity and historical economic benefit capability; it takes into account both the scale of output and the market verification of output quality (reflected through reward value), avoiding the evaluation bias that may be caused by a single historical indicator, and improving the representativeness, credibility and complementarity of the historical dimension score with the current competition dimension.

[0063] Furthermore, by dynamically adjusting the weighting of the fifth dimension's score based on the total number of submissions, a protective weighting mechanism and sample compensation strategy for inactive participants are implemented. Specifically, when a participant submits fewer submissions, the system automatically increases the weight of historical performance in the overall score, allowing novice or infrequent participants to obtain a more reasonable comprehensive evaluation based on their historical accumulation. This effectively offsets the excessive negative impact of statistical randomness from a small number of submissions in the short term on their overall score. Conversely, for highly active participants, current performance dominates, and historical weight is correspondingly reduced. This dynamic weighting mechanism significantly improves the inclusivity and fairness of the scoring system for different participant groups.

[0064] The core of the aforementioned online judging method lies in comprehensively evaluating participants through multi-dimensional scoring. However, if the original participant data used to calculate these scores (such as the total number of submissions, the number of qualified submissions, submission quality scores, and award values) is not generated and managed in a timely, accurate, and reliable manner, subsequent scoring calculations and results will face issues of data lag, inconsistency, and even errors, thus affecting the effectiveness and fairness of the entire judging method. Specifically, the lack of a dynamic data generation and persistence mechanism closely integrated with the competition's workflow will prevent the underlying data used for scoring from reflecting the participant's latest status in real time, thereby making the scoring results unable to accurately reflect the participant's true abilities and contributions.

[0065] In this embodiment, the data of each participant is generated and persistently stored in the following manner: In response to a participant's submission, the system updates the total number of submissions for that participant. This mechanism aims to track and record each participant's workload and participation level in real time. When a participant successfully uploads or submits a submission through the competition platform's user interface or application programming interface (API), the system captures this "submission operation" event. In response, the system's backend service or data processing module immediately performs an atomic increment operation on the "total number of submissions" field corresponding to that participant in the database. This process ensures that this metric accurately reflects the participant's latest submission status, providing foundational data for subsequent dimensional scoring calculations.

[0066] In response to judges' approval of submitted manuscripts, the system updates the number of approved manuscripts and the manuscript quality score for each participant. This step ensures that the quality and success rate of participants' work can be quantified in a timely manner. When a judge completes their review of a participant's submission and marks it as "approved," the system triggers the corresponding "Approved" event. In response, the system immediately updates the participant's "Number of Approved Manuscripts" field, incrementing it. Simultaneously, based on the judges' specific scores for the approved manuscripts, the system updates the participant's "Manuscript Quality Score." This update may involve incorporating the new score into an average calculation or aggregating it using other preset methods to reflect the overall quality level of the participant's work. These update operations should also ensure data consistency through database transactions.

[0067] In response to reward distribution events associated with qualified submissions, the system updates the corresponding reward value for each participant. This mechanism accurately records the economic or other forms of reward a participant receives for their qualified work. Once a qualified submission meets the reward distribution criteria (e.g., winning an award or reaching a specific prize threshold), the system triggers a "reward distribution event." In response, the system adds the reward amount or value to the participant's "Reward Value" field stored in the database. This ensures that the participant's total reward value remains up-to-date, providing accurate data for assessing their economic earning potential.

[0068] The participant data includes the number of qualified submissions, submission quality scores, total number of submissions, and prize value. This clearly defines the four core raw indicators that form the basis of the online competition's scoring method. These data points are key attributes maintained by the system for each participant; they originate directly from the aforementioned dynamic generation process and are persistently stored. Specifically, the "number of qualified submissions" reflects the pass rate; the "submission quality score" quantifies the professional level of the work; the "total number of submissions" reflects the participant's participation and investment; and the "prize value" measures the market value or recognition of their work. These data collectively constitute the direct input for the multi-dimensional intelligent scoring method to calculate scores across various dimensions.

[0069] In this embodiment, the data of each participant is generated and persistently stored in the following way: throughout the entire online competition, key information related to each participant (such as their submission status, judging results, and awards) is dynamically created, updated, and saved in a reliable and non-volatile manner. This persistent storage is typically implemented through a database system, such as a relational database (e.g., MySQL, PostgreSQL) or a NoSQL database (e.g., MongoDB), to ensure data integrity, consistency, and availability after a system restart. Data generation means that this data is not a preset static value, but rather generated in real time based on the actual behavior of the participants and system events during the competition.

[0070] This invention establishes a data update mechanism triggered by specific operational events (manuscript submission, review approval, and award distribution), enabling real-time generation and persistent storage of participant data. This event-driven data management model ensures that participant data (number of qualified manuscripts, manuscript quality scores, total number of submitted manuscripts, and award value) remains highly synchronized with the actual competition process, providing an accurate, timely, and consistent data foundation for subsequent multi-dimensional scoring calculations. Simultaneously, persistent storage guarantees data integrity, traceability, and system fault recovery capabilities, laying a solid data layer support for accurate scoring calculations.

[0071] In some of the embodiments described above in this application, an online work scoring method is proposed. This method, through multi-dimensional scoring calculation and ranking, can comprehensively evaluate the overall ability of participants. Simultaneously, various data of the participants (such as the total number of submissions, the number of qualified submissions, the submission quality score, and the reward value) are dynamically generated and persistently stored as events such as submission, approval by the review committee, and reward distribution occur. However, if the scoring sequence fails to respond promptly to these dynamic data changes, the scoring results may be lagging, failing to reflect the latest performance of the participants and the current status of the competition in real time, thereby affecting the accuracy and fairness of the scoring.

[0072] In this regard, this application further proposes that, in this embodiment, when any manuscript submission operation, review approval operation, or award distribution event is detected, the corresponding participant data is updated and S10-S60 is re-executed to regenerate the competition scoring sequence based on the updated participant data.

[0073] Specifically, "detecting any submission, approval, or award distribution event" means that the system can detect key business events related to changes in participant data in real time or near real time. "Submission" refers to the process of a participant uploading their work to the competition platform, which the system can monitor by listening to frontend submission requests, API calls, or submission events in the message queue. "Approval" refers to the action of judges marking a participant's submission as "approved" after review; this can be monitored by listening to confirmation button clicks on the judge's interface, backend service calls, or changes to the submission status field in the database. "Award distribution event" refers to the actual or confirmed distribution of rewards (such as prize money or points) associated with a participant's approved submission; this can be monitored by listening to notifications from the reward distribution module, transaction records in the financial system, or updates to the reward status field in the database. This monitoring mechanism ensures that the system can promptly capture all raw data changes that may affect participant scores.

[0074] Upon detecting the aforementioned events, the system immediately modifies the relevant statistical data for the affected participants, i.e., "updates the corresponding participant data." For example, when a submission is detected, the system increases the total number of submissions by that participant. When a judging pass is detected, the system increases the number of qualified submissions by that participant and updates their submission quality score based on the judges' scores (e.g., by calculating the average or weighted average score). When an award distribution event is detected, the system increases the award value received by that participant. These update operations typically involve atomic modifications to the participant data records stored in the database to ensure data consistency.

[0075] After updating the data of the affected participants, the system will re-execute all scoring calculation steps from S10 to S60, i.e., "re-execute S10-S60". S10 to S40 involve calculating the first-dimensional score, second-dimensional score, third-dimensional score, and fourth-dimensional score. The calculation of these dimensional scores may not only depend on the data of individual participants, but also on the global data of all participants (for example, the first-dimensional score depends on the maximum value of the reward value of all participants, and the fourth-dimensional score depends on the participant's ranking of the manuscript acceptance rate among all participants and the total number of participants). S50 involves weighted summation of the scores of each dimension to obtain a comprehensive score, and sorting them to obtain an initial score sequence. S60 involves comparison according to the priority order of preset auxiliary indicators in the case of the same comprehensive score to determine a unique competition score sequence. "Re-execute" means that the system will recalculate all scores and ratings from beginning to end for all participants, based on their latest updated data, rather than just updating the scores of individual participants whose data has changed.

[0076] By re-executing the complete scoring calculation process described above, the system will ultimately produce a completely new competition scoring sequence that reflects the latest state of the competition, namely, "a competition scoring sequence regenerated based on the updated data of each participant." This new initial scoring sequence is calculated based on the latest data set of all participants after all relevant events have occurred, ensuring the timeliness and accuracy of the scoring.

[0077] In other embodiments of the present invention, in this embodiment, S60 compares participants with the same overall score according to a preset priority order of auxiliary indicators, including: The process involves obtaining information on the level of attention for each participant's qualified submissions and calculating the level of attention based on this information. This information includes user views, likes, favorites, shares, completion rate, average dwell time, repeat visit rate, and comments for qualified submissions displayed online or offline. The parameter values ​​for each attention parameter are calculated based on their preset weighting coefficients to obtain the level of attention for each submission. For the same participant, the attention to their works is sparsed to obtain an attention sparsity value. Participants with the same overall score are then compared based on this attention sparsity value. By sparsening the attention to works, the accuracy of attention is improved, and the problem of biased attention caused by similar works is prevented.

[0078] Furthermore, interest sparsity is calculated based on the attention received by the work, resulting in attention sparsity values, including: Feature extraction is performed on attention information to obtain behavioral features, interest features, semantic features, and temporal features. An interest feature matrix is ​​then constructed based on these features. Behavioral features include mapping values ​​for user likes, collections, shares, and reading completion rates. Interest features include mapping values ​​for average user dwell time and repeat visit rate. Semantic features include sentiment weights of comment keywords, interest tag matching degree, and user profile alignment degree. Temporal features include attention growth slope, peak attention duration, and natural traffic percentage. For each feature, the feature parameters are mapped according to a preset matrix mapping rule to obtain sub-feature matrices. These sub-feature matrices are then superimposed to obtain the interest feature matrix. For the same participant, the interest overlap and distribution migration similarity between different qualified manuscripts are calculated based on the interest feature matrix. A heterogeneous graph is constructed based on the interest overlap and distribution migration similarity, and an interest overlap matrix is ​​generated based on the heterogeneous graph. Specifically, the matrix similarity of the interest feature matrices between different qualified manuscripts is calculated to obtain the interest overlap. The Walsh-Stein distance of the interest feature matrices between different qualified manuscripts is calculated to obtain the distribution migration similarity. User and interest tag features of qualified manuscripts are extracted and mapped to the same high-dimensional space to obtain an initial graph. Bidirectional directed edges are constructed between manuscripts, users, user-tags, and manuscript-tags. The interest overlap and distribution migration similarity are weighted to obtain edge weights. The weights of the bidirectional directed edges are superimposed based on the edge weights to obtain the heterogeneous graph. The weighting coefficients in the weighting process can be set according to requirements. The embedding vectors of the heterogeneous graph are extracted, and the vector similarity of the embedding vectors between different qualified manuscripts is calculated. An interest overlap matrix is ​​generated based on the vector similarity. Sparse coefficients are generated based on the interest overlap matrix, and the attention of the work is sparsified based on the sparse coefficients to obtain the attention sparse value; wherein, the interest overlap matrix is ​​decoded by a pre-trained matrix decoder to obtain the sparse coefficients, and the product between the sparse coefficients and the attention of the work is calculated to obtain the attention sparse value.

[0079] Preferably, before calculating the third dimension score using the preset second Wilson score interval algorithm based on the manuscript quality scores of each participant, the following steps are also included: Obtain the scoring information of each participant's qualified manuscripts, including the judges' scores and eye-tracking data for the same qualified manuscript; The scoring of judges and manuscript quality is adjusted based on the judges' eye-tracking data. In particular, adjusting the scoring of judges and manuscript quality based on the judges' eye-tracking data effectively improves the accuracy of the judges' scores and manuscript quality scores. In this step, the manuscript quality score can be obtained by calculating the average of the judges' scores for the same qualified manuscript.

[0080] Furthermore, the judges' scores and manuscript quality scores were adjusted based on the judges' eye-tracking data, including: Obtain eye movement coordinates from the judges' eye movement data and generate eye movement trajectories based on the eye movement coordinates; specifically, for each judge of a qualified manuscript, connect the corresponding eye movement coordinates in sequence to obtain the eye movement trajectory; The similarity between the eye-tracking trajectory and a preset trajectory is calculated. When the similarity exceeds a threshold, the judge's score corresponding to the eye-tracking trajectory is adjusted, and the manuscript quality score is updated based on the adjusted judge's score. The preset trajectory can be set according to needs, such as a horizontal line from top to bottom or from left to right. When the similarity exceeds the threshold, it is determined that the current judge is not focused on scoring. By using a preset adjustment coefficient, the weight of the judge's score corresponding to the eye-tracking trajectory in the manuscript quality score is reduced, thereby reducing the impact of the current judge's score on the manuscript quality score. Obtain the fixation duration corresponding to each eye movement coordinate, and determine the region of interest in the qualified manuscript based on the fixation duration; wherein, the different regions in the qualified manuscript are marked with the fixation duration, and if the total fixation duration in any region is greater than the duration threshold, the region is set as the region of interest. The duration threshold can be set according to the requirements. The number of judges' attention corresponding to each region of interest is obtained, and the regions of interest are weighted according to the number of judges' attention to obtain the interest weight. Specifically, for a region of interest, if the gaze duration of any judge is greater than the duration threshold, the number of judges' attention to that region of interest is incremented by 1. The final number of judges' attention corresponding to each region of interest is counted, and the interest weight is calculated based on the number of judges' attention and the total number of judges. When the number of judges' attention corresponding to any region of interest is higher, the quality of that region of interest is judged to be better. Based on the interest weight and gaze duration of each region of interest, the corresponding judges' scores are adjusted, and the manuscript quality score is updated based on the adjusted judges' scores. Specifically, for each qualified manuscript, the total interest weight and the corresponding total gaze duration are calculated, and a weighted interest score is obtained by weighting the total interest weight and the corresponding total gaze duration. The corresponding judges' scores are then adjusted based on the interest weight score. In this step, the higher the total interest weight and the corresponding total gaze duration, the higher the interest weight score, that is, the higher the judges' quality evaluation of the current qualified manuscript.

[0081] This invention achieves a dynamic, real-time update mechanism for the competition scoring sequence by monitoring key operational events and triggering score recalculation. When data changes affecting the scoring occur, such as manuscript submission, review approval, or award distribution, the system automatically re-executes the multi-dimensional scoring calculation process (S10-S60), ensuring that the scoring results reflect the latest competition situation and changes in participant performance in real time. This real-time feedback mechanism significantly improves the response speed and data timeliness of the scoring system, ensuring that competition scoring is always based on the latest and most complete participant performance data. This enhances the dynamic accuracy, real-time nature, and transparency of competition management, thereby improving the participant experience and the credibility of the competition.

[0082] like Figure 2 As shown, the present invention also provides an online work rating system 100, which includes: The calculation module 10 is used to calculate the first dimension score based on the maximum value of the reward value of each participant and the reward value of all participants; to calculate the second dimension score based on the number of qualified manuscripts of each participant and the total number of manuscripts submitted, using a preset first Wilson score interval algorithm; to obtain the score information of qualified manuscripts of each participant, including the judge's score and judge's eye movement data corresponding to the same qualified manuscript; to adjust the judge's score and manuscript quality score according to the judge's eye movement data; to calculate the third dimension score based on the manuscript quality score of each participant after the score adjustment, using a preset second Wilson score interval algorithm; and to calculate the fourth dimension score based on the participant's current ranking of manuscript qualification rate among all participants and the total number of participants. The scoring output module 20 is used to generate scoring results based on the first dimension score, the second dimension score, the third dimension score, and the fourth dimension score.

[0083] The calculation module 10 is also used to: obtain the eye movement coordinates in the judge's eye movement data, and generate an eye movement trajectory based on the eye movement coordinates; Calculate the trajectory similarity between the eye-tracking trajectory and the preset trajectory. When the trajectory similarity is greater than the trajectory threshold, adjust the judge's score corresponding to the eye-tracking trajectory, and update the manuscript quality score based on the adjusted judge's score. Obtain the fixation duration corresponding to each eye movement coordinate, and determine the region of interest in qualified manuscripts based on the fixation duration; Obtain the number of judges' attention for each region of interest, and assign weights to each region of interest based on the number of judges' attention to obtain the interest weights; Based on the interest weight and viewing time of each region of interest, the corresponding judges' scores are adjusted, and the manuscript quality score is updated based on the adjusted judges' scores.

[0084] The sorting module 30 is used to perform a weighted summation of at least the first dimension score, the second dimension score, the third dimension score and the fourth dimension score to obtain the comprehensive score of each participant and sort them to obtain the initial score sequence. The scoring and adjudication module 40 is used to determine whether there are participants with the same overall score in the initial scoring sequence. If there are, the participants with the same overall score are compared according to the preset priority order of auxiliary indicators to determine a unique competition scoring sequence. If there are no participants, the initial scoring sequence is used as the competition scoring sequence.

[0085] Preferably, before the sorting module 30 performs a weighted summation of the first dimension score, the second dimension score, the third dimension score, and the fourth dimension score, it also calculates the fifth dimension score based on the submission data of each participant within a preset historical time window; then, it performs a weighted summation of the first dimension score, the second dimension score, the third dimension score, the fourth dimension score, and the fifth dimension score to obtain the comprehensive score of each participant and sorts them to obtain the initial score sequence.

[0086] Furthermore, the system also includes a data update module, which is used to respond to the submission operation of the contestant's manuscript and update the total number of manuscripts submitted by the corresponding contestant; to respond to the judge's approval operation of the submitted manuscript of the contestant and update the number of qualified manuscripts and manuscript quality score of the corresponding contestant; and to respond to the reward distribution event associated with qualified manuscripts and update the reward value received by the corresponding contestant.

[0087] Preferably, the system further includes a scoring update module, which updates the corresponding participant data and calls the calculation module 10, sorting module 30 and scoring adjudication module 40 to re-execute the corresponding functions when any manuscript submission operation, review approval operation or award distribution event is detected, so as to regenerate the competition scoring sequence based on the updated participant data.

[0088] The scoring and adjudication module 40 is also used to: obtain the attention information of each participant's qualified submissions, and calculate the attention of the works based on the attention information; For the same participant, the attention to their work is analyzed to obtain an attention sparsity value, and participants with the same overall score are compared based on the attention sparsity value.

[0089] Furthermore, the scoring and adjudication module 40 is also used to: extract features from attention information to obtain behavioral features, interest features, semantic features and temporal features, and construct an interest feature matrix based on the behavioral features, interest features, semantic features and temporal features; For the same participant, the interest overlap and distribution migration similarity between different qualified manuscripts are calculated based on the interest feature matrix. A heterogeneous graph is constructed based on the interest overlap and distribution migration similarity, and an interest overlap matrix is ​​generated based on the heterogeneous graph. Sparse coefficients are generated based on the interest overlap matrix, and the attention of works is sparsed based on the sparse coefficients to obtain the attention sparse value.

[0090] This invention also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the memory described in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into a device. The computer-readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement... Figure 1 The online work scoring method is shown. The computer-readable storage medium may be a read-only memory, a disk, or an optical disk, etc.

[0091] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments and storage medium embodiments, since they are basically similar to method embodiments, the descriptions are relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0092] Furthermore, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0093] The foregoing description illustrates and describes preferred embodiments of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the inventive concept by means of the foregoing teachings or techniques or knowledge in related fields. Any modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the present invention.

Claims

1. An online work scoring method, characterized in that, Includes the following steps: S10. Calculate the first dimension score based on the reward value of each participant and the maximum reward value among all participants; S20. Based on the number of qualified manuscripts and the total number of manuscripts submitted by each participant, the second dimension score is calculated using the preset first Wilson score interval algorithm. S30. Obtain the scoring information of each participant's qualified manuscripts. The scoring information includes the judges' scores and judges' eye-tracking data for the same qualified manuscript. The judges' scores and manuscript quality scores were adjusted based on the judges' eye-tracking data. Based on the manuscript quality scores of each participant after the score adjustment, the third dimension score was calculated using the preset second Wilson score interval algorithm. S40. Based on the participant's current ranking in terms of manuscript pass rate among all participants and the total number of participants, calculate the fourth dimension score, and generate the score result based on the first dimension score, second dimension score, third dimension score and fourth dimension score; The scoring of judges and manuscript quality was adjusted based on the judges' eye-tracking data, including: Obtain the eye movement coordinates from the judges' eye movement data, and generate eye movement trajectories based on the eye movement coordinates; Calculate the trajectory similarity between the eye-tracking trajectory and the preset trajectory. When the trajectory similarity is greater than the trajectory threshold, adjust the judge's score corresponding to the eye-tracking trajectory, and update the manuscript quality score based on the adjusted judge's score. Obtain the fixation duration corresponding to each eye movement coordinate, and determine the region of interest in qualified manuscripts based on the fixation duration; Obtain the number of judges' attention for each region of interest, and assign weights to each region of interest based on the number of judges' attention to obtain the interest weights; Based on the interest weight and viewing time of each region of interest, the corresponding judges' scores are adjusted, and the manuscript quality score is updated based on the adjusted judges' scores.

2. The online work scoring method according to claim 1, characterized in that, The algorithm for the first Wilson score interval preset in S20 is as follows: ;in, For the second dimension of scoring, To determine the pass rate of the corresponding participants' submissions, The total number of submissions for the corresponding participant is denoted by z, which is a preset confidence level parameter; the manuscript qualification rate is specifically the ratio of the number of qualified manuscripts for the corresponding participant to the total number of submissions. In S30, the preset algorithm for the second Wilson score interval is as follows: ;in, For the third dimension of scoring, This represents the ratio of the average quality score for each participant's submission to the maximum quality score. This corresponds to the number of qualified submissions from each participant.

3. The online work scoring method according to claim 1, characterized in that, After generating the rating results based on the first, second, third, and fourth dimension ratings, the following are also included: At least the scores from the first, second, third, and fourth dimensions are weighted and summed to obtain the comprehensive scores of each participant, which are then sorted to obtain the initial score sequence.

4. The online work scoring method according to claim 3, characterized in that, After obtaining and ranking the overall scores of all participants, the following also applies: Determine if there are participants with the same overall score in the initial scoring sequence; if so, compare the participants with the same overall score according to the preset priority order of auxiliary indicators to determine a unique competition scoring sequence; if not, use the initial scoring sequence as the competition scoring sequence.

5. The online work scoring method according to claim 4, characterized in that, Based on the preset priority order of auxiliary indicators, participants with the same overall score are compared, including: Identify whether this online competition is related to award allocation; If related to award allocation, the following criteria will be compared in order: the legal status of participants with the same overall score, the reward value they have received, the manuscript qualification rate, the manuscript quality score, and the total number of manuscripts submitted. If no awards are associated, the following criteria will be compared in order of importance: the reward value, manuscript pass rate, manuscript quality score, and total number of manuscripts submitted by participants with the same overall score.

6. The online work scoring method according to claim 3, characterized in that, Before performing the weighted summation, the fifth dimension score is calculated based on the submission data of each participant within a preset historical time window. The scores from the first, second, third, fourth, and fifth dimensions are weighted and summed to obtain the comprehensive scores of each participant, which are then sorted to obtain the initial score sequence.

7. The online work scoring method according to claim 4, characterized in that, In S60, participants with the same overall score are compared according to the preset priority order of auxiliary indicators, including: Obtain the attention information of each participant's qualified submissions, and calculate the attention score of each work based on the attention information; For the same participant, the attention to their work is analyzed to obtain an attention sparsity value, and participants with the same overall score are compared based on the attention sparsity value.

8. The online work scoring method according to claim 7, characterized in that, Interest sparsity is calculated based on the attention received by a work, resulting in attention sparsity values, including: Feature extraction is performed on attention information to obtain behavioral features, interest features, semantic features and temporal features, and an interest feature matrix is ​​constructed based on the behavioral features, interest features, semantic features and temporal features; For the same participant, the interest overlap and distribution migration similarity between different qualified manuscripts are calculated based on the interest feature matrix. A heterogeneous graph is constructed based on the interest overlap and distribution migration similarity, and an interest overlap matrix is ​​generated based on the heterogeneous graph. Sparse coefficients are generated based on the interest overlap matrix, and the attention of works is sparsed based on the sparse coefficients to obtain the attention sparse value.

9. An online work rating system, characterized in that, include: The calculation module is used to calculate the first dimension score based on the maximum value of each participant's reward value and the total reward values ​​of all participants; Based on the number of qualified manuscripts and the total number of manuscripts submitted by each participant, the second dimension score is calculated using a preset first Wilson score interval algorithm; the score information of each participant's qualified manuscripts is obtained, including the judges' scores and judges' eye-tracking data for the same qualified manuscript; the judges' scores and manuscript quality scores are adjusted based on the judges' eye-tracking data; based on the adjusted manuscript quality scores of each participant, the third dimension score is calculated using a preset second Wilson score interval algorithm; the fourth dimension score is calculated based on the participant's current ranking in terms of manuscript qualification rate among all participants and the total number of participants. The scoring output module is used to generate scoring results based on the first dimension score, the second dimension score, the third dimension score, and the fourth dimension score; The calculation module is also used to: obtain the eye movement coordinates from the judges' eye movement data, and generate eye movement trajectories based on the eye movement coordinates; Calculate the trajectory similarity between the eye-tracking trajectory and the preset trajectory. When the trajectory similarity is greater than the trajectory threshold, adjust the judge's score corresponding to the eye-tracking trajectory, and update the manuscript quality score based on the adjusted judge's score. Obtain the fixation duration corresponding to each eye movement coordinate, and determine the region of interest in qualified manuscripts based on the fixation duration; Obtain the number of judges' attention for each region of interest, and assign weights to each region of interest based on the number of judges' attention to obtain the interest weights; Based on the interest weight and viewing time of each region of interest, the corresponding judges' scores are adjusted, and the manuscript quality score is updated based on the adjusted judges' scores.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an online work rating program, which, when executed by a processor, implements the steps of the online work rating method as described in any one of claims 1 to 8.

Images