Peer appraisal method and device and medium
By combining rule configuration information with a declarative, programmatic configuration interface, the flexibility issues of expert matching and avoidance rules in peer review systems are resolved, enabling efficient and accurate expert screening and avoidance rule configuration, and improving the system's flexibility and adaptability.
Patent Information
- Application Number
- CN202511066895.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-31
- Publication Date
- 2025-11-14
AI Technical Summary
Existing peer review systems have limitations in expert matching logic, making it difficult to capture subtle differences and cross-domain characteristics, and complex matching rules cannot be dynamically adjusted. In terms of expert recusal, existing conflict of interest detection mechanisms are insufficient in their ability to identify conflicts of interest and cannot adapt to diverse review needs.
The system employs a hybrid rule configuration approach, combining declarative and procedural rules to determine expert selection rules. Expert selection and conflict avoidance rules are configured through both declarative and procedural configuration interfaces, enabling flexible configuration of expert matching and avoidance rules.
It improves the efficiency and accuracy of expert screening, enhances the flexibility and scalability of the peer review system, and ensures the fairness and adaptability of the review process.
Smart Images
Figure CN120952609A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a peer review method, apparatus and medium. Background Technology
[0002] Peer review is a mechanism where professionals in the same or related fields evaluate a piece of work, an achievement, or an application. It is widely used in academia, scientific research, and professional title evaluation. With the rapid development of information technology, peer review has shifted from a traditional manual matching model to an intelligent one. A peer review system, as an intelligent review management platform, can automatically match review experts, standardize the review process, and scientifically analyze the review results. However, current peer review systems have some problems.
[0003] In terms of expert matching, there are two main problems. First, simple matching rules rely on simple keyword matching or preset subject classifications. This matching method is overly simplistic, resulting in limited expressive power of the matching logic, low matching accuracy, and difficulty in capturing subtle differences and cross-domain characteristics between the content being evaluated and the expert's expertise. Second, while complex matching rules can improve matching accuracy, these rules are generally fixed in the code and cannot be dynamically adjusted according to specific scenarios, lacking flexibility and making it difficult to adapt to diverse evaluation needs.
[0004] Regarding expert recusal, existing conflict-of-interest detection mechanisms also have significant limitations. On one hand, they can only identify some superficial recusal relationships, such as whether the expert and the person being reviewed come from the same institution. For more complex relationships, such as long-term collaborations, teacher-student relationships, or competition, their ability to identify such relationships is clearly insufficient. On the other hand, recusal rules are usually fixed in the code and cannot be dynamically adjusted according to different review scenarios or the specific circumstances of the person being reviewed. This lack of flexibility limits the system's adaptability and its ability to ensure fairness in practical applications. Summary of the Invention
[0005] This application provides a peer review method, device, and medium to address the following issues in expert matching: simple matching rules have limited logical expression capabilities, low accuracy, and difficulty in capturing subtle differences and cross-domain characteristics between the reviewed content and the expert's expertise. Complex matching rules are fixed in the code, unable to be dynamically adjusted according to specific scenarios, lacking flexibility. Regarding expert avoidance, existing conflict of interest detection mechanisms can only identify some superficial avoidance relationships; their ability to identify more subtle relationships is significantly insufficient. Furthermore, avoidance rules are usually fixed in the code, unable to be dynamically adjusted according to different review scenarios or the specific circumstances of the reviewed individual.
[0006] The embodiments of this application adopt the following technical solutions:
[0007] On the one hand, embodiments of this application provide a peer review method, which includes:
[0008] Obtain the entered expert information and the information of the person being reviewed, and store the expert information in the expert data table and the information of the person being reviewed in the person being reviewed data table;
[0009] Based on the hybrid rule configuration information, the expert screening rules for each reviewer are determined; the hybrid rule configuration information includes declarative simple logic rules and / or procedural complex rules.
[0010] Based on the avoidance rule configuration information, the expert conflict avoidance rule for each person being reviewed is determined; the avoidance rule configuration information includes the conflict relationship between the expert field and the person being reviewed field.
[0011] When receiving an expert assignment request, the expert data table is matched with the reviewee data table according to the expert screening rules to obtain a list of candidate experts who meet the conditions for each reviewee.
[0012] According to the expert conflict avoidance rule, experts with conflicts are excluded from the candidate expert list to obtain the expert list;
[0013] In one example, the fields of the data table of the person being evaluated include at least one of the following: application ID, name, unit name, job title, applied major, application level, and flexible field; the flexible field is used for user-defined field configuration.
[0014] The fields in the expert data table include at least one of the following: expert name, unit name, job title, professional title level, and professional title name.
[0015] In one example, determining the expert screening rules for each reviewer based on the hybrid rule configuration information specifically includes:
[0016] The field comparison conditions and logical combination conditions between different field comparison conditions are obtained through the declarative configuration interface, and the field comparison conditions and logical combination conditions are parsed into structured logical expressions.
[0017] And / or,
[0018] The expert screening function is obtained through predefined function calls in the programmatic configuration interface; and / or, the expert screening script is obtained through the script editor in the programmatic configuration interface; the expert screening script is injected with expert information and the information of the current person being evaluated when it is executed.
[0019] In one example, determining the expert conflict avoidance rule based on the avoidance rule configuration information specifically includes:
[0020] Obtain the avoidance fields that constitute the avoidance relationship through the avoidance rule configuration interface;
[0021] Different avoidance relationships are converted into different avoidance rules;
[0022] Generate an expert conflict avoidance table based on different avoidance rules.
[0023] In one example, the fields of the expert conflict avoidance table include at least several of the following:
[0024] Rule ID, Rule Name, Expert Model, Expert Field, Reviewer Data Table, Reviewer Data Field.
[0025] In one example, the step of matching the expert data table with the reviewee data table according to the expert screening rules to obtain a list of candidate experts who meet the criteria for each reviewee specifically includes:
[0026] Load the information of the person being evaluated from the evaluation table;
[0027] According to simple logic rules, the information of the person being reviewed is matched in the expert data table to obtain a first initial candidate expert list; and / or, according to complex rules, the information of the person being reviewed is matched in the expert data table to obtain a second initial candidate expert list;
[0028] The first initial candidate expert list is determined as the candidate expert list for the person being reviewed; or, the second initial candidate expert list is determined as the candidate expert list for the person being reviewed; or, the first initial candidate expert list and the second initial candidate expert list are deduplicated and merged to obtain the candidate expert list for the person being reviewed.
[0029] The process continues until each reviewer in the reviewer data table that requires an expert to be assigned is completed, resulting in a list of qualified candidate experts for each reviewer.
[0030] In one example, assigning review experts to each reviewer based on their expert list specifically includes:
[0031] Determine whether the number of experts in the expert list of each reviewer meets the minimum threshold set by their respective expert selection rules;
[0032] Those who meet the minimum quantity threshold are identified as target respondents;
[0033] Determine the order in which the reviewers for each objective are assigned;
[0034] Based on the order in which each target is assigned to the reviewer, the designated number of target experts are selected sequentially from their respective expert lists.
[0035] In one example, the step of selecting a specified number of target experts from their respective expert lists according to the assignment order of each target reviewer specifically includes:
[0036] When performing the expert selection operation on the current target person being evaluated, the evaluation task count value associated with each expert in the expert list of the current target person being evaluated is obtained; the evaluation task count value is used to record the number of tasks currently assigned to the expert;
[0037] Select the specified number of target experts from the expert list according to the evaluation task count value from low to high.
[0038] Increment the count value of the evaluation task associated with the target expert by 1;
[0039] Continue processing the next target reviewer according to the assigned order of the target reviewers, and repeat the process until the expert selection operation for each target reviewer is completed.
[0040] On the other hand, embodiments of this application provide a peer review device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a peer review method as described above.
[0041] On the other hand, embodiments of this application provide a peer review non-volatile computer storage medium storing computer-executable instructions capable of executing any of the peer review methods described above.
[0042] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:
[0043] By configuring hybrid rule settings through both declarative and programmatic configuration interfaces, and combining simple logical rules with complex programmatic rules to determine expert screening rules, the system enables the formulation of different screening rules for different screening scenarios. This improves the efficiency and accuracy of expert screening, as well as the flexibility and scalability of the peer review system.
[0044] Furthermore, the system allows for the configuration of expert conflict avoidance rules through an interface. This enables the configuration of different expert conflict avoidance rules for different review scenarios, thereby improving the flexibility and scalability of the peer review system, as well as the accuracy of expert avoidance. Attached Figure Description
[0045] To more clearly illustrate the technical solution of this application, some embodiments of this application will be described in detail below with reference to the accompanying drawings, in which:
[0046] Figure 1 A flowchart illustrating a peer review method provided in this application embodiment;
[0047] Figure 2 A schematic diagram of a declaration configuration interface provided in an embodiment of this application;
[0048] Figure 3 A schematic diagram of a programmatic configuration interface provided in an embodiment of this application;
[0049] Figure 4 This is a schematic diagram of the structure of a peer review device provided in an embodiment of this application. Detailed Implementation
[0050] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0051] Some embodiments of this application will now be described in detail with reference to the accompanying drawings.
[0052] Figure 1 This is a flowchart illustrating a peer review method provided in an embodiment of this application. This method can be applied to different business domains. The process can be executed by computing devices in the corresponding domain, and certain input parameters or intermediate results in the process can be manually adjusted to help improve accuracy.
[0053] The analysis method involved in the embodiments of this application can be implemented by a terminal device or a server, and this application does not impose any special limitations on it. For ease of understanding and description, the following embodiments are all described in detail using a server as an example.
[0054] It should be noted that the server can be a single device or a system composed of multiple devices, i.e., a distributed server. This application does not make any specific limitations on this.
[0055] Figure 1 The process includes the following steps:
[0056] S101. Obtain the entered expert information and the information of the person being reviewed, and store the expert information in the expert data table and the information of the person being reviewed in the person being reviewed data table.
[0057] Expert information refers to data used to describe and record the experts participating in the review process. Reviewee information refers to data used to describe and record the personnel who need to be reviewed.
[0058] The expert data table and the reviewee data table are database tables used to store and manage expert information and reviewee information, respectively. They each consist of a series of fields, each containing a specific meaning and data type. Related data for both experts and reviewees are synchronized with the fields in these tables through a one-to-one mapping relationship.
[0059] Specifically, the fields in the data table of the person being evaluated include at least several of the following: applicant identification (ID), name, organization name, job title, applied specialty, applied level, and flexible fields. The fields in the data table may also include, for example, ranking, organization ID, department name, application series, applied specialty, supporting documents, attachments, etc., but this embodiment does not impose any limitations on this.
[0060] The flexible field is used for user-defined field configuration. Different information is recorded in different review scenarios. For example, in the graduation thesis review scenario, the flexible field can be configured to include information such as the thesis's innovative points, the supervisor's name, and the thesis' publication status. This application embodiment does not impose any limitations on this.
[0061] Flexible fields allow users to add or modify field content at any time without redesigning the entire data table structure, improving the scalability and flexibility of the evaluated data table.
[0062] The application ID is a field used by the person being reviewed to uniquely identify the application record, which facilitates the query and management of application records. For example, in the scenario of graduation thesis review, each student's graduation thesis application record will have a unique application ID, such as "20230518001". This ID can be used to quickly retrieve the student's thesis application information in the system.
[0063] For example, some data in the evaluation subject's data table can be shown in Table 1:
[0064] Table 1
[0065]
[0066] Specifically, the fields of the expert data table include at least several of the following: expert name, institution name, job title, professional title level, and professional title name. The fields of the expert data table may also include, for example, professional title series, but this embodiment does not impose any limitations on this.
[0067] The expert data table contains some data, as shown in Table 2:
[0068] Table 2
[0069]
[0070] S102. Determine the expert screening rules based on the hybrid rule configuration information. The hybrid rule configuration information includes declarative simple logical rules and / or programmatic complex rules.
[0071] Among them, the hybrid rule configuration information is used to define the configuration method of expert screening rules, which are rules used to select appropriate experts from the expert data table.
[0072] Declarative simple logic rules are a way of defining rules based on explicit and fixed logical conditions. These rules can be expressed, for example, in the form of field + operator + value. The logical relationships are simple and direct, and they are usually used to describe simple conditions and logical relationships.
[0073] Programmatic complex rules are a way of defining rules through programming logic. Programmatic complex rules can include complex logic, dynamic conditions, loops, conditional branches, etc. They can handle more complex scenarios and support dynamic data processing and complex logical judgments.
[0074] For example, declarative simple logic rules and procedural complex rules are classified and analyzed to clarify their respective conditions and calculation logic. Based on the expert matching needs of the evaluated person, the rule execution method is determined, and a complete screening rule structure is constructed to determine the expert screening rules.
[0075] The method of determining rule execution may include, for example, first executing declarative rules for preliminary screening, and then applying procedural rules for in-depth evaluation. Alternatively, both declarative and procedural rules may be executed simultaneously for screening. Or, only declarative rules or only procedural rules may be executed for screening. This application does not impose any limitations on these methods.
[0076] Based on this, and according to the hybrid rule configuration information, the expert screening rules are determined, specifically including:
[0077] The field comparison conditions and logical combinations of different field comparison conditions are obtained through the declarative configuration interface, and the field comparison conditions and logical combinations of conditions are parsed into structured logical expressions; and / or, the expert screening function is obtained through predefined function calls in the programmatic configuration interface; and / or, the expert screening script is obtained through the script editor in the programmatic configuration interface; the expert screening script is injected with expert information and the information of the current person being evaluated when it is executed.
[0078] The declarative configuration interface allows users to define rules by selecting and combining predefined fields, operators, and values without writing code. Figure 2 This is a schematic diagram of a declaration configuration interface provided in an embodiment of this application. Figure 2 As shown, a declarative configuration interface may include, for example, a field selector, an operator selector, a value input box, and a logic combiner. The field selector is used to select the fields to be compared, such as the expert's organization or position. The operator selector is used to select the operators for the fields, such as equal to, contain, not equal to, greater than, and less than. The value input box is used to enter the comparison value of the fields, such as training supervisor or XX position. The logic combiner is used to combine multiple conditions using logical operators such as AND, AND, and OR. This application embodiment does not impose limitations on this.
[0079] One specific implementation involves obtaining the rule comparison conditions "the applied major equals Computer Science" and "the applied level equals Advanced" through a declarative configuration interface, and then logically combining these conditions using "AND". Subsequently, the field comparison conditions and logical combination conditions are parsed into structured logical expressions to determine the expert selection rules.
[0080] The programmatic configuration interface allows users to define more complex rule logic by calling predefined functions and / or writing scripts. The predefined functions are pre-defined functions that users can directly call to complete specific tasks within the programmatic configuration interface. The expert selection function is based on a combination of predefined functions and is used to select experts who meet specific criteria according to specific logic.
[0081] Figure 3 This is a schematic diagram of a programmable configuration interface provided in an embodiment of this application, such as... Figure 3 As shown, the expert selection function can be configured from the optional function interface. On the left side of the interface, a series of predefined functions are listed, allowing users to choose the appropriate function as needed. The formula area in the middle is the main workspace for users to input and edit functions. At the bottom of the interface, users can define the logical relationships between the functions, combining them logically into a complete expert selection function.
[0082] An optional implementation involves selecting a predefined function for matching experts based on the applicant field through a programmatic configuration interface. For example, a function named FIND_EXPERTS_BY_APPLICANT_FIELD can be used as the expert screening function.
[0083] A script editor is an interface used by users to write and edit scripts, allowing users to write complex logic, such as loops, conditional statements, and calls to other built-in functions. The script can be, for example, a simplified version of Python, or a domain-specific language; this application embodiment does not impose such limitations.
[0084] For example, script code for implementing complex expert selection logic can be written through the script editor of the programmatic configuration interface, and an expert selection script can be obtained. This application embodiment does not limit the form of the script.
[0085] Based on this, the expert selection function and the expert selection script determine the expert selection rules.
[0086] S103. Determine the expert conflict avoidance rules based on the avoidance rule configuration information. The avoidance rule configuration information includes the conflict relationship between the expert field and the reviewee field.
[0087] The expert conflict avoidance rule specifies which relationships between expert fields and the reviewed party fields should be avoided. A conflict relationship refers to a connection or interest between the expert and the reviewed party that may affect the objectivity and impartiality of the review. Examples include kinship, work relationships, and economic interests; however, this embodiment does not impose such limitations.
[0088] For example, based on the avoidance rule configuration information, the conflict relationship between the expert field and the reviewee field is determined, and the fields in the expert database table that conflict with the reviewee database table are used as expert conflict avoidance rules.
[0089] Specifically, based on the avoidance rule configuration information, the expert conflict avoidance rules are determined. This includes: obtaining the avoidance fields that constitute the avoidance relationship through the avoidance rule configuration interface; converting different avoidance relationships into different avoidance rules; and generating an expert conflict avoidance table based on the different avoidance rules.
[0090] In this context, "avoidance relationship" refers to the relationship where direct review between the reviewee and the expert needs to be avoided to prevent conflicts of interest or other situations that may affect impartiality. The avoidance field is a specific information field used to determine whether an avoidance relationship exists between the expert and the reviewee. Examples of avoidance fields include institution name, project name, paper title, and kinship. This application's embodiments do not impose such limitations.
[0091] The expert avoidance conflict table identifies which experts and reviewees have a relationship requiring recusal, along with the specific reasons for recusal. The table includes at least several of the following fields: rule ID, rule name, expert model, expert field, reviewee data table, and reviewee data field. Some fields in the expert avoidance conflict table can be as follows: Figure 3 As shown:
[0092] Figure 3
[0093]
[0094] One specific implementation involves retrieving the avoidance fields that constitute the avoidance relationship based on the avoidance rule configuration interface. These fields might include the expert's and the reviewed person's affiliation, kinship, and paper title. Different avoidance relationships are then converted into different avoidance rules; for example, avoidance is required if the expert works at the same institution, if they are related by kinship, or if they have collaborated on the same paper. Based on these different avoidance rules, an expert conflict avoidance table is generated.
[0095] S104. When receiving an expert assignment request, match the expert data table with the reviewee data table according to the expert screening rules to obtain a list of candidate experts who meet the conditions for each reviewee.
[0096] For example, upon receiving an expert assignment request, the system obtains hybrid rule configuration information based on the declarative configuration interface and / or the hybrid rule configuration interface, determines the expert screening rules, and compares the information in the expert data table with the information in the reviewee data table in reverse order according to the expert screening rules to find a list of qualified experts and obtain a list of qualified candidate experts for each reviewee.
[0097] Specifically, based on the expert selection rules, the expert data table is matched with the reviewee data table to obtain a list of candidate experts who meet the criteria for each reviewee, including:
[0098] Load the information of the person being reviewed from the reviewer data table. Based on simple logical rules, match the information of the person being reviewed against the expert data table to obtain a first initial candidate expert list. And / or, based on complex rules, match the information of the person being reviewed against the expert data table to obtain a second initial candidate expert list.
[0099] The simple logic rules are the expert selection rules configured in the declarative configuration interface, while the complex rules are the expert selection rules configured in the programmatic configuration interface, including expert selection functions and / or expert selection scripts. For example, expert information such as the expert's ID and name can be added to the candidate expert list; this embodiment of the application does not impose any limitations on this.
[0100] Optionally, one specific implementation involves loading the information of the evaluated individuals from the evaluated individuals' data table upon receiving an expert assignment request. Simple logical rules are parsed into logical expressions, and these expressions are applied to each expert in the expert data table for evaluation. Expert IDs that meet the rules are then added to the first initial candidate expert list.
[0101] Optionally, one specific implementation involves loading the information of the person being reviewed from the reviewer's data table upon receiving an expert assignment request. The expert filtering function is then called, passing the reviewer's information as a parameter. This function retrieves the current reviewer's context information, such as the application ID, and queries a specified field value (e.g., the applied specialty) based on that application ID. Experts whose field values match the reviewer's specified field values are then found in the expert data table, and their matching expert IDs are added to the second initial candidate expert list.
[0102] Optionally, in one specific implementation, upon receiving an expert assignment request, the information of the person being reviewed is loaded from the reviewee database. Subsequently, the expert screening script accesses predefined context variables, which may include, for example, the information of the person being reviewed and the expert database. The rule execution engine executes the expert screening script based on the complex logic designed in its design. The expert screening script determines which experts in the expert database meet the criteria and adds the IDs of the qualified experts to a second initial candidate expert list.
[0103] The first initial list of candidate experts is determined as the candidate expert list for the person being reviewed. Alternatively, the second initial list of candidate experts is determined as the candidate expert list for the person being reviewed. Alternatively, the first initial list of candidate experts and the second initial list of candidate experts are merged after deduplication to obtain the candidate expert list for the person being reviewed. This process continues until all persons requiring expert assignment in the person being reviewed data table have been traversed, resulting in a candidate expert list that meets the criteria for each person being reviewed.
[0104] Deduplication merging refers to combining elements from two or more lists together while removing duplicate elements to generate a new list without duplicates.
[0105] S105. According to the expert conflict avoidance rule, experts with conflicts are excluded from the candidate expert list to obtain the expert list.
[0106] For example, an expert conflict avoidance table is determined according to the expert conflict avoidance rules. The expert conflict avoidance rule table is compared with the candidate expert list. Experts who are repeated in the two tables are experts with conflicts. They are removed from the candidate expert list to obtain the expert list.
[0107] S106. Assign review experts to each reviewer based on the expert list for each reviewer.
[0108] Specifically, for example, it is determined whether the number of experts in the expert list of each reviewee meets the minimum threshold set by their respective expert selection rules. Reviewees meeting the minimum threshold are identified as target reviewees. The assignment order for each target reviewee is determined. Based on the assignment order of each target reviewee, a specified number of target experts are selected sequentially from their respective expert lists.
[0109] The minimum number threshold refers to the minimum number of experts that must be met in this review task. The target reviewees refer to those who meet the expert selection rules and the minimum number threshold, and who have a sufficient number of qualified experts to choose from, enabling effective review.
[0110] Assignment order refers to the order in which experts are allocated among multiple target reviewers according to certain rules or standards. For example, the assignment order can be set based on the urgency of each reviewer's task, prioritizing time-sensitive or higher-priority review tasks. Alternatively, random assignment or round-robin mechanisms can be used to set the assignment order. This application does not impose any limitations on this.
[0111] Based on this, in order to further improve the efficiency of the evaluation, according to the assignment order of each target being evaluated, a specified number of target experts are selected sequentially from their respective expert lists, specifically including:
[0112] When performing the expert selection operation on the current target reviewer, the system retrieves the review task count value associated with each expert in the expert list for that target reviewer. The review task count value is used to record the number of tasks currently assigned to each expert.
[0113] Based on the evaluation task count value in ascending order, select the specified number of target experts from the expert list for the current target reviewer. Increment the evaluation task count value associated with each target expert by 1. Continue processing the next target reviewer according to the assignment order of the target reviewers, and repeat the process until the expert selection operation for each target reviewer is completed.
[0114] It should be noted that the embodiments of this application do not limit the way the target experts are associated with the evaluation task count. The evaluation task count can be stored in an expert database, or it can be managed and stored in the form of an in-memory data structure, a distributed database, etc., which will not be elaborated further.
[0115] By recording the evaluation task count for each expert and allocating tasks according to that count, we can ensure an even distribution of workload among experts, avoiding situations where some experts are overburdened while others are relatively idle. This ensures that each expert can complete their tasks efficiently with a reasonable workload, thus improving the efficiency of the evaluation process to some extent.
[0116] The method provided in this application allows for the configuration of mixed rule information and the determination of expert screening rules through both declarative and programmatic configuration interfaces. This method not only flexibly defines expert screening rules, improving the flexibility and scalability of the peer review system, but also combines simple logical rules with complex programmatic rules to determine expert screening rules, enabling the formulation of different screening rules for different screening scenarios and improving the efficiency and accuracy of expert screening.
[0117] Furthermore, this method allows for the configuration of expert conflict avoidance rules through an avoidance rule configuration interface. Different expert conflict avoidance rules can be configured for different review scenarios, which improves the flexibility and scalability of the peer review system, as well as the accuracy of expert avoidance, ensuring the fairness of the review process.
[0118] It should be noted that, although the embodiments in this application are based on... Figure 1 Steps S101 to S106 will be described sequentially, but this does not mean that steps S101 to S106 must be performed in a strict order. The reason this embodiment follows this order is... Figure 1 The order in which steps S101 to S106 are described is provided to facilitate understanding of the technical solutions of the embodiments of this application by those skilled in the art. In other words, in the embodiments of this application, the order of steps S101 to S106 can be appropriately adjusted according to actual needs.
[0119] Based on the same idea, some embodiments of this application also provide devices and non-volatile computer storage media corresponding to the above methods.
[0120] Figure 4 A schematic diagram of a peer review device provided in this application embodiment includes:
[0121] At least one processor. And,
[0122] A memory communicatively connected to the at least one processor.
[0123] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the peer review methods described above.
[0124] Some embodiments of this application provide a peer review non-volatile computer storage medium storing computer-executable instructions capable of executing any of the peer review methods described above.
[0125] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and medium embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the description of the method embodiments.
[0126] The devices and media provided in this application are one-to-one with the methods. Therefore, the devices and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.
[0127] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0128] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0129] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0130] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0131] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0132] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0133] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0134] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0135] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the technical principles of this application should fall within the protection scope of this application.
Claims
1. A peer review method, characterized in that, The method includes: Obtain the entered expert information and the information of the person being reviewed, and store the expert information in the expert data table and the information of the person being reviewed in the person being reviewed data table; Based on the hybrid rule configuration information, the expert screening rules for each reviewer are determined; the hybrid rule configuration information includes declarative simple logic rules and / or procedural complex rules. Based on the avoidance rule configuration information, the expert conflict avoidance rule for each person being reviewed is determined; the avoidance rule configuration information includes the conflict relationship between the expert field and the person being reviewed field. When receiving an expert assignment request, the expert data table is matched with the reviewee data table according to the expert screening rules to obtain a list of candidate experts who meet the conditions for each reviewee. According to the expert conflict avoidance rule, experts with conflicts are excluded from the candidate expert list to obtain the expert list; Based on the expert list for each person being reviewed, an expert is assigned to each person being reviewed.
2. The method according to claim 1, characterized in that, The fields in the data table of the person being evaluated include at least one of the following: application ID, name, unit name, job title, applied major, application level, and flexible fields; the flexible fields are used for user-defined field configuration. The fields in the expert data table include at least one of the following: expert name, unit name, job title, professional title level, and professional title name.
3. The method according to claim 1, characterized in that, The step of determining the expert selection rules for each reviewer based on the hybrid rule configuration information specifically includes: The field comparison conditions and logical combination conditions between different field comparison conditions are obtained through the declarative configuration interface, and the field comparison conditions and logical combination conditions are parsed into structured logical expressions. And / or, The expert screening function is obtained through predefined function calls in the programmatic configuration interface; and / or, the expert screening script is obtained through the script editor in the programmatic configuration interface; the expert screening script is injected with expert information and the information of the current person being evaluated when it is executed.
4. The method according to claim 1, characterized in that, The step of determining expert conflict avoidance rules based on avoidance rule configuration information specifically includes: Obtain the avoidance fields that constitute the avoidance relationship through the avoidance rule configuration interface; Different avoidance relationships are converted into different avoidance rules; Generate an expert conflict avoidance table based on different avoidance rules.
5. The method according to claim 4, characterized in that, The fields in the expert conflict avoidance table include at least one of the following: Rule ID, Rule Name, Expert Model, Expert Field, Reviewer Data Table, Reviewer Data Field.
6. The method according to claim 1, characterized in that, The step of matching the expert data table with the reviewee data table according to the expert screening rules to obtain a list of candidate experts who meet the criteria for each reviewee specifically includes: Load the information of the person being evaluated from the evaluation table; According to simple logic rules, the information of the person being reviewed is matched in the expert data table to obtain a first initial candidate expert list; and / or, according to complex rules, the information of the person being reviewed is matched in the expert data table to obtain a second initial candidate expert list; The first initial candidate expert list is determined as the candidate expert list for the person being reviewed; or, the second initial candidate expert list is determined as the candidate expert list for the person being reviewed; or, the first initial candidate expert list and the second initial candidate expert list are deduplicated and merged to obtain the candidate expert list for the person being reviewed. The process continues until each reviewer in the reviewer data table that requires an expert to be assigned is completed, resulting in a list of qualified candidate experts for each reviewer.
7. The method according to claim 1, characterized in that, The process of assigning review experts to each reviewer based on their expert list includes: Determine whether the number of experts in the expert list of each reviewer meets the minimum threshold set by their respective expert selection rules; Those who meet the minimum quantity threshold are identified as target respondents; Determine the order in which the reviewers for each objective are assigned; Based on the order in which each target is assigned to the reviewer, the designated number of target experts are selected sequentially from their respective expert lists.
8. The method according to claim 7, characterized in that, The step of selecting a specified number of target experts from their respective expert lists according to the assignment order of each target reviewer specifically includes: When performing the expert selection operation on the current target person being evaluated, the evaluation task count value associated with each expert in the expert list of the current target person being evaluated is obtained; the evaluation task count value is used to record the number of tasks currently assigned to the expert; Select the specified number of target experts from the expert list according to the evaluation task count value from low to high. Increment the count value of the evaluation task associated with the target expert by 1; Continue processing the next target reviewer according to the assigned order of the target reviewers, and repeat the process until the expert selection operation for each target reviewer is completed.
9. A peer review device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform a peer review method as described in any one of claims 1-8.
10. A peer-reviewed non-volatile computer storage medium storing computer-executable instructions, characterized in that, The computer-executable instructions are capable of executing a peer review method as described in any one of claims 1-8.