Apparatus and method for evaluating skewness of teacher rating on defined rating scale

Abstract

Apparatus, methods, systems, and program products are disclosed for evaluating skewness of teacher ratings on a defined rating scale. An apparatus includes a processor and a memory storing code executable by the processor. The code, when executed by the processor, receives an original teacher rating dataset on a defined rating scale, the original teacher rating dataset including a plurality of ratings from individual teachers. The code is executable to determine a rating skewness of the original teacher dataset by subtracting a central tendency metric of the plurality of ratings from a central tendency metric of a defined rating scale and dividing the result by a standard deviation of the plurality of ratings. The code is further executable to compare a rating skewness of the original teacher rating dataset to at least one comparative rating skewness to generate a skewness comparison report. The code is also executable to determine whether the individual instructors require instructor training based on the skewness comparison report and identify the required instructor training to be provided to the individual instructors. The code is also executable to enable one or more end users to access the data.

Description

Technical Field

[0001] The present disclosure relates generally to evaluating faculty ratings, and more particularly, to evaluating the skewness of faculty ratings on a defined rating scale. Background Art

[0002] When evaluating faculty ratings assigned to students, a common approach is to use skewness. Skewness, a statistical measure of the symmetry of data, can provide valuable insights into faculty rating data. A skewness value of zero indicates a perfectly symmetrical distribution, while a negative skewness value indicates a left-skewed distribution, and a positive skewness value indicates a right-skewed distribution. In other words, skewness is used as a measure to determine whether the distribution of faculty ratings is skewed or asymmetric. Skewness analysis can help identify potential biases in the distribution of faculty ratings (e.g., leniency or strictness in student ratings).

[0003] Existing methods for calculating skewness do not fully account for data measured on defined rating scales commonly used in faculty rating systems, which can lead to misleading or erroneous data. This limitation hinders accurate assessment of the skewness associated with rating scales, limiting the ability to effectively detect and address bias. Furthermore, existing methods only consider the skewness of individual faculty ratings, hindering the ability to make informed decisions based on relative performance and ability levels. Summary of the Invention

[0004] The subject matter of the present application has been developed in response to the current state of the art, and in particular, in response to the problems and needs that have been created or not yet fully addressed by typical skewness assessments. Generally speaking, the subject matter of the present application has been developed to provide a method of assessing skewness of faculty ratings that overcomes at least some of the above-mentioned shortcomings of the prior art.

[0005] Disclosed herein is a device for evaluating faculty ratings. The device includes a processor. The device also includes a memory storing code executable by the processor to receive a raw faculty rating dataset comprising multiple ratings from an individual faculty member. Each of the multiple ratings corresponds to a numerical value on a defined rating scale. The memory also stores code executable by the processor to determine rating skewness for the multiple ratings by subtracting a second central tendency measure for the multiple ratings from a first central tendency measure for the defined rating scale to generate a result, and dividing the result by the standard deviation of the multiple ratings. The memory also stores code executable by the processor to compare the skewness for the multiple ratings with at least one comparative rating skewness to generate a skewness comparison report. The memory also stores code executable by the processor to determine whether the individual faculty member requires faculty training based on the skewness comparison report. If it is determined that the individual faculty member requires faculty training, the memory also stores code executable by the processor to identify the required faculty training to be provided to the individual faculty member. Additionally, the memory stores code executable by the processor to enable one or more end users to access the rating skewness, the skewness comparison report, and the required faculty training. The foregoing subject matter of this paragraph characterizes Example 1 of the present disclosure.

[0006] The first central tendency measure is the average of the defined rating scale.The foregoing subject matter of this paragraph characterizes Example 2 of the present disclosure, wherein Example 2 also includes subject matter according to Example 1 above.

[0007] Alternatively, the first central tendency measure is the mode of the defined rating scale.The foregoing subject matter of this paragraph characterizes Example 3 of the present disclosure, wherein Example 3 also includes subject matter according to any of the above Examples 1-2.

[0008] The second central tendency measure is the mode of the plurality of ratings from the individual instructors.The foregoing subject matter of this paragraph characterizes Example 4 of the present disclosure, where Example 4 also includes subject matter according to any of Examples 1-3 above.

[0009] The second central tendency measure is the median of multiple ratings from individual instructors.The foregoing subject matter of this paragraph characterizes Example 5 of the present disclosure, wherein Example 5 also includes subject matter according to any of Examples 1-4 above.

[0010] The first central tendency measure is a predetermined reference value.The foregoing subject matter of this paragraph characterizes Example 6 of the present disclosure, wherein Example 6 also includes subject matter according to any one of the above Examples 1-5.

[0011] The plurality of ratings corresponds to one evaluation cycle. The foregoing subject matter of this paragraph characterizes Example 7 of the present disclosure, wherein Example 7 further includes subject matter according to any one of Examples 1-6 above.

[0012] The required instructor training is determined based on the magnitude of the rating skewness in the skewness comparison report.The foregoing subject matter of this paragraph characterizes Example 8 of the present disclosure, wherein Example 8 further includes subject matter according to any of the above Examples 1-7.

[0013] The defined rating scale is a numerical scale from 1 to 5. The foregoing subject matter of this paragraph characterizes Example 9 of the present disclosure, where Example 9 also includes subject matter according to any of the above Examples 1-8.

[0014] The individual instructors are flight instructors. The original instructor rating dataset includes performance ratings provided by flight instructors after flight training courses. The foregoing subject matter of this paragraph characterizes Example 10 of the present disclosure, wherein Example 10 also includes subject matter according to any of Examples 1-9 above.

[0015] The required instructor training identified for flight instructors includes aviation-specific teaching methods and flight simulation techniques.The foregoing subject matter of this paragraph characterizes Example 11 of the present disclosure, wherein Example 11 also includes subject matter according to Example 10 above.

[0016] The memory stores code executable by the processor to receive additional data sources. One or more end users can access the additional data sources. The foregoing subject matter of this paragraph represents Example 12 of the present disclosure, wherein Example 12 also includes subject matter according to any of the above Examples 1-11.

[0017] The skewness comparison report includes a visual representation of the rating skewness of the original instructor rating dataset.The foregoing subject matter of this paragraph characterizes Example 13 of the present disclosure, where Example 13 also includes subject matter according to any of the above Examples 1-12.

[0018] The memory stores code executable by the processor to receive a raw instructor rating dataset corresponding to each of a plurality of individual instructors, generate a skewness comparison report for each of the plurality of individual instructors, and analyze the generated skewness comparison report to compare the performance of each individual instructor with the performance of other instructors in the plurality of individual instructors. The foregoing subject matter of this paragraph characterizes Example 14 of the present disclosure, wherein Example 14 also includes subject matter according to any of Examples 1-13 above.

[0019] Receiving a raw instructor rating dataset includes receiving a raw instructor rating dataset comprising a plurality of ratings corresponding to a plurality of evaluation cycles. The memory stores code executable by the processor to track changes in rating skewness of the plurality of ratings from each of the plurality of evaluation cycles. The memory also stores code executable by the processor to adjust required instructor training based on observed changes in rating skewness over time. The foregoing subject matter of this paragraph characterizes Example 15 of the present disclosure, wherein Example 15 further includes subject matter according to any of Examples 1-14 above.

[0020] The memory stores code executable by the processor to analyze rating skewness of a plurality of ratings from each of a plurality of evaluation periods to identify trends in rating skewness across the plurality of evaluation periods. The memory also stores code executable by the processor to generate training recommendations based on the identified trends. The foregoing subject matter of this paragraph characterizes Example 16 of the present disclosure, wherein Example 16 also includes subject matter according to Example 15 above.

[0021] This document also discloses a method for evaluating faculty ratings. The method includes receiving an original faculty rating dataset. The original faculty rating dataset includes multiple ratings from a separate faculty. Each of the multiple ratings corresponds to a numerical value on a defined rating scale. The method also includes determining a rating skewness for the multiple ratings by subtracting a second central tendency measure for the multiple ratings from a first central tendency measure for the defined rating scale to generate a result, and dividing the result by a standard deviation of the multiple ratings. The method also includes comparing the rating skewness for the multiple ratings with at least one comparative rating skewness to generate a skewness comparison report. The method also includes determining whether the separate faculty requires faculty training based on the skewness comparison report. If it is determined that the separate faculty requires faculty training, the method also includes identifying the required faculty training to be provided to the separate faculty. The method additionally includes enabling one or more end users to access the rating skewness, the skewness comparison report, and the required faculty training. The foregoing subject matter of this paragraph represents Example 17 of the present disclosure.

[0022] The first central tendency measure is the average of the defined rating scale.The foregoing subject matter of this paragraph characterizes Example 18 of the present disclosure, wherein Example 18 also includes subject matter according to Example 17 above.

[0023] The second central tendency measure is the mode of the multiple ratings from the individual instructors.The foregoing subject matter of this paragraph characterizes Example 19 of the present disclosure, where Example 19 also includes subject matter according to any of the above Examples 17-18.

[0024] Also disclosed herein is a program product for evaluating faculty ratings, comprising a non-transitory computer-readable storage medium storing code. The code is configured to be executed by a processor to perform operations including receiving a raw faculty rating dataset. The raw faculty rating dataset includes multiple ratings from an individual faculty member. Each of the multiple ratings corresponds to a numerical value on a defined rating scale. The code is further configured to be executed by the processor to determine rating skewness for the multiple ratings by subtracting a second central tendency measure for the multiple ratings from a first central tendency measure for the defined rating scale to generate a result, and dividing the result by the standard deviation of the multiple ratings. The code is further configured to be executed by the processor to compare the rating skewness for the multiple ratings with at least one comparative rating skewness to generate a skewness comparison report. The code is additionally configured to be executed by the processor to determine whether the individual faculty member requires faculty training based on the skewness comparison report. If it is determined that the individual faculty member requires faculty training, the code is further configured to be executed by the processor to identify the required faculty training to be provided to the individual faculty member. The code is further configured to be executable by the processor to enable one or more end users to access the rating skewness, skewness comparison report, and required instructor training.The foregoing subject matter of this paragraph characterizes Example 20 of the present disclosure.

[0025] The described features, structures, advantages and / or characteristics of the subject matter of the present disclosure can be combined in one or more examples (including embodiments and / or implementations) in any suitable manner. In the following description, many specific details are provided to give a thorough understanding of the example of the subject matter of the present disclosure. Those skilled in the relevant art will recognize that the subject matter of the present disclosure can be put into practice when there is no one or more specific features, details, parts, materials and / or methods of a particular example, embodiment or implementation. In other cases, it will be recognized that additional features and advantages in certain examples, embodiments and / or implementations may not be present in all examples, embodiments or implementations. In addition, in some cases, known structures, materials or operations are not shown or described in detail to avoid blurring the aspects of the subject matter of the present disclosure. The features and advantages of the subject matter of the present disclosure will become more fully apparent from the following description and the appended claims, or can be learned by the practice of the subject matter as set forth below. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order that the advantages of the present subject matter may be more readily understood, a more particular description of the subject matter briefly described above will be presented by reference to specific examples illustrated in the accompanying drawings. It will be understood that these drawings depict only typical examples of the subject matter and that they are not to be considered limiting of its scope. The subject matter will be described and illustrated with additional specificity and detail through the use of the accompanying drawings, in which:

[0027] Figure 1 is a schematic block diagram illustrating one embodiment of a system for assessing skewness of faculty ratings;

[0028] Figure 2 is a schematic block diagram of an apparatus for evaluating skewness of instructor ratings according to one or more examples of the present disclosure;

[0029] Figure 3 is a schematic block diagram of another apparatus for evaluating skewness of instructor ratings according to one or more examples of the present disclosure;

[0030] Figure 4 is a schematic block diagram of another apparatus for evaluating skewness of instructor ratings according to one or more examples of the present disclosure;

[0031] Figure 5 is a schematic block diagram of a method for assessing skewness of faculty ratings according to one or more examples of the present disclosure; and

[0032] Figure 6 is a diagram of an example of a data availability module. DETAILED DESCRIPTION

[0033] Throughout this specification, reference to "one example," "an example," or similar language means that a particular feature, structure, or characteristic described in connection with the example is included in at least one example of the subject matter of the present disclosure. Appearances of the phrases "in one example," "in an example," and similar language throughout this specification may, but do not necessarily, all refer to the same example. Similarly, use of the term "implementation" means an implementation having a particular feature, structure, or characteristic described in connection with one or more examples of the subject matter of the present disclosure, however, an implementation may be associated with one or more examples without an explicit correlation to indicate otherwise.

[0034] Furthermore, the described features, advantages, and characteristics of the embodiments may be combined in any suitable manner. Those skilled in the relevant art will recognize that an embodiment may be practiced without one or more of the specific features or advantages of a particular embodiment. In other cases, additional features and advantages may be recognized in some embodiments that may not be present in all embodiments.

[0035] These features and advantages of the embodiments will become more fully apparent from the following description and the appended claims, or may be learned through the practice of the embodiments as set forth below. As will be appreciated by those skilled in the art, aspects of the present invention may be embodied as systems, methods, and / or computer program products. Thus, aspects of the present invention may take the form of entirely hardware embodiments, entirely software embodiments (including firmware, resident software, microcode, etc.), or embodiments combining software and hardware aspects (all of which may generally be referred to herein as "circuits," "modules," or "systems"). Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer-readable media having program code embodied thereon.

[0036] This article discloses examples of devices and methods for evaluating the skewness of teacher ratings. The following provides some features of at least some examples of teacher rating assessment devices, methods, systems and program products. Individual teachers usually need to award student ratings (e.g., grades) after completing a course or training. Typically, a rating scale is used to give a rating to a student, wherein each value on the defined rating scale is a numerical value. As used herein, a defined rating scale refers to a rating system in which ratings are given using integers or scores within the specified rating scale range. For example, a defined rating scale can have a rating range from 1 to 5, wherein each of the values ​​1, 2, 3, 4 or 5 can be awarded to a student. In other examples, scores can also be included, for example, 1.5, 2.5, 3.5 or 4.5 can be additionally awarded to a student. In some examples, the defined rating scale is based on a Likert scale, which is a psychometric rating scale in which a numerical value corresponds to a teacher's preference for a specific statement or topic. For example, each response option on a Likert scale is assigned a numerical value that can range from 1 to 5, where 1 indicates the lowest or most unfavorable response and 5 indicates the highest or more favorable response. The response options provided to faculty members typically represent varying degrees of agreement or disagreement, satisfaction, frequency, or intensity. Thus, the Likert scale allows for quantification of the degree to which faculty members agree with, are satisfied with, or measure an attribute with students. In other examples, the defined rating scale is based on a pass / fail model, where grades at or above a certain value are considered passing grades and grades below a certain value are considered failing grades.

[0037] In the field of pilot and aviation training, instructor evaluations of students (e.g., pilots) are of great significance. Ensuring that pilots receive comprehensive and effective training is crucial to their future performance. Instructors play an important role in evaluating and assigning ratings to pilots based on their performance during tests or courses. However, inherent biases or other factors may hinder the direct comparison of ratings from individual instructors with a standardized baseline or ratings from other instructors. Furthermore, it is often challenging to compare ratings from two different instructors without a systematic approach. To address these limitations, statistical measures of rating skewness can be used to evaluate instructor ratings. By evaluating the distribution of ratings provided by an instructor and comparing them to a comparative rating skewness (e.g., baseline skewness), potential biases and behavioral patterns can be identified across individual instructors. This information not only helps reveal potential instructor tendencies (e.g., leniency or strictness), but can also be used to inform additional training needs for individual instructors.

[0038] Figure 1 1 is a schematic block diagram illustrating an example of a general information system 100 for evaluating the skewness of instructor ratings and identifying required instructor training when necessary. The system 100 includes one or more information processing devices 102, one or more data management devices 104, and a data network 106. Figure 1 A specific number of information processing devices 102 , data management apparatuses 104 , and data networks 106 are depicted in FIG. 1 , but one skilled in the art will recognize, based on this disclosure, that any number may be included in system 100 .

[0039] In one example, system 100 includes one or more information processing devices 102. Information processing device 102 may be embodied as one or more of a desktop computer, a laptop computer, a tablet computer, a smartphone, a smartwatch, a smart TV, or another computing device including a processor (e.g., a central processing unit ("CPU"), a processor core, a field programmable gate array ("FPGA") or other programmable logic, an application specific integrated circuit ("ASIC"), a controller, a microcontroller, and / or another semiconductor integrated circuit device), a volatile component and / or a non-volatile storage medium, a display, a connection to a display, etc. Information processing device 102 is part of an instructor monitoring system that captures, monitors, accesses, stores, and provides test data (e.g., raw scores, ratings), student information, instructor information, and the like.

[0040] Generally speaking, in one example, the data management device 104 is configured to receive a raw instructor rating data set associated with an individual instructor, determine the rating skewness of the raw instructor data set, compare the rating skewness of the raw instructor data set with at least one comparative rating skewness, and generate a skewness comparison report, determine whether the individual instructor requires instructor training, and if so, identify the required instructor training, and make all data available to an end user. In one embodiment, the data management device 104 is part of an instructor monitoring system that can be located at a student testing site or at an off-site location. Figure 2-Figure 4 The data management device 104 is described in more detail.

[0041] In one example, the data management device 104 may include a hardware device, such as a security hardware dongle or other hardware appliance device (e.g., a set-top box, a network appliance, etc.), which is connected to the data management device 104 via a wired connection (e.g., a universal serial bus ("USB") connection) or a wireless connection (e.g., a Bluetooth connection). The data management device 104 may be connected to a device (e.g., a laptop computer, server, tablet computer, smartphone, network router or switch, etc.) using Wi-Fi, near field communication ("NFC"), etc.); connected to an electronic display device (e.g., a television or monitor using an HDMI port, a DisplayPort port, a Mini DisplayPort port, a VGA port, a DVI port, etc.); etc. The hardware appliances of the data management device 104 may include a power interface, a wired and / or wireless network interface, a graphics interface for connecting to a display, and / or a semiconductor integrated circuit device as described below, which is configured to perform the functions described herein with respect to the data management device 104.

[0042] In such an embodiment, the data management device 104 may include a semiconductor integrated circuit device (e.g., one or more chips, dies, or other discrete logic hardware), such as a field programmable gate array (“FPGA”) or other programmable logic, firmware for the FPGA or other programmable logic, microcode for execution on a microcontroller, an application specific integrated circuit (“ASIC”), a processor, a processor core, etc. In one embodiment, the data management device 104 may be mounted on a printed circuit board having one or more wires or connections (e.g., to volatile memory, non-volatile storage media, network interfaces, peripheral devices, graphics / display interfaces, etc.). The hardware device may include one or more pins, pads, or other electrical connections configured to send and receive data (e.g., in communication with one or more wires of a printed circuit board, etc.), as well as one or more hardware circuits and / or other circuits configured to perform various functions of the data management device 104.

[0043] In some embodiments, the semiconductor integrated circuit device or other hardware appliance of the data management apparatus 104 includes one or more volatile storage media and / or is communicatively coupled to one or more volatile storage media, which may include but is not limited to random access memory ("RAM"), dynamic RAM ("DRAM"), cache, etc. In one example, the semiconductor integrated circuit device or other hardware appliance of the data management apparatus 104 includes one or more non-volatile storage media and / or is communicatively coupled to one or more non-volatile storage media, which may include but is not limited to: NAND flash memory, NOR flash memory, nano random access memory (nanoRAM or "NRAM"), nanocrystal wire-based memory, silicon oxide-based sub-10 nm process memory, graphene memory, silicon oxide nitride oxysilicon ("SONOS"), resistive RAM ("RRAM"), programmable metallization cell ("PMC"), conductive bridging RAM ("CBRAM"), magnetoresistive RAM ("MRAM"), dynamic RAM ("DRAM"), phase change RAM ("PRAM" or "PCM"), magnetic storage media (e.g., hard disk, tape), optical storage media, etc.

[0044] In one embodiment, the data network 106 comprises a digital communication network that transmits digital communications. The data network 106 may include a wireless network such as a wireless cellular network, a wireless network such as a Wi-Fi network, The data network 106 may include a local wireless network such as a wireless network, a near field communication ("NFC") network, an ad hoc network, etc. The data network 106 may include a wide area network ("WAN"), a storage area network ("SAN"), a local area network ("LAN") (e.g., a home network), a fiber optic network, the Internet, or other digital communication networks. The data network 106 may include two or more networks. The data network 106 may include one or more servers, routers, switches, and / or other network devices. The data network 106 may also include one or more computer-readable storage media (e.g., a hard drive, an optical drive, non-volatile memory, RAM, etc.).

[0045] The wireless connection may be a mobile phone network. The wireless connection may also employ a Wi-Fi network based on any one of the Institute of Electrical and Electronics Engineers ("IEEE") 802.11 standards. Alternatively, the wireless connection may be In addition, wireless connections may utilize radio frequency identification (“RFID”) communications, which include those of the International Organization for Standardization (“ISO”), the International Electrotechnical Commission (“IEC”), the American Institute of Materials and DASH7 TM Alliance and EPCGlobal TM Established RFID standards.

[0046] The wireless connection may be an infrared connection, which includes at least The wireless connection may be a connection that complies with the Infrared Physical Layer Specification ("IrPHY") defined herein. Alternatively, the wireless connection may be a cellular telephone network communication. All standards and / or connection types include the latest versions and revisions of the standards and / or connection types as of the date of filing this application.

[0047] refer to Figure 2 , shows an example of an apparatus 200 for evaluating instructor ratings. The apparatus 200 includes the data management apparatus 104. The data management apparatus 104 may include embodiments of a data receiving module 202, a skewness determination module 204, a comparison module 206, a training determination module 208, a training identification module 210, and a data availability module 212, which are described in more detail below.

[0048] In one example, the data receiving module 202 is configured to receive a raw faculty rating dataset for an individual faculty member. The raw faculty rating dataset includes multiple ratings, each of which corresponds to a numerical value on a defined rating scale. Each of the multiple ratings corresponds to an evaluation period, during which a cohesive rating grouping from multiple students is created. As used herein, an evaluation period refers to a defined time frame or specific duration during which an individual faculty member assigns ratings to assess the performance, effectiveness, or progress of the student or entity being evaluated. In other words, an evaluation period serves as a unique unit of assessment. The raw faculty rating dataset can contain multiple such rating groupings, each corresponding to a different evaluation period, and each rating grouping can be individually assessed for skewness. The raw faculty rating dataset can contain one faculty rating for each student, such that the rating reflects the value / grade for the evaluation period (e.g., a single test or overall rating for a course). Alternatively, the raw faculty rating dataset can contain more than one faculty rating for each student, such that an individual student has multiple ratings corresponding to different evaluation periods. In some examples, multiple ratings for each individual student can be combined into a single representative rating, effectively creating a new single evaluation period. This combination process can involve calculating an average of the multiple ratings to produce a single composite rating that includes the student's evaluation. In other examples, the device 200 can evaluate each of the different evaluation periods separately.

[0049] An individual instructor may be an instructor who provides instruction to a variety of student types. In some examples, an individual instructor may serve as a flight instructor. Thus, a flight instructor may provide instruction to a range of individuals, including pilot trainees, novice pilots, experienced pilots seeking further training and education, and the like. Furthermore, a flight instructor's teaching environment may include a variety of settings (e.g., flight schools, aviation academies, airline training programs, aircraft manufacturer training programs, board certification programs, and the like). A raw instructor rating dataset may contain flight instructor ratings for a flight training course (e.g., a single test or task), a multi-task evaluation, or an overall rating for the course.

[0050] The data receiving module 202 may receive raw instructor rating data sets from individual instructors, administrators, supervisors of individual instructors, end users, etc. The data receiving module 202 may receive or access real-time rating data, pre-captured data, locally stored data, etc.

[0051] The skewness determination module 204 determines the rating skewness of the multiple ratings of an individual faculty member based on the evaluation period. If the original faculty rating dataset includes more than one evaluation period, each evaluation period will be analyzed separately. The rating skewness is calculated by subtracting the second central tendency measure of the multiple ratings from the first central tendency measure of the defined rating scale and dividing the result by the standard deviation of the multiple ratings. The skewness equation can be summarized as: Skewness = (CTM (rs) -CTM (pr) ) / SD (pr) , where CTM is a central tendency measure, rs is a defined rating scale, pr is a number of ratings, and SD is the standard deviation. As used herein, a central tendency measure is a statistical measure that describes or represents the center or typical value of a data set as a single value that summarizes the distribution of data points. Typically, the central tendency measure will be represented by the mean, median, or mode of the data set. The skewness of a data set can be calculated using any of the central tendency measures. However, depending on the data set, certain central tendency measures may better represent the skewness. For example, a first central tendency measure may be the mean of a defined rating scale, while a second central tendency measure may be the mode or median of a number of ratings. In other examples, the first central tendency measure may be the mode or median of a defined rating scale, while the second central tendency measure may be the median or mode of a number of ratings. In yet other examples, the first central tendency measure may be a predetermined reference value within a defined rating scale (e.g., when a skewness value other than the calculated mean, median, or mode of a defined rating scale is preferred or required).

[0052] The rating skewness algorithm differs from the algorithm traditionally used to calculate skewness. That is, the first central tendency measure has been altered from the traditional algorithm to account for the defined rating scale. Thus, the first central tendency measure has shifted from using a central tendency measure for multiple ratings to using a central tendency measure for the defined rating scale. Since skewness is a measure that indicates how far apart the central tendency measure of a distribution (e.g., the mean, median, or mode) is in terms of standard deviation, shifting the first central tendency measure relative to the rating scale will not affect the standard deviation and will maintain the second central tendency measure. In other words, by shifting the first central tendency measure to the mean of the defined rating scale (or other central tendency measure), the degree of asymmetry in the rating distribution as a function of the defined rating scale can be determined. For example, when the defined rating scale is from 1 to 5, and when the first central tendency measure is the mean of the defined rating scale, the first central tendency measure is 3. Assuming a rating of 3 is neutral, values ​​greater than 3 are considered high, while values ​​less than 3 are considered low. In some examples, it can be assumed that the value of the second tendency measure can determine the leniency or strictness of the ratings given by individual instructors. That is, a second central tendency measure greater than 3 may be considered tolerant, while a second central tendency measure less than 3 may be considered strict.

[0053] Apparatus 200 also includes a comparison module 206. Comparison module 206 compares the skewness of the raw faculty rating dataset with at least one comparative rating skewness and generates a skewness comparison report. In some examples, the at least one comparative rating skewness may be a baseline skewness. The baseline skewness serves as a reference point and may represent the desired skewness value or optimal skewness that faculty ratings should ideally exhibit. By comparing the skewness of the raw faculty rating dataset with the baseline skewness, apparatus 200 can identify any deviations or differences from the expected distribution pattern of the multiple ratings. In other examples, the at least one comparative rating skewness may be the skewness of raw faculty rating datasets from other individual faculty. In this way, raw faculty rating data from multiple individual faculty can be compared, highlighting variations in their rating distributions. By examining how the skewness of the raw faculty rating dataset compares to the skewness of other faculty, valuable insights into the leniency or strictness of faculty ratings can be gained. The skewness comparison report highlights the differences between the raw faculty rating dataset and the at least one comparative rating skewness. The skewness comparison report may include a numerical representation of the rating skewness of the raw faculty rating dataset. For example, the skewness comparison report can include the results from the skewness determination module 204, which are numerically compared to at least one comparative rating skewness. In other examples, the skewness comparison report can include a visual representation of the rating skewness of the original faculty rating dataset. For example, a chart or graph can be used to display the skewness comparison report to an end user. In some examples, the skewness comparison report can be graphically represented on a kurtosis plot to highlight the kurtosis value and tolerance value for each dataset, thereby enhancing the visualization of the comparative analysis.

[0054] The training determination module 208 of the apparatus 200 is configured to determine whether an individual instructor requires instructor training based on the skewness comparison report. By analyzing the skewness comparison report, the training determination module 208 can evaluate the distribution characteristics of the instructor rating dataset. This evaluation helps identify any deviations or irregularities in the ratings, which may necessitate additional training and instructional support. The training determination module 208 may have a defined training threshold. The defined training threshold serves as a benchmark, allowing the determination module 208 to distinguish between subtle fluctuations and significant changes in ratings. If the deviation observed in the skewness of an individual instructor's ratings exceeds the defined training threshold, the training determination module 208 may automatically determine that training is required. In some examples, in response to determining that training is required, the training determination module 208 digitally sends a message to the individual instructor or otherwise alerts the individual instructor of the required training, thereby notifying the individual instructor of the required training. Furthermore, in some examples, the training determination module 208 provides the individual instructor with relevant information regarding the required training along with the message or reminder (e.g., schedule information). Additionally, or alternatively, in response to determining that training is required, the training determination module 208 can send a message to a stakeholder (e.g., a supervisor, manager, or the organization to which the individual instructor belongs) or otherwise alert the stakeholder (e.g., a supervisor, manager, or the organization to which the individual instructor belongs) that the individual instructor requires training, enabling the stakeholder to initiate any necessary follow-up actions in response to the training request. The follow-up actions prompted by the training determination module 208 can involve coordinating training logistics, allocating resources, or facilitating access to training materials, among other things.

[0055] In some examples, if the training determination module 208 determines that an individual instructor requires instructor training, the training identification module 210 is configured to automatically identify the required instructor training (e.g., from a plurality of pre-determined instructor training programs) to provide to the individual instructor. The training identification module 210 can automatically generate a training plan for the individual instructor based on the training need or area of ​​improvement. In some examples, the training identification module 210 can identify required general instructor training. For example, general instructor training can address a wide range of teaching techniques and methods. Additionally, general instructor training can ensure that basic teaching skills are strengthened and aligned with the overall goal of improving teaching competence. In other examples, the training identification module 201 can identify a specific type or nature of training that should be provided to the individual instructor. In other words, the identified required instructor training can be tailored to address the specific areas of improvement or challenges highlighted in the report. This ensures that the training provided to the individual instructor is highly relevant and directly addresses the observed deficiencies in their teaching performance. In some examples, the required instructor training can be determined based on the magnitude of the rating skewness in the skewness comparison report. For example, the duration of the required instructor training may be increased or decreased based on the magnitude of the rating skewness. These adjustments may involve lengthening or shortening the required instructor training, or incorporating additional modules to target specific areas of improvement. The instructor training identified by the training identification module 210 may be included in messages or reminders provided to instructors and / or stakeholders by the training determination module 208.

[0056] In some cases, the training identification module 210 may use other relevant factors (e.g., duration of instruction, trainee feedback surveys, etc.) in addition to the information collected from the skewness comparison report to determine the most appropriate training method. If the individual instructor is a flight instructor, the required instructor training identified for the flight instructor may include aviation-specific teaching methods and flight simulation techniques. In some examples, the required instructor training may include integrating flight simulation exercises into the training plan to improve practical skills in flight instruction.

[0057] After the training identification module 210 identifies the required instructor training, arrange that independent instructor carries out the required instructor training. In some examples, can send a message or otherwise remind that the required instructor training of independent instructor has been arranged. Therefore, training instructor or training plan provides required instructor training for independent instructor. Required instructor training can be provided with online format (comprising the option of such as on-site interactive course, computer guidance module or pre-recorded training materials) by training instructor or training plan. Alternatively, required instructor training can be face-to-face training (in-person training) carried out together with training instructor. The selection of training format may depend on the unique needs and the target associated with the required training required and identified by the training identification module 210. Access rights (for example, digital security certificate) to training materials, modules or other resources can be provided to independent instructor. These resources can be used for independent instructor during required instructor training, and in some examples, after completing required training, independent instructor can use these resources. The availability of resource port training can support the continuous skill development of independent instructor.

[0058] In response to determining that training is required by the training determination module 208, the required identified instructor training is provided to the individual instructor, and the individual instructor receives the required identified instructor training. In some examples, the individual instructor is monitored during the required instructor training provided to the instructor to track the progress of the training program in real time. By monitoring progress, the required instructor training can be dynamically adjusted as needed to ensure the effectiveness of the required instructor training and the progress of the individual instructor. After completing the required instructor training, in some examples, the individual instructor may be issued a certificate or training badge to indicate successful completion of the required instructor training. Additionally, after completing the required instructor training, the individual instructor may be surveyed to collect feedback on the effectiveness of the training. A post-training survey may be conducted to gather insights into the effectiveness of the training program and to ensure that the required instructor training is addressing the needs of the individual instructor receiving the required training or future instructors. Additionally, the port training survey may inquire about the impact of the training on the effectiveness of the teaching and whether any further support or resources are required or needed.

[0059] Data availability module 212 is configured to at least make rating skewness, skewness comparison reports, and required instructor training accessible to one or more end users, terminal devices, other systems, and the like. For example, data availability module 212 may make the aforementioned data accessible via an application programming interface (API), a file sharing site, a remotely accessible data store, and the like. In some examples, the end user is a manager who oversees individual instructors. The end user can utilize the data available through data availability module 212 to analyze the performance and training needs of the individual instructors under their supervision. By reviewing rating skewness, skewness comparison reports, and required instructor training, the end user gains valuable insights into areas where training methods may require improvement or adjustment. This enables the end user to make informed decisions regarding the professional development and performance management of their instructors. In specific industries, such as aviation, training leaders within training and safety organizations may benefit from access to data availability module 212. By reviewing pilot and instructor competency rating comparisons, training leaders can identify areas where training methods and practices need improvement to enhance overall aviation safety. Accessing relevant data helps monitor trends, identify training gaps, and implement targeted strategies for improvement.

[0060] In some examples, the data availability module 212 can be configured to receive additional data sources applicable to individual instructors. For example, the additional data sources may include trainee feedback surveys, historical skewness comparison reports, records of completed training, etc. Thus, the additional data sources can be made accessible to one or more end users, thereby providing a comprehensive and informed approach to instructor evaluation and training decision making.

[0061] refer to Figure 3 , another embodiment of an apparatus 300 for evaluating instructor ratings is shown. The apparatus 300 includes the data management apparatus 104. The data management apparatus 104 may include embodiments of the data receiving module 202, the skewness determination module 204, the comparison module 206, the training determination module 208, the training identification module 210, and the data availability module 212, as described above and in more detail below. In addition, the data management apparatus 104 may include an instance of the performance comparison module 302, which is described in more detail below.

[0062] The data receiving module 202 can be configured to receive raw instructor rating datasets corresponding to a plurality of individual instructors. For example, the data receiving module 202 can receive raw instructor rating datasets corresponding to a first individual instructor 304, a second individual instructor 306, a third individual instructor 308, and a fourth individual instructor 310. Although a specific number of individual instructors is shown, those skilled in the art will recognize, based on this disclosure, that any number of individual instructors may be included in the apparatus 300. The raw instructor rating dataset for each of the plurality of individual instructors (304, 306, 308, 310) includes multiple ratings corresponding to at least one evaluation period. In some examples, the raw instructor rating dataset for each of the plurality of individual instructors may correspond to more than one evaluation period. That is, more than one instructor rating for each student may be included in the raw instructor rating dataset and may be merged or analyzed separately.

[0063] If the original faculty rating dataset includes more than one evaluation period, the skewness determination module 204 will determine the rating skewness of the multiple ratings of each of the multiple individual faculty members (304, 306, 308, 310) for each evaluation period. That is, the rating skewness will be calculated by subtracting the second central tendency measure of the multiple ratings of one of the multiple individual faculty members from the first central tendency measure of the defined rating scale, and dividing the result by the standard deviation of the multiple ratings of the one of the multiple individual faculty members. Because the same first central tendency measure of the defined rating scale is applied to each of the multiple individual faculty members, the rating skewness is standardized when calculating the rating skewness for the same evaluation period, thereby facilitating relative performance comparisons at a later stage.

[0064] The comparison module 206 compares the rating skewness of each of the plurality of individual instructors (304, 306, 308, 310) with at least one comparative rating skewness and generates a skewness comparison report for each of the individual instructors (304, 306, 308, 310). In some examples, the at least one comparative rating skewness is a baseline skewness, such that each rating skewness is compared to the same baseline skewness. In other examples, the at least one comparative rating skewness is the rating skewness of another of the individual instructors (304, 306, 308, 310). For example, the first individual instructor 304 may be compared with the second individual instructor 306 and / or the third individual instructor 308 and / or the fourth individual instructor 310. In some cases, each of the skewness comparison reports may include the same information when comparing the rating skewness of the individual instructor with the rating skewness of the other of the plurality of individual instructors (304, 306, 308, 310). The skewness comparison report may include the skewness of the ratings for each of the plurality of individual trainers (304, 306, 308, 310) (in a numerical representation or a visual representation, such as a kurtosis plot). The training determination module 208 and the training identification module 210 are configured to determine and identify the training for each of the plurality of individual trainers (304, 306, 308, 310) based on the corresponding skewness comparison report.

[0065] The data availability module 212 is configured to provide one or more end users with access to at least the rating skewness, skewness comparison reports, and required instructor training for each of a plurality of individual instructors (304, 306, 308, 310). The data availability module 212 may include a drill-down capability. The drill-down capability may categorize the data into specific students, locations, or other identifying categories. For example, the drill-down capability may be used to identify which airline the student rated by the individual instructor was from. Additionally, the drill-down capability may be used to identify the specific student testing site at which the student rating was conducted. Thus, the drill-down capability allows the data availability module 212 to organize the data into specific categories. In other words, the drill-down capability allows for further comparisons to be made between specific categories (e.g., specific student testing sites, student organizations, etc.).

[0066] The performance comparison module 302 is configured to analyze the generated skewness comparison report to compare the performance of each of the plurality of individual instructors (304, 306, 308, 310) with the performance of the other individual instructors (304, 306, 308, 310). Comparing the individual instructors with their peers can explain the individual instructors' strengths, areas for improvement, and unique teaching methods. In some cases, this can enable end users to use the data to foster a culture of continuous improvement and customized instructor training, ultimately improving the quality of education and training provided by the individual instructors.

[0067] refer to Figure 4 , another embodiment of an apparatus 400 for evaluating instructor ratings is shown. The apparatus 400 includes the data management apparatus 104. The data management apparatus 104 may include embodiments of the data receiving module 202, the skewness determination module 204, the comparison module 206, the training determination module 208, the training identification module 210, and the data availability module 212, as described above and in more detail below. In addition, the data management apparatus 104 may include instances of the skewness tracking module 402 and the training adjustment module 404.

[0068] The data receiving module 202 can be configured to receive raw instructor rating datasets corresponding to a plurality of evaluation periods (406, 408, 410, 412). For example, the data receiving module 202 can receive raw instructor rating datasets corresponding to a first evaluation period 406, a second evaluation period 408, a third evaluation period 410, and a fourth evaluation period 412. Although a specific number of evaluation periods are shown, those skilled in the art will recognize, based on this disclosure, that any number of evaluation periods can be included in the apparatus 400. The raw instructor rating dataset is specific to a single individual instructor and includes data for a plurality of evaluation periods (406, 408, 401, 412), wherein each of the plurality of evaluation periods (406, 408, 401, 412) includes a plurality of ratings. In some examples, the material taught and assessed during the evaluation periods (406, 408, 410, 412) is the same. For example, the material taught and assessed during each evaluation period (406, 408, 410, 412) is the same (e.g., the same task or course taught at different points in time). In other examples, the material taught and assessed during the evaluation period (406, 408, 401, 412) may include different materials taught and assessed at different points in time.

[0069] Skewness tracking module 402 is configured to track changes in rating skewness for a plurality of ratings from each of a plurality of evaluation cycles. That is, skewness tracking module 402 analyzes changes in rating distributions over time. Changes in rating skewness may indicate potential changes in teaching effectiveness or changes in assessment quality over time. Skewness tracking module 402 enables the identification of emerging trends, areas for improvement, or noteworthy developments in instructor rating data. The ability to detect temporal shifts facilitates a comprehensive and up-to-date understanding of instructor performance, enabling data-driven decision making and informed adjustments to training programs or teaching methods.

[0070] The training adjustment module 404 is configured to adjust required instructor training based on observed changes in rating skewness over time. By tracking changes in rating skewness over time, the training adjustment module 404 allows for dynamic adjustments to individual instructor training plans, leveraging data to make informed, data-driven decisions about the content, focus, and delivery of required instructor training. The training adjustment module 404 can refine required instructor training to address specific areas of improvement or emerging challenges. This adaptive approach ensures that the training provided to individual instructors remains relevant, impactful, and aligned with their evolving needs.

[0071] The data availability module 212 may also be configured to analyze the rating skewness from each of the multiple evaluation cycles to identify trends in the rating skewness across the multiple evaluation cycles. In other examples, according to the apparatus 300, trends in the rating skewness of multiple individual instructors within a single evaluation cycle may be identified. Trends may include recurring patterns or fluctuations. Training recommendations may be generated based on the identified trends.

[0072] refer to Figure 5 , a method 500 for evaluating faculty ratings is shown. The method includes (block 502) receiving a raw faculty rating dataset. The raw faculty rating dataset includes a plurality of ratings from an individual faculty member, each of the plurality of ratings corresponding to a numerical value on a defined rating scale. The plurality of ratings corresponds to an evaluation review. In some examples, the raw faculty rating dataset may include more than one plurality of ratings, each of the plurality of ratings corresponding to one of a plurality of evaluation periods. Additionally, or alternatively, in other examples, a raw faculty rating dataset may be provided for each of a plurality of individual faculty members.

[0073] Method 500 also includes (block 504) determining the rating skewness for the plurality of ratings. The rating skewness is determined by subtracting a second central tendency measure for the plurality of ratings from a first central tendency measure for the defined rating scale to generate a result, and dividing the result by the standard deviation of the plurality of ratings. If the original faculty rating data set includes more than one evaluation period, each evaluation period can be analyzed separately. Any central tendency measure can be used to represent the data set. For example, the first central tendency measure can be the mean of the defined rating scale, while the second central tendency measure can be the mode or median of the plurality of ratings. In other examples, the first central tendency measure can be the mode or median of the defined rating scale, while the second central tendency measure can be the median or mode of the plurality of ratings. In still other examples, the first central tendency measure can be a predetermined reference value within the defined rating scale, such as when a skewness value other than the calculated mean, median, or mode of the defined rating scale is preferred or required. However, depending on the data set, certain central tendency measures may better represent skewness.

[0074] Method 500 also includes comparing the rating skewness of the plurality of ratings with at least one comparative rating skewness to generate a skewness comparison report. In some examples, the at least one comparative rating skewness is a baseline skewness. In other examples, the at least one comparative rating skewness may be the skewness of a raw faculty rating dataset from another individual faculty member. In this manner, raw faculty rating data from multiple individual faculty members can be compared to highlight variations in their rating distributions. The skewness comparison report may present the results, which may be displayed numerically or visually.

[0075] The method 500 also includes determining (block 508) whether the individual instructor requires instructor training based on the skewness comparison report. The method 500 also includes identifying (block 510) required instructor training to be provided to the individual instructor if it is determined that the individual instructor requires instructor training. In the case where the individual instructor is a flight instructor, the required instructor training identified for the flight instructor may include aviation-specific teaching methods and flight simulation techniques.

[0076] The method additionally includes providing one or more end users with access to the rating skewness, skewness comparison reports, and required instructor training (block 512). The end users can review and analyze the available data to assess the performance and training needs of individual instructors. This can allow the end users to make informed decisions regarding the professional development and performance management of an individual instructor or multiple individual instructors.

[0077] refer to Figure 6 , shows an example of data available in the data availability module 212. The data availability module 212 is configured to make data available to one or more end users. Figure 6 As shown, at least some of the data is shown in the data availability module 212. Other data not shown may be available by running additional reports, highlighting or clicking on a specific instructor, etc. For example, Figure 6 The data shown in highlights the similarities in grading styles between multiple individual instructors. However, those skilled in the art will recognize in light of this disclosure that the data can be displayed and organized in any manner as desired or determined by one or more end users.

[0078] The data availability module 212 shown includes a list of individual instructors 602. The individual instructors are from multiple student testing sites (such as Site A, Site B, and Site C). Therefore, the data can be further organized into individual instructors from only one student testing site (such as a campus), a subset of testing sites (such as a regional or statewide testing site), or an entire group of individual instructors (such as the world). The data availability module 212 also includes comparative data 604, which in some cases can be displayed as a visual representation of the skewness of the ratings of the individual instructors. As shown, the comparative data 604 is displayed as a graph that plots the skewness rating of each of the individual instructors. The data availability module 212 also includes identifying the individual instructors who need instructor training. For example, the data availability module 212 can display a list 606 of individual instructors who need additional training.

[0079] Many of the functional units described in this specification have been labeled as modules to more particularly emphasize their implementation independence. For example, a module can be implemented as a hardware circuit comprising custom very large scale integration ("VLSI") circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module can also be implemented in a programmable hardware device (e.g., a field programmable gate array ("FPGA"), programmable array logic, a programmable logic device, etc.).

[0080] Modules can also be implemented in software for execution by various types of processors. For example, an identified program code module can include one or more physical or logical blocks of computer instructions, which can be organized, for example, as objects, procedures, or functions. However, the executable files of the identified modules need not be physically located together, but can include different instructions stored in different locations that, when logically connected together, comprise the module and achieve the module's explicit purpose.

[0081] In fact, the module of program code can be a single instruction, or can be many instructions, and can even be distributed on several different code fragments, distributed between different programs and distributed across several memory devices.Similarly, operational data can be identified and illustrated in this article in the module, and can be embodied in any suitable form, and organized in the data structure of any suitable type.Operational data can be collected as a single data set, or can be distributed in different locations (including being distributed on different storage devices), and can exist at least in part only as the electronic signal on a system or network.When a module or the part of a module is realized in software, program code can be stored and / or propagated in one or more computer-readable media.

[0082] The computer program product may include a computer-readable storage medium (or multiple media) having computer-readable program instructions thereon for causing a processor to implement aspects of the present invention.

[0083] Computer-readable storage media can be a tangible device that can retain and store instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the above devices. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer disks, hard disks, random access memories ("RAM"), read-only memories ("ROM"), erasable programmable read-only memories ("EPROM" or flash memory), static random access memories ("SRAM"), portable compact disk read-only memories ("CD-ROM"), digital versatile disks ("DVD"), memory sticks, floppy disks, mechanical encoding devices (such as a convex structure in a punched card or a groove, on which instructions are recorded) and any suitable combination of the above. As used herein, computer-readable storage media itself should not be interpreted as temporary signals (such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagated by waveguides or other transmission media (for example, light pulses transmitted by fiber optic cables) or electrical signals transmitted by wires).

[0084] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network can include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in a computer-readable storage medium within the corresponding computing / processing device.

[0085] The computer-readable program instructions for implementing the operation of the present invention can be assembly instructions, instruction set architecture ("ISA") instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and traditional procedural programming languages ​​(such as "C" programming language or similar programming languages). The computer-readable program instructions can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer or completely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network (these networks include local area networks ("LAN") or wide area networks ("WAN")), or can be connected to an external computer (for example, by using the Internet of an Internet service provider). In some embodiments, an electronic circuit comprising, for example, a programmable logic circuit, a field programmable gate array ("FPGA") or a programmable logic array ("PLA") can execute the computer-readable program instructions by utilizing the state information of the computer-readable program instructions to personalize the electronic circuit so as to perform aspects of the present invention.

[0086] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0087] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, executed via the processor of the computer or other programmable data processing apparatus, create a device for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can direct a computer, programmable data processing apparatus, and / or other device to operate in a particular manner, such that the computer-readable storage medium having the instructions stored therein constitutes an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0088] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, such that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0089] The schematic flow charts and / or schematic block diagrams in the accompanying drawings illustrate the architecture, functions, and operations of possible implementations of the apparatus, system, method, and computer program product according to various embodiments of the present invention. In this regard, each block in the schematic flow charts and / or schematic block diagrams may represent a module, segment, or portion of code, which includes one or more executable instructions of a program code for implementing a specified logical function.

[0090] It should also be noted that in some alternative implementations, the functions described in the blocks may not occur in the order described in the figures. For example, depending on the functions involved, two blocks shown in succession may actually be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order. Other steps and methods are contemplated that are equivalent in function, logic, or effect to one or more blocks, or portions thereof, of the figures shown.

[0091] Although various arrow types and line types may be employed in the flowcharts and / or block diagrams, they are understood not to limit the scope of the corresponding embodiments. In fact, some arrows or other connectors may be used to indicate only the logical flow of the depicted embodiments. For example, arrows may indicate wait or monitoring periods of unspecified duration between enumerated steps of the depicted embodiments. It will also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system or a combination of dedicated hardware and program code that performs the specified functions or actions.

[0092] As used herein, a list with the conjunction "and / or" includes any single item in the list or a combination of items in the list. For example, a list of A, B, and / or C includes only A, only B, only C, a combination of A and B, a combination of B and C, a combination of A and C, or a combination of A, B, and C. As used herein, a list using the term "one or more of..." includes any single item in the list or a combination of items in the list. For example, one or more of A, B, and C includes only A, only B, only C, a combination of A and B, a combination of B and C, a combination of A and C, or a combination of A, B, and C. As used herein, a list using the term "one of..." includes the only one of any single item in the list. For example, "one of A, B, and C" includes only A, only B, or only C, and does not include a combination of A, B, and C. As used herein, “a member selected from the group consisting of A, B and C” includes only one of A, B or C and does not include the combination of A, B and C. As used herein, “a member selected from the group consisting of A, B and C and the combination thereof” includes only A, only B, only C, the combination of A and B, the combination of B and C, the combination of A and C, or the combination of A, B and C.

[0093] In the above description, certain terms may be used, such as "upward", "downward", "upper", "lower", "horizontal", "vertical", "left", "right", "above", "below", etc. Where applicable, these terms are used to provide some clarity when dealing with relative relationships. However, these terms are not intended to imply absolute relationships, positions and / or orientations. For example, with respect to an object, the "upper" surface can become the "lower" surface simply by turning the object over. However, it is still the same object. In addition, unless expressly provided otherwise, the terms "including", "comprising", "having" and their variations mean "including but not limited to". Unless expressly provided otherwise, an enumerated list of items does not imply that any or all items are mutually exclusive and / or mutually inclusive. Unless expressly provided otherwise, the terms "a", "an" and "the" also mean "one or more". In addition, the term "plurality" may be defined as "at least two"

[0094] As used herein, when used with a list of items, the phrase “at least one of…” means that different combinations of one or more of the listed items may be used, and only one item from the list may be required. The item may be a specific object, thing, or category. In other words, “at least one of…” means that any combination of items or number of items from the list may be used, but not all items in the list may be required. For example, “at least one of Item A, Item B, and Item C” may mean Item A; Item A and Item B; Item B; Item A, Item B, and Item C; or Item B and Item C. In some cases, “at least one of Item A, Item B, and Item C” may mean, for example, but not limited to, two of Item A, one of Item B, and ten of Item C; four of Item B and seven of Item C; or some other suitable combination.

[0095] Unless otherwise indicated, the terms "first," "second," etc., are used herein merely as labels and are not intended to impose any order, position, or hierarchy requirements on the items to which these terms refer. Furthermore, reference to an item, for example, "second," does not require or preclude the existence of an item, for example, "first," or lower-numbered, and / or an item, for example, "third," or higher-numbered.

[0096] As used herein, a system, device, structure, article, element, component, or hardware that is “configured to” perform a specified function is actually capable of performing the specified function without any modification, and not merely has the potential to perform the specified function after further modification. In other words, a system, device, structure, article, element, component, or hardware that is “configured to” perform a specified function is specifically selected, created, implemented, utilized, programmed, and / or designed for the purpose of performing the specified function. As used herein, “configured to” means existing characteristics of a system, device, structure, article, element, component, or hardware that enable the system, device, structure, article, element, component, or hardware to perform the specified function without further modification. For the purposes of this disclosure, a system, device, structure, article, element, component, or hardware described as “configured to” perform a particular function may additionally or alternatively be described as “adapted to” and / or “operable” to perform that function.

[0097] The schematic flow charts included herein are generally described as logical flow charts. Therefore, the order and labeled steps depicted indicate an example of the method presented. Other steps and methods that are equivalent in function, logic, or effect to one or more steps of the method shown or parts thereof can be envisioned. Additionally, the format and symbols used are provided to explain the logical steps of the method and are understood not to limit the scope of the method. Although various arrow types and line types can be used in the flow charts, they are understood not to limit the scope of the corresponding method. In fact, some arrows or other connectors can be used to indicate only the logical flow of the method. For example, an arrow can indicate a waiting or monitoring period of unspecified duration between the enumerated steps of the depicted method. Additionally, the order in which a particular method occurs may or may not strictly comply with the order of the corresponding steps shown.

[0098] The present subject matter may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects as illustrative only and not restrictive. All changes that come within the meaning and range of equivalence of the examples herein are intended to be included within their scope.

Claims

1. An apparatus (200) for evaluating teacher ratings, comprising: processor; and A memory storing code executable by the processor to: receiving (202) a raw instructor rating data set, the raw instructor rating data set comprising a plurality of ratings from individual instructors, each of the plurality of ratings corresponding to a numerical value on a defined rating scale; determining (204) a rating skewness of the plurality of ratings by: subtracting a second central tendency measure of the plurality of ratings from a first central tendency measure of the defined rating scale to generate a result, and dividing the result by a standard deviation of the plurality of ratings; comparing the skewness of the plurality of ratings to at least one comparative rating skewness (206) to generate a skewness comparison report; determining (208) whether the individual instructor requires instructor training based on the skewness comparison report; If it is determined (210) that the individual instructor requires instructor training, identifying the required instructor training to be provided to the individual instructor; as well as One or more end users are enabled (212) to access the rating skewness, the skewness comparison report, and the required instructor training.

2. The device (200) according to claim 1, wherein: The first central tendency measure is the mean of the defined rating scale.

3. The device (200) according to claim 1, wherein: The first central tendency measure is the mode of the defined rating scale.

4. The device (200) according to claim 1, wherein: The second central tendency measure is the mode of the plurality of ratings from the individual instructors.

5. The device (200) according to claim 1, wherein: The second central tendency measure is a median of the plurality of ratings from the individual instructors.

6. The device (200) according to claim 1, wherein: The first central tendency measure is a predetermined reference value.

7. The device (200) according to claim 1, wherein: The plurality of ratings corresponds to an evaluation period.

8. The device (200) according to claim 1, wherein: The required instructor training is determined based on the magnitude of the rating skewness in the skewness comparison report.

9. The device (200) according to claim 1, wherein: The defined rating scale is a numerical scale from 1 to 5.

10. The device (200) according to claim 1, wherein: The single instructor is a flight instructor; and The raw instructor rating data set includes performance ratings provided by the flight instructor after a flight training course.

11. The device (200) according to claim 10, wherein: The required instructor training identified for the flight instructor includes aviation-specific teaching methods and flight simulation techniques.

12. The device (200) according to claim 1, wherein: The memory further stores code executable by the processor to receive an additional data source; and The additional data sources may be accessible to one or more end users.

13. The device (200) according to claim 1, wherein: The skewness comparison report includes a visual representation of the rating skewness of the original instructor rating data set.

14. The apparatus (200) of claim 1, wherein the memory further stores code executable by the processor to: receiving a raw instructor rating data set corresponding to each of a plurality of individual instructors (304-310); generating a skewness comparison report for each of the plurality of individual instructors; and The generated skewness comparison report is analyzed (302) to compare the performance of each individual instructor with the performance of other instructors in the plurality of individual instructors.

15. The device (200) according to claim 1, wherein: Receiving a raw instructor ratings dataset includes receiving a raw instructor ratings dataset including a plurality of ratings, the plurality of ratings corresponding to a plurality of evaluation periods; and The memory also stores code executable by the processor to: tracking (402) changes in the rating skewness of the plurality of ratings from each of the plurality of evaluation cycles; and The required instructor training is adjusted (404) based on observed changes in the rating skewness over time.

16. The apparatus (200) of claim 15, wherein the memory further stores code executable by the processor to: analyzing the rating skewness of the plurality of ratings from each of the plurality of evaluation periods to identify a trend of the rating skewness within the plurality of evaluation periods; and Generate training recommendations based on the identified tendencies.

17. A method (500) for evaluating faculty ratings, comprising: receiving (502) a raw instructor rating data set, the raw instructor rating data set comprising a plurality of ratings from individual instructors, each of the plurality of ratings corresponding to a numerical value on a defined rating scale; determining (504) a rating skewness for the plurality of ratings by: subtracting a second central tendency measure for the plurality of ratings from a first central tendency measure for the defined rating scale to generate a result, and dividing the result by a standard deviation for the plurality of ratings; comparing the rating skewness of the plurality of ratings to at least one comparative rating skewness (506) to generate a skewness comparison report; determining (508) whether the individual instructor requires instructor training based on the skewness comparison report; If it is determined that the individual instructor requires instructor training, identifying (510) the required instructor training to be provided to the individual instructor; as well as One or more end users are enabled (512) to access the rating skewness, the skewness comparison report, and the required instructor training.

18. The method (500) of claim 17, wherein: The first central tendency measure is the mean of the defined rating scale.

19. The method (500) of claim 17, wherein: The second central tendency measure is the mode of the plurality of ratings from the individual instructors.

20. A program product for evaluating instructor ratings, comprising a non-transitory computer-readable storage medium storing code, the code configured to be executable by a processor to perform operations comprising: receiving a raw instructor ratings data set, the raw instructor ratings data set comprising a plurality of ratings from individual instructors, each of the plurality of ratings corresponding to a numerical value on a defined rating scale; determining a rating skewness of the plurality of ratings by: subtracting a second central tendency measure of the plurality of ratings from a first central tendency measure of the defined rating scale to generate a result, and dividing the result by a standard deviation of the plurality of ratings; comparing the rating skewness of the plurality of ratings to at least one comparative rating skewness rating to generate a skewness comparison report; determining whether the individual instructor requires instructor training based on the skewness comparison report; if it is determined that the individual instructor requires instructor training, identifying the required instructor training to be provided to the individual instructor; as well as One or more end users are enabled to access the rating skewness, the skewness comparison report, and the required instructor training.

Images