Learning device, learning method and program

Abstract

A training device comprising a training unit that updates a scoring model through training, the training unit estimating, on the basis of video data of a subject being scored in which a scene within a scoring event that requires scoring is captured, an estimated probability distribution in which a relative merits score that represents the relative merits of the subject being scored is employed as a probability variable, the estimated probability distribution indicating the probability that a scorer will apply each relative merits score when the scorer scores the scene. The training unit executes: a process for executing the scoring model on video data being scored, which is the video data of the video being scored and which is included in a scoring result training set, the scoring result training set being an aggregate of scoring training data that is a pair of the video data being scored and a scorer score indicated by the scorer who viewed the scene captured in the video being scored; a process for acquiring a correct answer distribution that indicates the distribution of scorer scores on the basis of the scoring result training set and the scorer score that is paired with the video data being scored, which is the subject of processing by the scoring model; and a process for updating the scoring model so that a difference between the estimated probability distribution obtained through the scoring model execution process and the correct answer distribution is reduced.

Description

[Technical Field]

[0001] The present invention relates to a learning device, a learning method, and a program. [Background technology]

[0002] There are events that require grading, such as judged competitions and grading tests that evaluate the surgical skills of medical students, where the results depend on the judge. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Yansong Tang, et al.,”Uncertainty-aware Score Distribution Learning for Action Quality Assessment” In CVPR. 2020. Summary of the Invention [Problem to be solved by the invention]

[0004] In such events, the scoring results may differ depending on the judge. As a result, for example, the ranking of the person being scored may differ depending on the judge. Naturally, if this occurs, it may increase dissatisfaction among the participants of the event, such as the person being scored and the spectators.

[0005] For example, if a performance is performed at the Olympics that many athletes and spectators believe to be first place, and the judges present a score that shows the performance to be third place, it is easy to imagine that many athletes and spectators would be dissatisfied. Therefore, there is a demand for technology that can alleviate this dissatisfaction among participants that arises from dependency on the judges.

[0006] In view of the above circumstances, an object of the present invention is to provide a technique for reducing dissatisfaction among participants in events that require scoring. [Means for solving the problem]

[0007] One aspect of the present invention includes a learning unit that updates, through learning, a scoring model that estimates a probability distribution in which merit scores representing the merits or demerits of the target video in the scoring event are random variables, based on video data of the target video, which is video showing a scene during a scoring event that requires scoring, and the estimated probability distribution is a probability distribution that indicates the probability that a grader will assign each merit score when scoring the scene. The learning unit updates the scoring model by learning, based on video data of the target video, which is video showing a scene during a scoring event that requires scoring, and the estimated probability distribution is a probability distribution in which merit scores representing the merits or demerits of the target video in the scoring event are random variables, the probability distribution indicating the probability that a grader will assign each merit score when scoring the scene. a scoring model execution process that executes the scoring model on video data to be scored that is included in a scoring result learning set, which is a collection of scoring learning data that is a pair of grader scores and grader scores; a correct distribution acquisition process that acquires a correct distribution that indicates a probability distribution of grader scores based on the scoring result learning set and the grader scores that are pairs of the video data to be scored that is the target of processing by the scoring model; and an update process that updates the scoring model so as to reduce the difference between the estimated probability distribution obtained by the scoring model execution process and the correct distribution.

[0008] One aspect of the present invention includes a learning step of updating, by learning, a scoring model that estimates an estimated probability distribution, based on video data of a video to be scored, which is a video showing a scene during a scoring event that requires scoring, the estimated probability distribution being a probability distribution in which merit scores representing the merits or demerits of the subject to be scored in the scoring event are random variables, the probability distribution indicating the probability that a grader will assign each merit score when scoring the scene, and the learning step includes updating, by learning, a scoring model that estimates an estimated probability distribution based on video data of the ... a grader will assign each merit score when scoring the scene, and the estimated probability distribution This is a learning method that executes a scoring model execution process that executes the scoring model on video data to be scored that is included in a scoring result learning set, which is a collection of scoring learning data that is a pair of a certain scorer score and a certain scorer score; a correct answer distribution acquisition process that acquires a correct answer distribution that shows the distribution of grader scores based on the scoring result learning set and the grader scores that are pairs with the video data to be scored that is the target of processing by the scoring model; and an update process that updates the scoring model so as to reduce the difference between the estimated probability distribution obtained by the scoring model execution process and the correct answer distribution.

[0009] One aspect of the present invention is a program for causing a computer to function as the learning device described above. [Effects of the Invention]

[0010] The present invention makes it possible to reduce frustration among participants in events that require scoring. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is an explanatory diagram illustrating an overview of a scoring system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a learning device according to an embodiment. [Figure 3] FIG. 2 is a diagram showing an example of the configuration of a control unit included in the learning device according to the embodiment. [Figure 4] 10 is a flowchart showing an example of a flow of processing executed by a learning device according to an embodiment. [Figure 5]FIG. 2 is a diagram illustrating an example of a hardware configuration of an estimation apparatus according to an embodiment. [Figure 6] FIG. 2 is a diagram showing an example of the configuration of a control unit included in the estimation device according to the embodiment. [Figure 7] 1 is a flowchart showing an example of a flow of processing executed by an estimation device according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] (Embodiment) FIG. 1 is an explanatory diagram outlining a scoring system 100 according to an embodiment. Hereinafter, an event requiring scoring, such as scoring competitions or grading tests to evaluate the surgical skills of medical students, will be referred to as a scoring event. Hereinafter, a score indicating the superiority or inferiority of an object being scored in a scoring event will be referred to as a superiority or inferiority score. An object being scored is an object that is being scored. An object being scored does not necessarily have to be a person, but may be an animal or an inorganic object such as ceramics. Hereinafter, the period during a scoring event will be referred to as the period during a scoring event.

[0013] An example of a judged event is a judged competition such as figure skating at the Olympics. If the judged event is figure skating, the merit score is a score such as a technical score or a performance composition score given to the athlete's performance.

[0014] The scoring system 100 includes a learning device 1 and an estimation device 2. The learning device 1 includes a learning unit 10. The learning unit 10 updates the scoring model through learning. The scoring model is a mathematical model that processes video data (hereinafter referred to as "video data to be scored") of a video that shows a scene during a scoring event (hereinafter referred to as "video to be scored").

[0015] The scoring model estimates an estimated probability distribution for scenes appearing in the scoring target video represented by the scoring target video data to be processed, based on the scoring target video data to be processed. Hereinafter, the scenes appearing in the scoring target video represented by the scoring target video data will be referred to as scoring target scenes.

[0016] The estimated probability distribution is a probability distribution in which the merit scores are random variables, and indicates the probability that a grader will assign each merit score when scoring a scene. The distribution shown by image G1 in Figure 1 is an example of an estimated probability distribution.

[0017] Note that updating a mathematical model through learning means updating the processing content of the mathematical model using a machine learning method.

[0018] The learning unit 10 updates the scoring model until a predetermined condition for terminating learning (hereinafter referred to as the "learning termination condition") is satisfied. The learning termination condition may be, for example, a condition that the change in the mathematical model due to the update is smaller than a predetermined change, or the learning termination condition may be, for example, a condition that a predetermined number of updates have been performed. The scoring model at the time the learning termination condition is satisfied (hereinafter referred to as the "trained scoring model") is used by the estimation device 2.

[0019] The learning unit 10 acquires a scoring result learning set. The scoring result learning set is a collection of multiple scoring learning data. The scoring learning data is a pair of video data to be scored and a grader score. The grader score is a merit / inferiority score given by a grader who has viewed the scene to be scored. In other words, the grader score included in the scoring result learning set is the result of scoring by an actual grader. The video data to be scored included in the scoring learning data is, for example, video data obtained in a past scoring event. Therefore, the scoring learning data is, for example, information obtained in a past scoring event.

[0020] In FIG. 1, image G2 is a diagram showing the relationship between the scoring result learning set, the scoring learning data, the video data to be scored, and the grader scores.

[0021] <Distribution of scoring results> By the way, in a grading event, it is rare for all the people being graded to have the same number of points. For example, if the scores for each person in the order of graded person A, graded person B, graded person C, graded person D, graded person E, graded person F, etc. are 10 points, 40 points, 60 points, 70 points, 70 points, 90 points, etc., there is often a wide range of points, and it is rare for all the people being graded to have a score of 50 points.

[0022] A person to be graded is a person who is graded, for example, in a graded sport, a person to be graded is an athlete in the graded sport. A person to be graded is an example of an object to be graded.

[0023] Since variations in the merit scores occur frequently, the distribution of the scoring results often also shows variations. The distribution of the scoring results is a frequency distribution of the scorers' scores, with the population being a set of scoring learning data that meets certain conditions for the selection of the scorers' scores (hereinafter referred to as "adoption conditions").

[0024] Therefore, the distribution of the scoring results indicates the number of scoring learning data pieces that represent the respective grader scores and are included in the population. More specifically, the grader scores that are paired with the video data to be scored that are the target of processing by the scoring model are the grader scores included in the scoring learning data pieces that include the video data to be scored that are the target of processing by the scoring model.

[0025] An example of an adoption condition is that the difference between the indicated grader score and the paired grader score of the video data to be scored that is the target of processing by the scoring model is within the Nth smallest difference (N is an integer greater than or equal to 1). Therefore, in this case, the grader score indicated by the scoring result distribution satisfies the condition that the difference between the indicated grader score and the paired grader score of the video data to be scored that is the target of processing by the scoring model is within the Nth smallest difference (N is an integer greater than or equal to 1). The frequency distribution shown by image G3 in Figure 1 is an example of a scoring result distribution.

[0026] Below, the learning device 1 will be explained using an example in which the distribution of scoring results has variance, but the distribution of scoring results does not necessarily have to have variance. A distribution with no variance is a distribution with a variance of 0.

[0027] In addition, obtaining the scoring result learning set may be a process of reading out a scoring result learning set that has been stored in advance in a predetermined storage device, or may be a process of obtaining a scoring result learning set that has been input by a user or the like.

[0028] <More detailed explanation of the learning unit 10> The learning unit 10 executes a scoring model execution process. The scoring model execution process is a process for executing a scoring model. The learning unit 10 executes a correct answer distribution acquisition process. The correct answer distribution acquisition process is a process for acquiring a correct answer distribution based on a scoring result learning set and the grader scores that are paired with the video data to be scored that is the processing target of the scoring model. The correct answer distribution is a probability distribution of the grader scores.

[0029] More specifically, the correct distribution is a distribution indicated by a probability function obtained by fitting the scoring result distribution with a predetermined parameterized probability function (hereinafter referred to as a "parametrized function") that satisfies characteristic conditions. To obtain the correct distribution means to obtain parameter values ​​that satisfy the condition that the difference between the scoring result distribution and the parameterized function is smaller than a predetermined difference.

[0030] The characteristic conditions include a condition that the function has one peak and a condition that the function has parameters that include a variance and a mean. An example of a parameterized function is a generalized normal distribution. The distribution shown in image G4 in Figure 1 is an example of a correct distribution.

[0031] The correct distribution acquisition process includes, for example, an adoption process, an adoption result distribution acquisition process, and a fitting process. The adoption process is a process of determining scored learning data that satisfies the scoring conditions based on the grader scores of pairs of video data to be scored that are the subject of processing by the scoring model. The adoption result distribution acquisition process is a process of obtaining an adoption result distribution based on a set of scored learning data that is determined to satisfy the scoring conditions by the adoption process. The fitting process is a process of obtaining a correct distribution based on the adoption result distribution.

[0032] In the fitting process, for example, maximum likelihood estimation may be performed. In addition, in the fitting process, for example, the kurtosis of the distribution of the adopted results may be used as the value of the shape parameter in the generalized normal distribution.

[0033] The learning unit 10 executes a determination process, which is a process for determining whether or not a learning termination condition is satisfied.

[0034] The learning unit 10 executes an update process. The scoring model is updated so as to reduce the difference between the estimated probability distribution estimated by the scoring model and the correct distribution obtained by the correct distribution acquisition process. In other words, the scoring model is updated by learning using the correct distribution as correct data so as to reduce the difference between the result of estimation of the scoring model and the correct data.

[0035] The difference between the estimated probability distribution estimated by the scoring model and the correct distribution obtained by the correct distribution acquisition process may be, for example, the KL divergence (Kullback-Leibler divergence) between the estimated probability distribution estimated by the scoring model and the correct distribution obtained by the correct distribution acquisition process.

[0036] The update process is executed, for example, when it is determined that the learning end condition is not satisfied as a result of the determination process. That is, the update process is executed when the learning end condition is not satisfied.

[0037] <Other conditions that may be included in the characteristic conditions> Generally, in scoring events, there is fierce competition among those who are given high-frequency scores, and even a small difference in points can significantly affect the ranking. Therefore, it is desirable for the peak of the correct answer distribution to be sharper near the scores with high probability. A sharper peak means that the variance in the scores is smaller. The scoring model is updated through an update process to estimate a distribution closer to the correct answer distribution.

[0038] Therefore, the sharper the peak of the correct answer distribution, the more the scoring model estimates a distribution with smaller variance at the peak. The smaller the variance at the peak, the more accurately the scoring model can rank the people being graded. Therefore, the sharper the peak of the correct answer distribution, the more accurately the scoring model can estimate a scoring result.

[0039] Therefore, the characteristic conditions may further include a parameter setting condition. The parameter setting condition includes a condition that, when the sparse-dense condition is satisfied, the variance and shape parameter values ​​of the parametrized function are smaller than a first predetermined value. Furthermore, the parameter setting condition includes a condition that, when the sparse-dense condition is not satisfied, the variance and shape parameter values ​​of the parametrized function are larger than a second predetermined value.

[0040] The first predetermined value is equal to or less than the second predetermined value. The sparse / dense condition is a predetermined condition related to the frequency of the grader scores paired with the video data to be scored that is the target of processing by the scoring model, and is a condition that the frequency in the scoring result distribution of the grader scores paired with the video data to be scored that is the target of processing by the scoring model is higher than a predetermined frequency.

[0041] By including parameter setting conditions in the characteristic conditions, the learning device 1 can obtain a trained scoring model that performs estimation with even higher accuracy.

[0042] <Hardware Description> 2 is a diagram showing an example of the hardware configuration of a learning device 1 according to an embodiment. The learning device 1 includes a control unit 11 having a processor 91, such as a CPU (Central Processing Unit), and a memory 92 connected via a bus, and executes a program. By executing the program, the learning device 1 functions as a device including the control unit 11, an input unit 12, a communication unit 13, a memory unit 14, and an output unit 15.

[0043] More specifically, the processor 91 reads out a program stored in the storage unit 14 and stores the read out program in the memory 92. When the processor 91 executes the program stored in the memory 92, the learning device 1 functions as a device including a control unit 11, an input unit 12, a communication unit 13, a storage unit 14, and an output unit 15.

[0044] The control unit 11 controls the operation of various functional units included in the learning device 1. The control unit 11 controls, for example, the operation of the output unit 15. The control unit 11 records, in the storage unit 14, various pieces of information generated by learning, for example.

[0045] Input unit 12 includes input devices such as a mouse, keyboard, and touch panel. Input unit 12 may be configured as an interface that connects these input devices to learning device 1. Input unit 12 accepts input of various types of information to learning device 1.

[0046] The communication unit 13 includes a communication interface for connecting the learning device 1 to an external device. The communication unit 13 communicates with the external device via wired or wireless communication. The external device is, for example, a device that transmits a scoring result learning set. The communication unit 13 acquires the scoring result learning set by communicating with the device that transmits the scoring result learning set.

[0047] The external device is, for example, the estimation device 2. The communication unit 13 transmits the trained scoring model to the estimation device 2 through communication with the estimation device 2. Note that transmitting the mathematical model means transmitting a computer program that causes a computer to execute the mathematical model. Note that the scoring result set does not necessarily have to be input via the communication unit 13, and may be input to the input unit 12.

[0048] The memory unit 14 is configured using a computer-readable storage medium device (non-transitory computer-readable recording medium) such as a magnetic hard disk device or a semiconductor storage device. The memory unit 14 stores various information related to the learning device 1. The memory unit 14 stores information input via the input unit 12 or the communication unit 13, for example. The memory unit 14 stores various information generated by executing learning, for example. The memory unit 14 stores, for example, a scoring model in advance. The memory unit 14 stores, for example, a learned scoring model.

[0049] The output unit 15 outputs various types of information. The output unit 15 includes a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or an organic EL (Electro-Luminescence) display. The output unit 15 may be configured as an interface that connects these display devices to the learning device 1. The output unit 15 outputs, for example, information input to the input unit 12. The output unit 15 may also display, for example, the results of learning.

[0050] 3 is a diagram showing an example of the configuration of the control unit 11 included in the learning device 1 in this embodiment. The control unit 11 includes a learning unit 10, a memory control unit 120, a communication control unit 130, and an output control unit 140. The memory control unit 120 records various information in the memory unit 14. The communication control unit 130 controls the operation of the communication unit 13. The output control unit 140 controls the operation of the output unit 15.

[0051] 4 is a flowchart showing an example of the flow of processing executed by the learning device 1 in an embodiment. The learning unit 10 acquires, from among the scoring target video data included in the scoring result learning set, scoring target video data that has not yet been processed by a scoring model (step S101). Note that acquiring the scoring target video data may be a process of reading the scoring target video data from a scoring result learning set that has been stored in advance in a predetermined storage device, or a process of acquiring the scoring target video data from a scoring result learning set input by a user or the like.

[0052] Next, the learning unit 10 executes the scoring model on the scoring target video data obtained in step S101 (step S102). By the processing of step S102, an estimated probability distribution for scenes appearing in the scoring target video indicated by the scoring target video data obtained in step S101 is obtained.

[0053] Next, the learning unit 10 executes a correct distribution acquisition process (step S103). By executing the correct distribution acquisition process, the learning unit 10 acquires a correct distribution based on the scoring result learning set and the grader scores that are paired with the video data to be scored that is the processing target of the scoring model.

[0054] Next, the learning unit 10 executes an update process (step S104). That is, the learning unit 10 updates the scoring model based on the estimated probability distribution obtained in step S102 and the correct distribution obtained in step S103 so as to reduce the difference between the estimated probability distribution and the correct distribution. Next, the learning unit 10 executes a determination process (step S105). That is, the learning unit 10 determines whether the learning termination condition is satisfied.

[0055] If the learning end condition is not satisfied (step S105: NO), the process returns to step S101. On the other hand, if the learning end condition is satisfied (step S105: YES), the process ends.

[0056] 5 is a diagram illustrating an example of a hardware configuration of the estimation device 2 according to an embodiment. The estimation device 2 includes a control unit 21 having a processor 93 such as a CPU and a memory 94 connected via a bus, and executes a program. By executing the program, the estimation device 2 functions as a device including the control unit 21, an input unit 22, a communication unit 23, a storage unit 24, and an output unit 25.

[0057] More specifically, the processor 93 reads out a program stored in the storage unit 24 and stores the read out program in the memory 93. When the processor 92 executes the program stored in the memory 93, the estimation device 2 functions as a device including a control unit 21, an input unit 22, a communication unit 23, a storage unit 24, and an output unit 25.

[0058] The control unit 21 controls the operation of various functional units included in the estimation device 2. The control unit 21 controls, for example, the operation of the output unit 25. The control unit 21 records, in the storage unit 24, various pieces of information generated by learning, for example.

[0059] The input unit 22 includes input devices such as a mouse, a keyboard, and a touch panel. The input unit 22 may be configured as an interface that connects these input devices to the estimation device 2. The input unit 22 accepts input of various types of information to the estimation device 2.

[0060] The communication unit 23 includes a communication interface for connecting the estimation device 2 to an external device. The communication unit 23 communicates with the external device via wired or wireless communication. The external device is, for example, a device that is a transmission source of an estimation target. For the estimation device 2, the estimation target is video data to be scored. The communication unit 23 acquires the estimation target by communicating with the device that is a transmission source of the estimation target.

[0061] The external device is, for example, the learning device 1. The communication unit 23 acquires the learned scoring model obtained by the learning device 1 through communication with the learning device 1. Note that transmitting a mathematical model means acquiring a computer program representing the mathematical model. Note that the video data to be scored does not necessarily have to be input via the communication unit 23, but may also be input to the input unit 22.

[0062] The storage unit 24 is configured using a computer-readable storage medium device (non-transitory computer-readable recording medium) such as a magnetic hard disk device or a semiconductor storage device. The storage unit 24 stores various information related to the estimation device 2. The storage unit 24 stores information input via, for example, the input unit 22 or the communication unit 23. The storage unit 24 stores various information generated by, for example, the execution of a trained scoring model. The storage unit 24 stores, for example, a trained scoring model.

[0063] The output unit 25 outputs various types of information. The output unit 25 includes a display device such as a CRT display, a liquid crystal display, or an organic EL display. The output unit 25 may be configured as an interface that connects these display devices to the estimation device 2. The output unit 25 outputs, for example, information input to the input unit 22. The output unit 25 may also display, for example, the results of estimation using a trained scoring model.

[0064] 6 is a diagram illustrating an example of the configuration of the control unit 21 included in the estimation device 2 according to the embodiment. The control unit 21 includes a trained scoring model executing unit 20, an object acquiring unit 210, a memory control unit 220, a communication control unit 230, and an output control unit 240.

[0065] The trained scoring model execution unit 20 executes the trained scoring model on the estimation target. The target acquisition unit 210 acquires the estimation target input to the input unit 22 or the communication unit 23. That is, the target acquisition unit 210 acquires the scoring target video data input to the input unit 22 or the communication unit 23. The memory control unit 220 records various information in the memory unit 24. The communication control unit 230 controls the operation of the communication unit 23. The output control unit 240 controls the operation of the output unit 25.

[0066] 7 is a flowchart showing an example of the flow of processing executed by estimation device 2 in an embodiment. Target acquisition unit 210 acquires video data to be scored that is to be estimated (step S201). Next, trained scoring model execution unit 20 executes the trained scoring model on the video data to be scored acquired in step S201 (step S202). By the processing of step S202, an estimated probability distribution for the scene depicted in the video represented by the video data to be scored acquired in step 201 is obtained. Next, output control unit 240 controls the operation of output unit 25 to cause output unit 25 to output the estimated probability distribution obtained in step S202 (step S203).

[0067] As described above, the learning device 1 configured in this manner updates the scoring model so as to reduce the difference between the estimated probability distribution estimated by the scoring model and the correct distribution obtained in the correct distribution acquisition process. That is, in the learning device 1, the scoring model is updated by learning using the correct distribution as correct data so as to reduce the difference between the result of estimation of the scoring model and the correct data.

[0068] The scoring result training set is the result of scoring by a grader. Furthermore, the more scoring training data included in the scoring result training set, the smaller the variance in the distribution of scoring results for the same scene. Therefore, as learning progresses through such updates, the more frequently the scoring model estimates results that are closer to the results actually given by a grader and have less variance in the results for the same scene than the actual grader.

[0069] Therefore, a learning device 1 that updates the scoring model in this way can reduce dissatisfaction among participants in events that require scoring.

[0070] Furthermore, the estimation device 2 configured in this manner performs estimation using the learned scoring model obtained by the learning device 1, thereby reducing frustration among participants in events that require scoring.

[0071] (Variation) When the parameterized function is a generalized normal distribution, the score given by the grader for the pair of video data to be scored that is the processing target of the scoring model may be used as the average value of the generalized normal distribution.

[0072] The learning device 1 may be implemented using multiple information processing devices connected to each other via a network, in which case the functional units of the learning device 1 may be distributed and implemented across the multiple information processing devices.

[0073] The estimation device 2 may be implemented using a plurality of information processing devices communicably connected via a network, in which case the respective functional units of the estimation device 2 may be distributed and implemented among the plurality of information processing devices.

[0074] All or part of the functions of the learning device 1 and the estimation device 2 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. The program may be transmitted via a telecommunications line.

[0075] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Explanation of symbols]

[0076] 100...Scoring system, 1...Learning device, 2...Estimation device, 10...Learning unit, 11...Control unit, 12...Input unit, 13...Communication unit, 14...Memory unit, 15...Output unit, 120...Memory control unit, 130...Communication control unit, 140...Output control unit, 21...Control unit, 22...Input unit, 23...Communication unit, 24...Memory unit, 25...Output unit, 20...Learned scoring model execution unit, 210...Object acquisition unit, 220...Memory control unit, 230...Communication control unit, 240...Output control unit, 91...Processor, 92...Memory, 93...Processor, 94...Memory

Claims

1. a learning unit that updates, through learning, a scoring model that estimates an estimated probability distribution, based on video data of a video to be scored, which is a video of a scene during a scoring event that requires scoring, the estimated probability distribution being a probability distribution in which merit scores representing the merits or demerits of the video to be scored in the scoring event are random variables, the probability being a probability that a grader will assign each merit score when scoring the scene; Equipped with the learning unit executes a scoring model execution process for executing the scoring model on the scoring target video data included in a scoring result learning set, which is a collection of scoring learning data, each of which is a set of scoring target video data, which is video data of a video to be scored, and a grader score, which is a score of merit or demerit given by a grader who has viewed a scene depicted in the scoring target video; a correct answer distribution acquisition process for acquiring a correct answer distribution indicating a probability distribution of the grader scores based on the scoring result learning set and the grader scores that are paired with the video data to be scored that is the processing target of the scoring model; an update process for updating the scoring model so as to reduce the difference between the estimated probability distribution obtained by the scoring model execution process and the correct distribution; Run The correct distribution is a distribution indicated by a probability function obtained by fitting a scoring result distribution, which is a frequency distribution of grader scores and has a population of a set of scoring learning data that satisfies adoption conditions, which are predetermined conditions related to the selection of grader scores, with a predetermined parameterized probability function that satisfies characteristic conditions including the condition of having one peak and the condition of including a variance and a mean as parameters, the grader score indicated by the scoring result distribution satisfies the condition that the difference between the grader score that is paired with the scoring target video data that is the processing target of the scoring model is within the Nth smallest difference (N is an integer of 1 or more); The characteristic condition further comprises: A condition regarding the frequency of a scorer score paired with video data to be scored that is the processing target of the scoring model, a sparse-dense condition, which is a condition that the frequency in the scoring result distribution of the scorers' scores that are paired with the video data to be scored that is the processing target of the scoring model is higher than a predetermined frequency, is satisfied, the value of the variance and shape parameter of the parameterized predetermined probability function is smaller than a first predetermined value; if the sparseness / denseness condition is not satisfied, the values ​​of the variance and shape parameter of the parameterized predetermined probability function are greater than a second predetermined value; Conditions including, a parameter setting condition, the first predetermined value is equal to or less than the second predetermined value; Learning device.

2. a learning step of updating, through learning, a scoring model that estimates an estimated probability distribution, based on video data of a video to be scored, which is a video showing a scene during a scoring event that requires scoring, the estimated probability distribution being a probability distribution in which merit / demerit scores representing the merit / demerit of the subject to be scored in the scoring event are used as random variables, the probability being that a scorer will assign each merit / demerit score when scoring the scene; and The learning step includes a scoring model execution process for executing the scoring model on the video data to be scored included in a scoring result learning set, which is a set of scoring learning data, each set being a pair of video data to be scored, which is video data of a video to be scored, and a grader score, which is a score of merit or demerit given by a grader who has viewed a scene depicted in the video to be scored; a correct answer distribution acquisition process for acquiring a correct answer distribution indicating a distribution of grader scores based on the scoring result learning set and the grader scores that are paired with the video data to be scored that is the processing target of the scoring model; an update process for updating the scoring model so as to reduce the difference between the estimated probability distribution obtained by the scoring model execution process and the correct distribution; Run The correct distribution is a distribution indicated by a probability function obtained by fitting a scoring result distribution, which is a frequency distribution of grader scores and has a population of a set of scoring learning data that satisfies adoption conditions, which are predetermined conditions related to the selection of grader scores, with a predetermined parameterized probability function that satisfies characteristic conditions including the condition of having one peak and the condition of including a variance and a mean as parameters, the grader score indicated by the scoring result distribution satisfies the condition that the difference between the grader score that is paired with the scoring target video data that is the processing target of the scoring model is within the Nth smallest difference (N is an integer of 1 or more); The characteristic condition further comprises: A condition regarding the frequency of a scorer score paired with video data to be scored that is the processing target of the scoring model, a sparse-dense condition, which is a condition that the frequency in the scoring result distribution of the scorers' scores that are paired with the video data to be scored that is the processing target of the scoring model is higher than a predetermined frequency, is satisfied, the value of the variance and shape parameter of the parameterized predetermined probability function is smaller than a first predetermined value; if the sparseness / denseness condition is not satisfied, the values ​​of the variance and shape parameter of the parameterized predetermined probability function are greater than a second predetermined value; Conditions including, a parameter setting condition, the first predetermined value is equal to or less than the second predetermined value; How to learn.

3. A program for causing a computer to function as the learning device according to claim 1.

Images