Information processing device, information processing method, and program

Abstract

In order to provide a technology for flexibly identifying a process indicated by a video, an information processing device (1) comprises an acquisition unit (11) that acquires a video, a calculation unit (12) that calculates the degree of certainty for whether or not the video satisfies one or a plurality of conditions set in advance, and an identification unit (13) that identifies a process indicated by the video on the basis of the degree of certainty for each condition and a score indicating the degree of contribution to identification of the process by the condition.

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] There is a demand for technology that can automatically determine which process among a set of processes is being performed from a video image of the process being performed.

[0003] Patent Document 1 describes a work management device that identifies work based on the feature amounts of the work of a worker detected by a sensor and estimates the progress of the work being performed by the worker. In the process of identifying work by the work management device described in Patent Document 1, the work is broken down into multiple elemental work tasks, and if a characteristic action is present in the extracted feature amounts, the elemental work corresponding to the characteristic work is identified, and if a characteristic action is not present in the extracted feature amounts, the work is identified as an unknown task. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-163556 Summary of the Invention [Problem to be solved by the invention]

[0005] The work management device described in Patent Document 1 identifies elemental work tasks by determining whether or not a characteristic action exists in the extracted feature values, i.e., by binary input. Therefore, the work management device described in Patent Document 1 has the problem of being unable to flexibly identify work tasks according to the feature values ​​extracted from the video.

[0006] One aspect of the present invention has been made in view of the above-mentioned problems, and one object thereof is to provide a technique for flexibly determining the process represented by a video. [Means for solving the problem]

[0007] An information processing device according to one aspect of the present invention includes an acquisition means for acquiring an image, a calculation means for calculating a degree of certainty as to whether the image satisfies one or more predetermined conditions, and a discrimination means for discriminating the process represented by the image based on the degree of certainty for each of the conditions and a score indicating the degree to which the condition contributes to the discrimination of the process.

[0008] An information processing method according to one aspect of the present invention includes an information processing device acquiring video, calculating a degree of certainty as to whether the video satisfies one or more predetermined conditions, and determining the process represented by the video based on the degree of certainty for each of the conditions and a score indicating the degree to which the condition contributes to determining the process.

[0009] A program according to one aspect of the present invention is a program that causes a computer to function as an information processing device, and the program causes the computer to function as an acquisition means that acquires video, a calculation means that calculates a degree of certainty as to whether the video satisfies one or more specified conditions, and a discrimination means that discriminates the process represented by the video based on the degree of certainty for each of the conditions and a score that indicates the degree to which the condition contributes to discriminating the process. [Effects of the Invention]

[0010] According to one aspect of the present invention, the process represented by the video can be flexibly determined. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a configuration of an information processing device according to a first exemplary embodiment of the present invention. [Figure 2] 1 is a flowchart showing the flow of an information processing method according to a first exemplary embodiment of the present invention. [Figure 3] FIG. 10 is a block diagram showing the configuration of an information processing device according to a second exemplary embodiment of the present invention. [Figure 4] FIG. 10 is a flowchart showing an example of the flow of processes executed by a receiving unit and a generating unit in the second exemplary embodiment of the present invention. [Figure 5] FIG. 10 is a diagram showing an example of one or more predetermined elements in exemplary embodiment 2 of the present invention. [Figure 6] FIG. 10 is a diagram showing another example of one or more predetermined elements in the second exemplary embodiment of the present invention. [Figure 7] FIG. 10 is a flowchart showing an example of the flow of processes executed by a receiving unit and a generating unit in the second exemplary embodiment of the present invention. [Figure 8] FIG. 10 is a diagram showing an example of a screen for accepting input of a score in the second exemplary embodiment of the present invention. [Figure 9] FIG. 10 is a diagram showing an example of a score table in the second exemplary embodiment of the present invention. [Figure 10] FIG. 10 is a flowchart showing the flow of an information processing method according to a second exemplary embodiment of the present invention. [Figure 11] FIG. 10 is a diagram showing an example of an image including a discrimination result, which is displayed in the second exemplary embodiment of the present invention. [Figure 12] FIG. 10 is a diagram showing another example of an image including a discrimination result, which is displayed in the second exemplary embodiment of the present invention. [Figure 13] FIG. 2 is a block diagram illustrating an example of a hardware configuration of an information processing device according to each exemplary embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0012] Exemplary Embodiment 1 A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described below.

[0013] (Overview of information processing device 1) The information processing device 1 according to this exemplary embodiment is a device that acquires an image and determines the process represented by the image.

[0014] The video acquired by the information processing device 1 is a video captured of at least one process in a series of processes being performed. A series of processes refers to a plurality of processes each performed in a predetermined order. Examples of a series of processes include, but are not limited to, a series of tasks at a construction site, a series of tasks on a production line in a factory, and exercises involving multiple predetermined movements.

[0015] (Configuration of information processing device 1) The configuration of an information processing device 1 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of an information processing device 1 according to this exemplary embodiment.

[0016] 1, the information processing device 1 is configured to include an acquisition unit 11, a calculation unit 12, and a determination unit 13. In this exemplary embodiment, the acquisition unit 11, the calculation unit 12, and the determination unit 13 are configured to respectively realize an acquisition means, a calculation means, and a determination means.

[0017] The acquisition unit 11 acquires an image. The acquisition unit 11 supplies the acquired image to the calculation unit 12.

[0018] The calculation unit 12 calculates a degree of certainty as to whether or not the video supplied from the acquisition unit 11 satisfies one or more predetermined conditions. As an example, the calculation unit 12 is a learning model that has been trained to estimate whether or not the video satisfies one or more predetermined conditions and calculate a degree of certainty of the estimation result. Examples of the one or more predetermined conditions include, but are not limited to, that an element is included in the video, that an element included in the video is present at a predetermined position, that an element included in the video is in a predetermined state, and that a predetermined sound is included in the video.

[0019] Although the specific configuration of the calculation unit 12 does not limit this exemplary embodiment, for example, a convolution neural network (CNN), a recurrent neural network (RNN), or a combination thereof may be used. Also, a non-neural network model such as a random forest or a support vector machine may be used.

[0020] The discrimination unit 13 discriminates the process represented by the video acquired by the acquisition unit 11 based on the confidence level for each condition and the score indicating the degree to which the condition contributes to the process discrimination, which are supplied from the calculation unit 12. The score indicating the degree to which the condition contributes to the process discrimination is an index indicating the degree of relevance between the condition and the process. For example, if the score indicates a high relevance between the condition and the process, the process is more likely to be determined to be the process when the condition is met. On the other hand, if the score indicates a high relevance between the condition and the process, the process is less likely to be determined to be the process even when the condition is met. For example, if the score for a certain process is higher than a predetermined threshold, the discrimination unit 13 discriminates that the video represents the certain process.

[0021] As described above, the information processing device 1 according to this exemplary embodiment employs a configuration including an acquisition unit 11 that acquires video, a calculation unit 12 that calculates the degree of certainty as to whether the video satisfies one or more predetermined conditions, and a discrimination unit 13 that discriminates the process represented by the video based on the degree of certainty for each condition and a score indicating the degree to which the condition contributes to the discrimination of the process.

[0022] Therefore, the information processing device 1 according to this exemplary embodiment can distinguish processes according to the degree of certainty and the score. For example, when the information processing device 1 determines whether a certain video represents process A, if the score indicates that condition A has a low degree of contribution to the determination of process A, the information processing device 1 may determine that the certain video does not represent process A even if the degree of certainty for condition A is high. On the other hand, if the score indicates that condition B has a high degree of contribution to the determination of process A, the information processing device 1 may determine that the certain video represents process A even if the degree of certainty for condition B is low. In other words, the information processing device 1 can achieve the effect of being able to flexibly distinguish processes represented by video.

[0023] (Flow of information processing method S1) The flow of the information processing method S1 according to this exemplary embodiment will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the information processing method S1 according to this exemplary embodiment.

[0024] (Step S11) In step S11, the acquisition unit 11 acquires an image. The acquisition unit 11 supplies the acquired image to the calculation unit 12.

[0025] (Step S12) In step S12, the calculation unit 12 calculates a degree of certainty as to whether or not the video supplied from the acquisition unit 11 in step S11 satisfies one or more predetermined conditions. The calculation unit 12 supplies the calculated degree of certainty to the determination unit 13.

[0026] (Step S13) In step S13, the discrimination unit 13 discriminates the process represented by the video acquired by the acquisition unit 11 in step S11 based on the confidence level for each of the conditions supplied from the calculation unit 12 in step S12 and the score indicating the degree to which the condition contributes to the discrimination of the process.

[0027] As described above, in the information processing method S1 according to this exemplary embodiment, the acquisition unit 11 acquires video in step S11, the calculation unit 12 calculates a degree of certainty as to whether the video satisfies one or more predetermined conditions in step S12, and the discrimination unit 13 discriminates the process represented by the video based on the degree of certainty for each condition and a score indicating the degree to which the condition contributes to the discrimination of the process in step S13. Therefore, the information processing method S1 according to this exemplary embodiment can achieve the same effect as the information processing device 1 described above.

[0028] Exemplary Embodiment 2 A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are given the same reference numerals, and their description will be omitted as appropriate.

[0029] (Configuration of information processing device 2) The configuration of the information processing device 2 according to this exemplary embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the information processing device 2 according to this exemplary embodiment.

[0030] As shown in FIG. 3, the information processing device 2 includes a communication unit 21, an input unit 22, an output unit 23, a display panel 24, a storage unit 25, and a control unit 26.

[0031] The communication unit 21 is a communication module that communicates with other devices via a network (not shown). For example, the communication unit 21 outputs data supplied from a control unit 26 (described later) to other devices via the network, and acquires data output from other devices via the network and supplies the data to the control unit 26.

[0032] The specific configuration of the network through which the communication unit 21 communicates with other devices does not limit this embodiment, but as an example, a wireless LAN (Local Area Network), a wired LAN, a WAN (Wide Area Network), a public line network, a mobile data communication network, or a combination of these networks can be used.

[0033] The input unit 22 is an interface for acquiring data from other connected devices. As an example, the input unit 22 acquires input information indicating video and operation inputs received from the user, and supplies the input information to the control unit 26.

[0034] The output unit 23 is an interface that outputs data to another device connected thereto. For example, when a display device is connected to the output unit 23, the output unit 23 outputs video information indicating the video supplied from the control unit 26 to the display device.

[0035] The display panel 24 is a device that displays images. For example, the display panel 24 receives image information supplied from the control unit 26 and displays an image indicated by the image information.

[0036] The storage unit 25 stores data referenced by the control unit 26. Examples of data stored in the storage unit 25 include video, input information, a score table, and a determination result indicating the determination result. The video and input information are as described above. The score table will be described later.

[0037] (Function of control unit 26) The control unit 26 controls each component of the information processing device 2. As shown in FIG. 3 , the control unit 26 also functions as an acquisition unit 11, a reception unit 261, a generation unit 262, a calculation unit 12, a determination unit 13, and a display unit 263. In this exemplary embodiment, the acquisition unit 11, the calculation unit 12, and the determination unit 13 are configured to respectively realize an acquisition means, a calculation means, and a determination means. In this exemplary embodiment, the reception unit 261 is configured to realize a first reception means, a second reception means, and a third reception means. In this exemplary embodiment, the generation unit 262 is configured to realize a first generation means, a second generation means, and a third generation means.

[0038] The acquisition unit 11 acquires data supplied from the communication unit 21 or the input unit 22. Examples of data acquired by the acquisition unit 11 include video and sound related to the video. Examples of sound related to the video include sound of a predetermined frequency emitted by an object included in the video, and audio of predetermined words uttered by a person included in the video. The acquisition unit 11 stores the acquired data in the storage unit 25.

[0039] The receiving unit 261 receives an input from a user. As an example, the receiving unit 261 receives, as an input from a user, an input indicated by input information acquired from the input unit 22. The receiving unit 261 stores the acquired input information in the storage unit 25.

[0040] An example of an input from a user that is received by the receiving unit 261 is an input related to one or more predetermined elements. The one or more predetermined elements are elements included in the video, and examples thereof include an object, an action, a position, and a color.

[0041] Another example of an input from a user that the receiving unit 261 receives is an input regarding the selection of one or more predetermined conditions. Examples of the one or more predetermined conditions are as described above. Yet another example of an input from a user that the receiving unit 261 receives is an input of a score regarding the selected condition, which indicates the degree to which the condition contributes to process discrimination.

[0042] An example of the process in which the receiving unit 261 receives these inputs from the user will be described later.

[0043] The generating unit 262 generates data in response to input from a user. As an example, the generating unit 262 references input information stored in the storage unit 25 and generates one or more predetermined conditions or a score table. Here, the score table is a table including scores for each of one or more predetermined conditions and each score for each of a plurality of processes. Examples of score tables will be described later.

[0044] Furthermore, when the input information stored in the memory unit 25 indicates an input regarding one or more predetermined elements (in other words, when the receiving unit 261 receives an input regarding one or more predetermined elements), the generation unit 262 may generate a condition including that the video contains one or more predetermined elements.

[0045] The generating unit 262 stores the generated data in the storage unit 25. An example of the process in which the generating unit 262 generates one or more predetermined conditions or score tables will be described later.

[0046] The calculation unit 12 calculates a degree of certainty as to whether or not a video satisfies one or more predetermined conditions. As an example, the calculation unit calculates a degree of certainty as to whether or not a video stored in the storage unit 25 satisfies one or more predetermined conditions stored in the storage unit 25. An example configuration of the calculation unit 12 is as described above.

[0047] The calculation unit 12 stores the calculated certainty factors in the storage unit 25. When the calculation unit 12 calculates the certainty factors for each of a plurality of conditions, the calculation unit 12 associates each calculated certainty factor with the condition used to calculate the certainty factor and stores the calculated certainty factors in the storage unit 25. The time intervals at which the calculation unit 12 calculates the certainty factors will be described later.

[0048] The discrimination unit 13 discriminates the process represented by the video based on the confidence level for each condition and a score indicating the degree to which the condition contributes to the discrimination of the process. As an example, the discrimination unit 13 refers to the confidence level and a score table stored in the storage unit 25, and calculates a score for each of one or more processes. The discrimination unit 13 then discriminates the process represented by the video based on the score. The discrimination unit 13 stores the discrimination result in the storage unit 25. An example of the process by which the discrimination unit 13 discriminates the process represented by the video will be described later.

[0049] Display unit 263 generates video information indicating the video to be displayed and supplies the video information. As an example, display unit 263 supplies the video information generated by referring to the video stored in storage unit 25 and the discrimination result to at least one of output unit 23 and display panel 24. Hereinafter, supplying the video information by display unit 263 to at least one of output unit 23 and display panel 24 is also expressed as displaying the video by display unit 263.

[0050] (Time interval at which the calculation unit 12 calculates the confidence factor) There is no particular limitation on when the calculation unit 12 calculates the certainty factor, and one example is a configuration in which the calculation is performed at a predetermined time interval. The predetermined time interval at which the calculation unit 12 calculates the certainty factor is not particularly limited, and is preferably the same as the time interval at which the discrimination unit 13 makes a discrimination, or shorter than the time interval at which the discrimination unit 13 makes a discrimination. As one example, the calculation unit 12 may calculate the certainty factor at a time interval set by a user. As another example, the calculation unit 12 may be configured to calculate the certainty factor at a predetermined interval (one frame interval, ten minute interval, three hour interval, etc.). As yet another example, the calculation unit 12 may change the time interval at which the calculation unit 12 calculates the certainty factor depending on the discrimination result by the discrimination unit 13.

[0051] As an example of a process in which the calculation unit 12 changes the time interval at which it calculates the confidence level in accordance with the determination result by the determination unit 13, the calculation unit 12 may be configured to change the time interval at which it calculates the confidence level for each process indicated by the determination result. As an example, when the determination result by the determination unit 13 is "process A", the calculation unit 12 calculates the confidence level at one-hour intervals, and when the determination result by the determination unit 13 is "process B", the calculation unit 12 calculates the confidence level at five-hour intervals.

[0052] As another example of a process in which the calculation unit 12 changes the time interval at which it calculates the certainty factor in accordance with the determination result by the determination unit 13, the calculation unit 12 may be configured to shorten the time interval at which it calculates the certainty factor for a predetermined period after a change in the process indicated by the determination result by the determination unit 13. As an example, in a configuration in which the calculation unit 12 calculates the certainty factor at five-hour intervals, if the determination result by the determination unit 13 changes from "Process A" to "Process B," the calculation unit 12 may shorten the time interval at which it calculates the certainty factor from five hours to one hour for three hours after the determination result by the determination unit 13 becomes "Process B."

[0053] As yet another example of the process of changing the time interval at which the calculation unit 12 calculates the certainty factor in accordance with the determination result by the discrimination unit 13, the calculation unit 12 may be configured to lengthen the time interval at which the calculation unit 12 calculates the certainty factor when the determination result by the discrimination unit 13 has not changed for a predetermined period of time. As an example, in a state in which the calculation unit 12 calculates the certainty factor at one-hour intervals, when the period during which the determination result by the discrimination unit 13 is "Process A" continues for one day, the calculation unit 12 may lengthen the interval at which the calculation unit 12 calculates the certainty factor from one hour to five hours. In this configuration, when the determination result by the discrimination unit 13 changes (for example, when the determination result by the discrimination unit 13 changes from "Process A" to "Process B"), the calculation unit 12 may return the time interval at which the calculation unit 12 calculates the certainty factor to one hour.

[0054] Here, the time interval at which the discrimination unit 13 performs the process of discriminating the process represented by the video may also be changed to match the time interval at which the calculation unit 12 calculates the confidence factor. In other words, the discrimination unit 13 may be configured to change the time interval at which the discrimination process is performed depending on the discrimination result by the discrimination unit 13. That is, the discrimination unit 13 may be configured to change the time interval at which the discrimination process will be performed in the future depending on the discrimination result performed in the past. With this configuration, the information processing device 2 can suitably reduce the processing load depending on the discrimination result.

[0055] (Example of processing executed by the reception unit 261 and the generation unit 262) An example of processing executed by the receiving unit 261 and the generating unit 262 will be described with reference to Fig. 4. Fig. 4 is a flow diagram showing an example of the flow of processing S2 executed by the receiving unit 261 and the generating unit 262 in this exemplary embodiment. In processing S2, the receiving unit 261 receives input related to one or more predetermined elements, and the generating unit 262 generates one or more predetermined conditions. The flow of processing S2 will be described.

[0056] (Step S21) In step S21, the receiving unit 261 receives an input of a condition name. As an example, when the input unit 22 receives an operation input from a user indicating that the condition name is “condition a”, the receiving unit 261 stores input information indicating that the condition name is “condition a” in the storage unit 25.

[0057] (Step S22) In step S22, the receiving unit 261 receives a selection of a target video that is the subject of the generation conditions. As an example, when the input unit 22 receives an operation input from the user indicating that the target video is "video A," the receiving unit 261 stores in the storage unit 25 input information indicating that the target video is "video A."

[0058] (Step S23) In step S23, the receiving unit 261 receives input related to one or more predetermined elements. An example of processing by the receiving unit 261 to receive input related to one or more predetermined elements will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of one or more predetermined elements in this exemplary embodiment.

[0059] First, the display unit 263 displays the target video selected in step S22. As an example, the display unit 263 refers to the input information stored in the storage unit 25 and acquires the target video indicated by the input information from the storage unit 25. The display unit 263 displays video information indicating the acquired target video on the display panel 24. For example, as shown in FIG. 5, the display unit 263 displays the target video including the object OBJ1 as an element in the region R1 on the display panel 24.

[0060] Next, the receiving unit 261 receives input regarding one or more predetermined elements. For example, the receiving unit 261 acquires input information indicating that the one or more predetermined elements are object OBJ1 included in area R1 in the diagram shown in FIG. 5. The receiving unit 261 stores the acquired input information in the storage unit 25.

[0061] Another example of the process in which the receiving unit 261 receives input related to one or more predetermined elements will be described with reference to Fig. 6. Fig. 6 is a diagram showing another example of one or more predetermined elements in this exemplary embodiment.

[0062] For example, the display unit 263 displays a target video including object OBJ2 as an element on the display panel 24, as shown in the upper part of Fig. 6. Next, the receiving unit 261 acquires input information indicating that one or more predetermined elements in the target video shown in the upper part of Fig. 6 are object OBJ2 and the color of object OBJ2. For example, if the color of object OBJ2 is red, the receiving unit 261 acquires input information indicating that the one or more predetermined elements are red object OBJ2.

[0063] (Step S24) In step S24, the generating unit 262 generates a condition. For example, the generating unit 262 references the input information stored in the storage unit 25 by the receiving unit 261 in step S23 to generate the condition.

[0064] As an example, if the input information referred to by the generation unit 262 indicates that a specified element or elements are object OBJ1 included in region R1 described using Figure 5, the generation unit 262 generates the object OBJ1 with the condition that the object OBJ1 is included in region R1.

[0065] As another example of a condition generated by the generation unit 262, if the input information referred to by the generation unit 262 indicates that the predetermined element or elements are the red object OBJ2 described using the upper side of Figure 6, the generation unit 262 generates an image with the condition that the red object OBJ2 is included in the target image.

[0066] The generating unit 262 stores the generated conditions in the storage unit 25.

[0067] (Step S25) In step S25, the receiving unit 261 determines whether or not input regarding other conditions has been received.

[0068] If it is determined in step S25 that an input relating to another condition has been received (step S25: YES), the receiving unit 261 executes step S21 again to receive an input of a condition name.

[0069] On the other hand, if it is determined in step S25 that inputs relating to other conditions have not been received (step S25: NO), the process S2 ends.

[0070] (Another example of processing executed by the reception unit 261 and the generation unit 262) Another example of the processing executed by the receiving unit 261 and the generating unit 262 will be described with reference to Fig. 7. Fig. 7 is a flow diagram showing an example of the flow of processing S3 executed by the receiving unit 261 and the generating unit 262 in this exemplary embodiment. In processing S3, the receiving unit 261 receives an input regarding the selection of one or more predetermined conditions and an input of a score regarding the selected condition, and the generating unit 262 generates a score table.

[0071] (Step S31) In step S31, the receiving unit 261 receives a selection of a process. As an example, when the input unit 22 receives an operation input from the user indicating that “Process A” has been selected, the receiving unit 261 stores in the storage unit 25 input information indicating that the selected process is “Process A.”

[0072] (Step S32) In step S32, the receiving unit 261 receives a selection of a condition. As an example, when the input unit 22 receives an operation input from the user indicating that “condition a” has been selected, the receiving unit 261 stores in the storage unit 25 input information indicating that the selected condition is “condition a.”

[0073] As another example, when the input unit 22 receives an operation input from the user indicating that "condition a" and "condition c" have been selected, the receiving unit 261 stores input information in the memory unit 25 indicating that the selected conditions are "condition a" and "condition c."

[0074] (Step S33) In step S33, the receiving unit 261 determines whether or not a plurality of conditions are selected in step S32.

[0075] As an example, the receiving unit 261 refers to the input information stored in the storage unit 25, and if the input information indicates that the selected condition is "condition a," the receiving unit 261 determines in step S33 that there are not multiple conditions selected in step S32 (step S33: NO). Then, the receiving unit 261 next executes step S36, which will be described later.

[0076] (Step S34) As another example, the receiving unit 261 refers to the input information stored in the memory unit 25, and if the input information indicates that the selected conditions are "condition a" and "condition c," then in step S33 the receiving unit 261 determines that multiple conditions were selected in step S32 (step S33: YES).

[0077] In this case, in step S34, the receiving unit 261 receives input regarding the selection of multiple conditions. As an example, when the input unit 22 receives an operation input indicating "condition a" and "condition c" from the user, the receiving unit 261 stores in the storage unit 25 input information indicating that the input regarding the selection of multiple conditions is "condition a" and "condition c."

[0078] Furthermore, in step S34, the receiving unit 261 may further receive a condition name as an input regarding the selection of multiple conditions. As an example, when the input unit 22 receives an operation input from the user indicating that the condition name of "condition a" and "condition c" is "condition α", the receiving unit 261 stores in the storage unit 25 input information indicating that the condition name of "condition a" and "condition c" is "condition α".

[0079] (Step S35) In step S35, the generation unit 262 generates new conditions based on the selected conditions. For example, the generation unit 262 references the input information stored in the storage unit 25 by the reception unit 261 in step S34 to generate new conditions.

[0080] As an example, if the input information referenced by the generation unit 262 indicates that the condition name is "condition α" and the condition contents are "condition a" and "condition c," the generation unit 262 generates the condition that the condition name "condition α" satisfies "condition a" and "condition c."

[0081] (Step S36) In step S36, the receiving unit 261 receives an input of a score. The receiving unit 261 stores input information indicating the received score in the storage unit 25. An example of the process in which the receiving unit 261 receives an input of a score will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of a screen for receiving an input of a score in this exemplary embodiment.

[0082] First, the display unit 263 refers to the input information stored in the storage unit 25, and displays on the display panel 24 image information showing an image prompting the user to input the degree to which the condition selected in step S32 contributes to the discrimination of the process selected in step S31. As an example, if the process selected in step S31 is "process B" and the condition selected in step S32 is "condition b," an image prompting the user to input the degree to which "condition b" contributes to the discrimination of "process B," as shown in FIG. 8, is displayed on the display panel 24.

[0083] Here, the display unit 263 may be configured to display a table ST1 in which each step, each condition, and the input score are associated with each other, as shown in FIG.

[0084] (Step S37) In step S37, the generation unit 262 generates a score table. As an example, the generation unit 262 generates the score table by referring to the input information received by the reception unit 261 in step S36. An example of the score table generated by the generation unit 262 will be described with reference to Fig. 9. Fig. 9 is a diagram showing an example of the score table in this exemplary embodiment.

[0085] 9, the generation unit 262 generates a score table including scores for each of one or more conditions (condition a, condition b, and condition c) and each score for each of multiple processes (process A, process B, and process C). In the score table shown in the upper part of Fig. 9, for example, the degree to which "condition a" contributes to the discrimination of "process A" is "0.5," and the degree to which "condition b" contributes to the discrimination of "process B" is "-0.2."

[0086] Furthermore, when a condition is generated by a plurality of conditions, the generation unit 262 may generate a score table that also includes the condition name and the content of the condition for that condition name, as shown in the lower part of Fig. 9. In the score table shown in the lower part of Fig. 9, the condition name "condition α" indicates "condition a" and "condition c," and the degree to which "condition α" contributes to the discrimination of "process A" is "0.5," and the degree to which "condition α" contributes to the discrimination of "process B" is "0."

[0087] (Step S38) In step S38, the receiving unit 261 determines whether or not an input related to another process has been received.

[0088] If it is determined in step S38 that an input relating to another process has been received (step S38: YES), the receiving unit 261 executes step S31 again to receive a selection of a process.

[0089] On the other hand, if it is determined in step S38 that input related to other processes has not been received (step S38: NO), the process S3 ends.

[0090] (An example of processing executed by the determination unit 13) An example of the processing executed by the discrimination unit 13 will be described with reference to the score table shown in Fig. 9. The discrimination unit 13 calculates a discrimination value, which will be described later, and determines the process represented by the video according to the discrimination value.

[0091] As an example, if the confidence level for "condition a" calculated by the calculation unit 12 is "0.2," the discrimination unit 13 calculates, as a judgment value, the product of the confidence level "0.2" and the score of each process associated with "condition a" in the score table shown in the upper part of Figure 9, as follows: Process A: 0.2×0.5=0.1 Process B:0.2×0.7=0.14 Process C:0.2×0=0 The discrimination unit 13 then discriminates the process represented by the video based on the calculated judgment value. As an example, the discrimination unit 13 discriminates the process with the highest judgment value among the calculated judgment values ​​as the process represented by the video. In the above example, the process with the highest judgment value is "0.14" for "Process B," so the discrimination unit 13 determines that the process represented by the video is "Process B."

[0092] As another example, the determination unit 13 may be configured to determine that a process having the highest determination value is the process represented by the video when the determination value for that process is higher than a threshold value.

[0093] For example, if the threshold is set to "0.12," the discrimination unit 13 determines whether the highest judgment value "0.14" is higher than the threshold in the above example. Because the judgment value "0.14" is higher than the threshold "0.12," the discrimination unit 13 determines that the process represented by the video is "Process B," with a judgment value of "0.14." On the other hand, if the threshold is set to "0.15," the highest judgment value "0.14" is lower than the threshold in the above example, so the discrimination unit 13 determines that there is no process represented by the video.

[0094] (Another example of processing executed by the determination unit 13) As another example, a case where there are multiple conditions in the score table will be described using the score table also shown in the upper part of Fig. 9. When there are multiple conditions, the discrimination unit 13 discriminates the process represented by the video based on a weighted sum of the certainty factors for each condition, where the score for that condition is used as a weighting coefficient. In other words, the discrimination unit 13 calculates, as a judgment value, a weighted sum of the certainty factors for each condition, where the score for that condition is used as a weighting coefficient.

[0095] As an example, if the certainty factors calculated by the calculation unit 12 are "0.2" for "condition a," "0" for "condition b," and "0.8" for "condition c," the discrimination unit 13 calculates a weighted sum using the score as a weighting coefficient as follows: Process A: 0.2×0.5+0×0+0.8×1=0.9 Process B:0.2×0.7+0×(-0.2)+0.8×0=0.14 Process C: 0.2×0+0×0+0.8×0.5=0.4 The discrimination unit 13 then uses the calculated weighted sum as a judgment value and, as in the example described above, discriminates the process represented by the video based on the judgment value. That is, the discrimination unit 13 determines that the process represented by the video is "Process A," which has the highest judgment value of "0.9." Alternatively, as in the example described above, the discrimination unit 13 may set a threshold value and discriminate the process represented by the video that has the highest judgment value higher than the threshold value.

[0096] (Information processing method S4) The flow of the information processing method S4 in which the information processing device 2 according to this exemplary embodiment determines the process represented by the video will be described with reference to Fig. 10. Fig. 10 is a flow diagram showing the flow of the information processing method S4 according to this exemplary embodiment.

[0097] (Step S41) In step S41, the receiving unit 261 receives the selection of a model. Here, a model is a collection of conditions and scores (score table) that are referenced to distinguish between predetermined processes. For example, "Model A" is a model that collects conditions and scores that are referenced to distinguish between "Process A," "Process B," and "Process C," and "Model B" is a model that collects conditions and scores that are referenced to distinguish between "Process A," "Process B," and "Process D."

[0098] (Step S42) In step S42, the receiving unit 261 determines whether or not an input for changing a model has been received. Examples of changing a model include adding a process to the model, adding a condition, changing the content of a condition, and changing the score.

[0099] (Step S43) If it is determined in step S42 that an input to change the model has been received (step S42: YES), the receiving unit 261 receives the change content of the model in step S43. Then, the generation unit 262 changes the model by referring to the change content received by the receiving unit 261. The generation unit 262 may use the processing described with reference to FIGS. 4 and 7 as the processing for changing the model.

[0100] (Step S44) In step S42, if it is determined that an input to change the model has not been received (step S42: NO), the receiving unit 261 determines whether or not a command to execute a determination process to determine the process represented by the video has been received.

[0101] If it is determined in step S44 that the execution of the determination process has not been accepted (step S44: NO), the process in the information processing method S4 ends.

[0102] (Step S45) If it is determined in step S44 that the execution of the determination process has been accepted (step S44: YES), in step S45 the accepting unit 261 accepts a selection of which video to use as the target video. The accepting unit 261 stores input information indicating the selected target video in the storage unit 25.

[0103] (Step S46) In step S46, the calculation unit 12 refers to the input information stored in the storage unit 25 and acquires the target video from the storage unit 25. Then, the calculation unit 12 refers to the conditions stored in the storage unit 25 and calculates a certainty factor as to whether the acquired target video satisfies one or more predetermined conditions. The calculation unit 12 stores the calculated certainty factor in the storage unit 25. Here, as described above, when the calculation unit 12 calculates a certainty factor for each of a plurality of conditions, the calculation unit 12 stores each of the calculated certainty factors in the storage unit 25 in association with the condition used to calculate the certainty factor.

[0104] As an example of the certainty factor calculated by the calculation unit 12, when the condition is that the color of object OBJ2 included as an element in the video is red, as shown in the upper part of Fig. 6, the certainty factor calculated by the calculation unit 12 is high when the color of object OBJ2 is black (other than red), as shown in the lower part of Fig. 6.

[0105] Furthermore, the calculation unit 12 may be configured to calculate a degree of certainty as to whether or not the sound associated with the target video satisfies one or more predetermined conditions. For example, if the condition is that a predetermined phrase is contained in the audio included as an element in the video, the degree of certainty calculated by the calculation unit 12 will be high if the predetermined phrase is contained in the audio associated with the target video. With this configuration, the information processing device 2 can flexibly determine the process represented by the video even when a sound associated with the video is used.

[0106] (Step S47) In step S47, the discrimination unit 13 refers to the confidence levels stored in the memory unit 25 and discriminates the process represented by the video based on the confidence level for each condition and a score indicating the degree to which the condition contributes to the discrimination of the process. As an example, the discrimination unit 13 refers to a score table to discriminate the process represented by the video. The process by which the discrimination unit 13 discriminates the process represented by the video is as described above. The discrimination unit 13 stores the discrimination result in the memory unit 25.

[0107] (Example of video including the discrimination result) An example of an image including the determination result displayed by the display unit 263 will be described with reference to Fig. 11. Fig. 11 is a diagram showing an example of an image including the determination result displayed in this exemplary embodiment.

[0108] As an example, the display unit 263 displays a target video including the discrimination result DR and the judgment value JV on the display panel 24, as shown in FIG.

[0109] Furthermore, when an element is included in the target video, the display unit 263 may further display a confidence level as to whether or not a condition for including the element is satisfied. As an example, as shown in Fig. 11 , the display unit 263 may display the target video on the display panel 24, further including a confidence level CF1 as to whether or not a condition for including an object OBJ4, which is an element included in the target video, is satisfied. For this configuration, the calculation unit 12 may associate the calculated confidence level with one or more predetermined elements referenced in each of one or more predetermined conditions for calculating the confidence level.

[0110] Furthermore, the display unit 263 may further display the judgment value. As an example, the display unit 263 may display on the display panel 24 a target video that further includes a judgment value JV1 calculated with reference to the confidence factor CF1, as shown in Fig. 11. For this configuration, the determination unit 13 may associate the calculated judgment value with one or more predetermined elements referenced in each of one or more predetermined conditions for calculating the judgment value.

[0111] With these configurations, the information processing device 2 can present to the user the certainty factor calculated by the calculation unit 12 and the judgment value calculated by the discrimination unit 13. Therefore, the information processing device 2 can allow the user to confirm whether the certainty factor calculated by the calculation unit 12 and the judgment value calculated by the discrimination unit 13 are valid.

[0112] 11, the display unit 263 may similarly display on the display panel 24 a target video image that further includes a confidence factor CF2 and a judgment value JV2 for the object OBJ5.

[0113] (Another example of video containing the classification results) Another example of an image including a determination result displayed by the display unit 263 will be described with reference to Fig. 12. Fig. 12 is a diagram showing another example of an image including a determination result displayed in this exemplary embodiment.

[0114] The display unit 263 may display any multiple images included in the target video and the discrimination results for the any multiple images. As an example, as shown in Fig. 12, the display unit 263 displays a video including any multiple images PIC1, PIC2, ​​and PIC3 included in the target video on the display panel 24. The images PIC1, PIC2, ​​and PIC3 are not particularly limited, and may be images at the time when the discrimination result changes, or may be images of the first frame in the divided video obtained by dividing the target video at predetermined time intervals, or any images included in the divided video.

[0115] Then, the display unit 263 displays on the display panel 24 an image including the discrimination results DR1, DR2, and DR3 of the plurality of images PIC1, PIC2, ​​and PIC3, respectively.

[0116] In addition, the display unit 263 may display on the display panel 24 an image that further includes time information TI1, time information TI2, and time information TI3 that indicate the times when the multiple images PIC1, PIC2, ​​and PIC3 were captured, respectively.

[0117] With these configurations, the information processing device 2 can present to the user which process each of the multiple images PIC1, PIC2, ​​and PIC3 has been determined to be in. Furthermore, if the images PIC1, PIC2, ​​and PIC3 are images at the time when the determination results have changed, the information processing device 2 can present to the user that the determination results have changed for the images PIC1, PIC2, ​​and PIC3.

[0118] As described above, the information processing device 2 according to this exemplary embodiment employs a configuration including an acquisition unit 11 that acquires video, a calculation unit 12 that calculates a degree of certainty as to whether or not a condition is met, provided that the video contains one or more predetermined elements, and a discrimination unit 13 that determines the process represented by the video by referring to a score table that includes scores for each of the one or more conditions, each score being for each of a plurality of processes.

[0119] Therefore, according to the information processing device 2 of this exemplary embodiment, since one or more predetermined elements included in the video are used as conditions, it is possible to suitably identify the process. Furthermore, according to the information processing device 2 of this exemplary embodiment, since a score table including scores for each of one or more conditions and each of multiple processes is referenced, it is possible to suitably identify the process even if there are multiple conditions and processes.

[0120] [Software implementation example] Some or all of the functions of the information processing devices 1 and 2 may be realized by hardware such as an integrated circuit (IC chip), or by software.

[0121] In the latter case, the information processing devices 1 and 2 are realized, for example, by a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in FIG. 13. The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for operating the computer C as the information processing devices 1 and 2. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the information processing devices 1 and 2.

[0122] The processor C1 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0123] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer.

[0124] The program P can also be recorded on a non-transitory, tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communications network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0125] [Appendix 1] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiments are also included in the technical scope of the present invention.

[0126] [Appendix 2] Some or all of the above-described embodiments can also be described as follows, but the present invention is not limited to the following described aspects.

[0127] (Appendix 1) An information processing device comprising: an acquisition means for acquiring an image; a calculation means for calculating a degree of certainty as to whether the image satisfies one or more predetermined conditions; and a determination means for determining the process represented by the image based on the degree of certainty for each of the conditions and a score indicating the degree to which the condition contributes to the determination of the process.

[0128] (Appendix 2) 2. The information processing device according to claim 1, wherein each of the conditions includes that the video contains one or more predetermined elements.

[0129] (Appendix 3) The information processing device described in Appendix 2, further comprising: a first receiving means for receiving input regarding the specified one or more elements; and a first generating means for generating the condition by referring to the input received by the first receiving means.

[0130] (Appendix 4) The information processing device according to claim 2 or 3, wherein the calculation means associates the calculated certainty with the one or more predetermined elements referenced in each of the one or more predetermined conditions for calculating the certainty.

[0131] (Appendix 5) The information processing device described in any one of Appendices 1 to 4, wherein the discrimination means discriminates the process represented by the video based on a weighted sum of the confidence levels for each of the conditions, the weighted sum using the score for the condition as a weighting coefficient.

[0132] (Appendix 6) The information processing device according to any one of appendices 1 to 5, wherein the discrimination means discriminates the process represented by the video by referring to a score table including scores for each of one or more conditions and each score for each of a plurality of processes.

[0133] (Appendix 7) The information processing device described in Appendix 6 further comprises a second receiving means for receiving input regarding the selection of one or more conditions, a third receiving means for receiving input of the score regarding the selected condition, and a second generating means for generating the score table based on the input of the score.

[0134] (Appendix 8) The information processing device according to claim 7, wherein the second receiving means receives input regarding the selection of a plurality of conditions, and further comprises a third generating means that generates new conditions based on the selected plurality of conditions.

[0135] (Appendix 9) 9. The information processing device according to any one of claims 1 to 8, wherein the determining means changes a time interval for a determination process to be performed in the future depending on a result of a determination performed in the past.

[0136] (Appendix 10) An information processing device according to any one of claims 1 to 9, wherein the acquisition means further acquires sound related to the video, and the determination means further refers to the sound to determine the process represented by the video.

[0137] (Appendix 11) An information processing method including the steps of: an information processing device acquiring an image; calculating a degree of certainty as to whether the image satisfies one or more predetermined conditions; and determining a process represented by the image based on the degree of certainty for each of the conditions and a score indicating the degree to which the condition contributes to the determination of the process.

[0138] (Appendix 12) A program that causes a computer to function as an information processing device, the program causing the computer to function as: an acquisition means that acquires video; a calculation means that calculates a degree of certainty as to whether the video satisfies one or more predetermined conditions; and a discrimination means that discriminates a process represented by the video based on the degree of certainty for each of the conditions and a score that indicates the degree to which the condition contributes to the discrimination of the process.

[0139] [Appendix 3] Some or all of the above-described embodiments can also be expressed as follows.

[0140] An information processing device comprising at least one processor that executes an acquisition process for acquiring video, a calculation process for calculating a degree of certainty as to whether the video satisfies one or more predetermined conditions, and a determination process for determining the process represented by the video based on the degree of certainty for each of the conditions and a score indicating the degree to which the condition contributes to the determination of the process.

[0141] The information processing device may further include a memory that stores a program for causing the processor to execute the acquisition process, the calculation process, and the determination process. The program may be recorded on a computer-readable, non-transitory, tangible recording medium. [Explanation of symbols]

[0142] 1, 2 Information processing device 11 Acquisition Department 12 Calculation section 13 Discrimination part 26 Control Unit 261 Reception Department 262 Generation part 263 Display section

Claims

1. an acquisition means for acquiring a video; a calculation means for calculating a degree of certainty as to whether the video satisfies one or more predetermined conditions; a discrimination means for discriminating the process represented by the video based on a degree of certainty for each of the conditions and a score indicating the degree to which the condition contributes to discrimination of the process; Equipped with The determining means changes the time interval for a determination process to be performed in the future in accordance with a result of a determination performed in the past. Information processing device.

2. The above conditions are: The video contains one or more predetermined elements. The information processing device according to claim 1 ,

3. a first receiving means for receiving an input relating to the predetermined one or more elements; a first generating means for generating the condition by referring to the input received by the first receiving means; The information processing device according to claim 2 , further comprising:

4. The calculation means associates the calculated certainty with the predetermined one or more elements referred to in each of the predetermined one or more conditions for calculating the certainty.

4. The information processing device according to claim 2 or 3.

5. The determining means determines the process represented by the video based on a weighted sum of certainty factors for each of the conditions, the weighted sum having a score for the condition as a weighting coefficient. The information processing device according to claim 1 .

6. The determining means determines the process represented by the image by referring to a score table including scores for each of one or more conditions and each of a plurality of processes. The information processing device according to claim 1 .

7. a second receiving means for receiving an input regarding the selection of one or more conditions; a third receiving means for receiving an input of the score related to the selected condition; a second generating means for generating the score table based on the input of the scores; The information processing apparatus according to claim 6 , further comprising:

8. The information processing device Acquiring video and calculating a confidence level as to whether the video satisfies one or more predetermined conditions; Identifying the process represented by the video based on a confidence level for each of the conditions and a score indicating the degree to which the condition contributes to the process identification; Including, Change the time interval for future discrimination processes depending on past discrimination results Information processing methods.

9. A program that causes a computer to function as an information processing device, The program causes the computer to: an acquisition means for acquiring a video; a calculation means for calculating a degree of certainty as to whether the video satisfies one or more predetermined conditions; a determination means for determining the process represented by the video based on a degree of certainty for each of the conditions and a score indicating the degree to which the condition contributes to the determination of the process; It functions as The determining means changes the time interval for a determination process to be performed in the future depending on the result of the determination performed in the past. program.

Images