Delay estimation device, clustering processing device, and program

Abstract

To provide a delay estimation device capable of accurately estimating a delay time of transmission on the basis of text and information about the transmission time.SOLUTION: A delay estimation device comprises a model that inputs text and a transmission time of the text and outputs the delay time of the transmission time. In one embodiment, the model connects a BERT-type word embedding layer being a layer on an input side and a neural network regression layer being a layer on an output side. The BERT-type word embedding layer inputs the text and the transmission time of the text. The neural network regression layer inputs the information output from the BERT-type word embedding layer and outputs the delay time of the transmission time.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a delay estimation device, a clustering processing device, and a program. [Background technology]

[0002] In recent years, deep learning methods have been used to analyze the content of text. Text analysis technology that uses deep learning methods utilizes features called distributed text representations. There has been vigorous research into learning and utilization methods for models and distributed representations that utilize deep learning. In particular, a model called BERT (Bidirectional Encoder Representations from Transformers) has been shown through experiments to achieve the best performance compared to other deep learning models in classification tasks such as document classification and entailment determination, and regression tasks such as determining sentence similarity.

[0003] Non-Patent Document 1 and Non-Patent Document 2 describe text distributed representations. Non-Patent Document 3 describes a method using contrastive learning as an improvement to the learning method using distributed representations. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean, "Efficient Estimation of Word Representations in Vector Space", arXiv:1301.3781, ICLR 2013, 2013. [Non-Patent Document 2] Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”, NAACL 2019, 2019. [Non-Patent Document 3] Nils Reimers, Iryna Gurevych, “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks”, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019. Summary of the Invention [Problem to be solved by the invention]

[0005] It is possible to use text analysis technology to analyze social media. Social media is a medium designed to enable a wide range of people to interact with each other through social interactions via communication networks such as the Internet. In many cases, texts posted by people on social media are accompanied by information about the time of posting (date and time).

[0006] However, there has been no method to link and utilize text and time of posting information simultaneously, and it has rarely been tackled as a research topic. In particular, there is almost no prior research on the integrated use of text and time of posting information or the modeling of such use.

[0007] For example, text (such as tweets) generated in real time by viewers while watching content is often deeply related to the content at that time. However, not all viewers generate text perfectly synchronized with the content, but rather the text is generated and sent with a delay for each viewer. However, the amount of delay is generally unknown. Simply using the time the text was sent as is does not allow for accurate text analysis (for example, clustering of texts sent individually on social media).

[0008] The present invention has been made in consideration of the above-mentioned circumstances, and aims to provide a delay estimation device, a clustering processing device, and a program that can accurately estimate a call delay time based on text and call time information. [Means for solving the problem]

[0009] [1] In order to solve the above problem, a delay estimation device according to one aspect of the present invention includes a model that receives a text and a sending time of the text as input and outputs a delay time of the sending time.

[0010] [2] Moreover, one aspect of the present invention is that in the delay estimation device of [1] above, the text is text related to a scene in the content to be distributed, and the delay time is a delay time from the time when the scene is distributed to the time when the text is transmitted.

[0011] [3] Also, one aspect of the present invention is a delay estimation device according to the above [1] or [2], wherein the model is configured by connecting a BERT-type word embedding layer, which is an input layer, and a neural network regression layer, which is an output layer, and the BERT-type word embedding layer inputs the text and the sending time of the text, and the neural network regression layer inputs information output from the BERT-type word embedding layer and outputs the delay time of the sending time.

[0012] [4] In one aspect of the present invention, the delay estimation device according to [3] further includes a control unit that controls the learning of the model, and the control unit performs unsupervised learning using a sentence as an input for the BERT-type word embedding layer as a first stage of learning of the model, and maintains values ​​of internal parameters of the BERT-type word embedding layer in the first stage. The control unit performs learning using a sentence pair as an input for the BERT-type word embedding layer and the neural network regression layer in a second stage of learning of the model, and performs learning in which a sentence pair is used as an input and the similarity between two sentences included in the sentence pair is given as ground truth data. The control unit controls the BERT-type word embedding layer and the neural network regression layer to control the learning of the BE in the second stage. The internal parameter values ​​of the RT-type word embedding layer are maintained, and the internal parameter values ​​of the neural network regression layer in the second stage are discarded. In the third stage of learning of the model, a sentence is input and learning is performed for the BERT-type word embedding layer and the neural network regression layer in which the delay time corresponding to the sentence is given as a correct answer. The internal parameter values ​​of the BERT-type word embedding layer in the third stage are maintained, and the internal parameter values ​​of the neural network regression layer in the third stage are maintained.

[0013] [5] Also, one aspect of the present invention is a clustering processing device comprising: the delay estimation device of [3] or [4] above; and a clustering processing unit that performs clustering processing based on a distributed representation output from a BERT-type word embedding layer of the delay estimation device based on an input text, a sending time of the text, and a delay time of the sending time output from the neural network regression layer of the delay estimation device based on the text.

[0014] [6] Furthermore, one aspect of the present invention is that the clustering processing device of [5] above further includes a correction unit that outputs a corrected sending time by correcting the sending time of the text based on a delay time of the sending time, and the clustering processing unit performs clustering processing based on the distributed representation and the corrected sending time.

[0015] [7] Another aspect of the present invention is a program for causing a computer to function as any one of the delay estimation devices described above in [1] to [4].

[0016] [8] Another aspect of the present invention is a program for causing a computer to function as the clustering processing device according to [5] or [6] above. Effect of the Invention

[0017] According to the present invention, it is possible to estimate the delay time of the transmission time of the text. [Brief description of the drawings]

[0018] [Figure 1] 1 is a block diagram showing a schematic functional configuration of a delay estimation device according to a first embodiment of the present invention. [Diagram 2] 10 is a schematic diagram showing the relationship between the timing of a scene in the content delivered according to the embodiment and an example of the time distribution of the number of submissions of viewer-generated text related to that scene. FIG. [Diagram 3] 4 is a schematic diagram showing an example of data input to the delay estimation device according to the embodiment. FIG. [Figure 4] 1 is a schematic diagram showing a list of norms (loss functions), types of output layers, and learning data (correct data, teacher data) at each stage of learning of a model of the delay estimation device of the embodiment. FIG. [Diagram 5] 4 is a schematic diagram showing a process flow of second-stage learning of a model in the delay estimation device of the embodiment. FIG. [Figure 6] 4 is a schematic diagram showing an example of learning data used in second-stage learning of the model in the delay estimation device of the embodiment. FIG. [Figure 7] 4 is a schematic diagram showing a process flow of a third stage of model learning in the delay estimation device according to the embodiment. FIG. [Figure 8] FIG. 11 is a block diagram showing a schematic functional configuration of a clustering processing device according to a second embodiment. [Figure 9] FIG. 11 is a block diagram showing a schematic functional configuration of a clustering processing device according to a modified example of the second embodiment. [Figure 10] 10 is a schematic diagram for explaining a clustering process performed by the clustering processing device of the embodiment (including a modification); FIG. [Figure 11] FIG. 11 is a schematic diagram showing an example of a result of a clustering process according to the embodiment (including a modified example). [Figure 12] FIG. 2 is a block diagram showing an example of the internal configuration of each device (computer configuration) in the first embodiment and the second embodiment (including modified examples). [Figure 13] FIG. 13 is a diagram for explaining the effect of the embodiment, and is a schematic diagram showing a comparison of regression performances using various models. [Figure 14] FIG. 13 is a diagram for explaining the effect of the embodiment, and is a schematic diagram showing a comparison of clustering performances using various models. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Next, embodiments of the present invention will be described with reference to the drawings. A delay estimation device according to a first embodiment and a clustering processing device according to a second embodiment have a function of analyzing texts posted on social media.

[0020] Audience-generated texts are sent in real time by viewers who are watching content, and are generally sent shortly after the viewing. In other words, the time of sending is a useful clue for identifying which scene in the content the audience-generated text is related to. In other words, when the content is a broadcast program, there is a strong correlation between the time when a particular scene is broadcast and the distribution of the sending times of audience-generated texts related to that scene. In addition, the sending times of audience-generated texts are distributed with a slight delay from the time when the scene is broadcast.

[0021] When analyzing and clustering such AGTs, the time each AGT was sent can be a useful clue. As a result of the clustering process, clusters are formed that correspond to broadcast scenes. In other words, it becomes possible to quickly grasp the audience reaction to each broadcast scene based on the AGT.

[0022] However, it is not self-evident how to utilize the time of transmission. When performing clustering, generally, features that quantify each transmission are used. As features, distributed representations based on deep learning can be used for text that represents the content of the transmission. For the time of transmission, the value of the timestamp can be used as is. However, even now when distributed text representations based on deep learning have become an effective technology, there is still an issue that the simple combination of text features and transmission time by the conventional method BERT cannot provide sufficient accuracy to analyze which scene the text transmitted during real-time viewing corresponds to.

[0023] In the embodiment described below, a transmission delay time is estimated (predicted) for text (so-called "tweets" and the like, which may generally be called "viewer-generated text") transmitted in real time by viewers watching distributed content (e.g., a broadcast program, etc.). By using the estimated delay time, it is expected that analysis of viewer-generated text can be performed with higher accuracy. In other words, compared to a simple combination of conventional technologies, the utility of viewer-generated text is increased in the embodiment described below.

[0024] The configuration of the devices of the first and second embodiments will now be described.

[0025] [First embodiment] FIG. 1 is a block diagram showing a schematic functional configuration of a delay estimation device according to this embodiment. As shown in the figure, the delay estimation device 1 includes a control unit 101, a word string input unit 111, a BERT-type word embedding layer 112, a pooling layer 113, a neural network regression layer 114, an outgoing delay time output unit 115, and a similarity calculation unit 121. The model 110 will be described later. Each functional unit constituting the delay estimation device 1 can be realized, for example, by a computer and a program. Each functional unit has a storage means as necessary. The storage means is, for example, a variable in a program or a memory allocated by the execution of a program. Also, non-volatile storage means such as a magnetic hard disk device or a solid state drive (SSD) may be used as necessary. Also, at least a part of the functions of each functional unit may be realized as a dedicated electronic circuit rather than a program. The functions of each unit are as follows.

[0026] The control unit 101 controls the entire processing of the delay estimation device 1. The control unit 101 at least performs control according to the processing execution mode of the delay estimation device 1. The delay estimation device 1 can execute processing by switching between a learning mode and an estimation mode. In the learning mode, the delay estimation device 1 performs machine learning of a model provided inside by using learning data. That is, the control unit 101 controls the learning of the model held by the delay estimation device 1. Also, in the estimation mode, the delay estimation device 1 estimates a delay time corresponding to input data (text and sending time) by using the above-mentioned trained model. The model to be learned is a model 110 described later. Specifically, the delay estimation device 1 performs model learning as a three-stage process from a first stage to a third stage. The specific contents of the learning of these three stages will be described in detail later. That is, the control unit 101 controls the execution of processing by the delay estimation device 1 while appropriately switching between each mode, namely, a first-stage learning mode, a second-stage learning mode, a third-stage learning mode, and an estimation mode. When processing in the first stage learning mode, the second stage learning mode, and the third stage learning mode, the control unit 101 supplies learning data (input data and supervised answer data) appropriate for each stage.

[0027] The word string input unit 111 supplies a word string corresponding to the text of the input data (the text is usually one or more sentences) to the BERT-type word embedding layer 112 described later. The input text (viewer-generated text) and the word string are equivalent information with different forms. In other words, the delay estimation device 1 can convert text into an equivalent word string, and conversely, can convert a word string into an equivalent text. When converting a text (sentence) into a word string, for example, a morphological analysis process realized as an existing technology is performed. Conversely, when converting a word string into a text (sentence), a process of sequentially concatenating the words that make up the string is performed.

[0028] In this embodiment, the text that the word string input unit 111 supplies to the BERT-type word embedding layer 112 is text related to a predetermined scene in content (e.g., a broadcast program, etc.) to be distributed (e.g., broadcast or distributed online). Viewers can send text using various SNS (social networking service) functions. The word string input unit 111 can convert text (viewer-generated text) acquired from a server device of the SNS into a word string and supply it to the BERT-type word embedding layer 112. At this time, the server device of the SNS passes the sending time of the text (viewer-generated text) to the word string input unit 111 together with the text (viewer-generated text). The sending time is, for example, the time when the server device of the SNS accepts the viewer-generated text transmitted from the terminal device of the viewer.

[0029] The BERT-type word embedding layer 112 inputs a word string passed from the word string input unit 111, and outputs a distributed representation corresponding to the original text. In other words, the BERT-type word embedding layer 112 inputs text information and information on the time of publication of the text, and outputs a distributed representation corresponding to the text. The BERT-type word embedding layer 112 passes the output distributed representation to the pooling layer 113. Note that BERT (Bidirectional Encoder Representations from Transformers) itself is an existing technology, and is realized as a form of processing using a neural network. Note that the distributed representation is a multidimensional numerical vector corresponding to the meaning of a word.

[0030] The pooling layer 113 aggregates the distributed representations for each token output from the BERT-type word embedding layer 112 to output a vector corresponding to a sentence. The output from the pooling layer 113 is passed to the neural network recurrent layer 114.

[0031] The neural network regression layer 114 calculates an output value based on the output from the pooling layer 113. The neural network regression layer 114 is realized as a fully connected neural network. The meaning of the data output by the neural network regression layer 114 depends on the training data used during training. Training of the model 110 including the neural network regression layer 114 is performed in three stages (from the first stage training to the third stage training) as described below. In the second stage training, the neural network regression layer 114 is trained so that the similarity between sentence A and sentence B constituting a sentence pair can be calculated. In the third stage training, the neural network regression layer 114 is trained so that the delay time of the input sentence (text) can be estimated and output. After completing the second stage training and before starting the third stage training, the values ​​of the internal parameters of the neural network regression layer 114 are discarded (i.e., reset). After the third stage of learning is completed, when the delay estimation device 1 operates in the estimation mode, the neural network recurrent layer 114 is expected to output an estimated value of the delay time with respect to the transmission time of the viewer generated text due to the action of the internal parameters optimized by the third stage of learning. Note that this delay time is the time when a specific scene in the content is distributed (time T 1 ) to the transmission time of the viewer generated text. In other words, the neural network regression layer 114, which has completed learning, inputs the information output from the BERT word embedding layer 112 and outputs the delay time of the transmission time.

[0032] When the delay estimation device 1 operates in the estimation mode, the transmission delay time output unit 115 outputs the output value from the neural network regression layer 114 (that is, the estimated value of the transmission delay time) to the outside.

[0033] The similarity calculation unit 121 calculates the similarity between sentence A and sentence B when the delay estimation device 1 operates in the second-stage learning mode. That is, the similarity calculation unit 121 calculates the similarity between a vector output by the neural network regression layer 114 based on sentence A and a vector output by the neural network regression layer 114 based on sentence B when the delay estimation device 1 operates in the second-stage learning mode. The similarity calculation unit 121 may calculate, for example, cosine similarity as the similarity. However, the similarity calculation unit 121 may calculate the similarity between sentence A and sentence B calculated by some method other than cosine similarity.

[0034] FIG. 2 is a schematic diagram (graph) showing the relationship between the timing of a scene in the distributed content and an example of the time distribution of the number of viewer-generated text posts related to that scene. In the figure, the horizontal axis corresponds to time, and the vertical axis corresponds to the number of viewer-generated text posts. The example graph shown in the figure shows the progress of the number of viewer-generated text posts corresponding to a specific scene. In the figure, the number of posts for each time period with a predetermined fixed width is shown as a bar graph. In the figure, P 1 is the time period during which a particular scene was distributed (e.g., broadcast). 1 The length of P 1 does not have to have a length. 1 is the time representative of the particular scene. 1 is the time period P 1 In the illustrated example, the time period P 1 The start time of 1 However, it does not necessarily correspond to the time period P 1 The start time of 1 It does not have to match. 1 When does not have length, it is usually 1 is time P 1 Matches the time period P 2 The viewer-generated texts related to the particular scene are distributed at time T 1The delay is different for each viewer generated text. In this example, the delay occurs at time T 1 The number of viewers generated texts sent per time slot increases at first, then peaks, and then decreases after the peak. In general, the distribution of viewers generated texts sent per time slot may be different from the distribution shown in the graph in this example.

[0035] FIG. 3 is a schematic diagram showing an example of data to be input to delay estimation device 1. As shown in the figure, the input data is a set of pairs of viewer generated text and its sending time. In this example, the sending time is expressed in the format of "hh:mm:ss" (hours, minutes, seconds), but the time may be expressed in other formats. For example, the sending time data may include information on the year, month, and day. Note that while the figure shows an example of four lines (four items) of data, the number of input data items is arbitrary. In both the learning mode and the estimation mode, delay estimation device 1 can process the input data one item at a time.

[0036] The delay estimation device 1 having the above configuration effectively utilizes the transmission time of viewer-generated text. However, the transmission time of text by a viewer viewing content in real time and the time when a scene corresponding to the matter described in the content of that text is distributed are not necessarily the same time. The delay estimation device 1 defines and treats the time difference between the two as the transmission delay time. Based on observation of actual data, the transmission delay time is not uniform, but differs for each transmitted text. The delay estimation device 1 performs modeling using the delay time as the objective variable and the content of the text as the explanatory variable.

[0037] The model is the model 110 shown in FIG. 1. That is, the model 110 includes a BERT-type word embedding layer 112, a pooling layer 113, and a neural network recurrent layer 114, and is configured by connecting these layers in order. That is, in the model 110, the input side layer is the BERT-type word embedding layer 112, and the output side layer is the neural network recurrent layer 114. The delay estimation device 1 trains the model 110 through a three-stage learning process. The trained model 110 inputs a word string and the transmission time of the word string, and outputs a transmission delay time (estimated value) corresponding to the word string. As described above, the text and the word string are equivalent. That is, the trained model 110 inputs a text and the transmission time of the text, and outputs a delay time of the transmission time.

[0038] Next, learning of the model 110 will be described with reference to Fig. 4. Fig. 4 shows in the form of a table the norm (loss function), the type of output layer, and learning data (correct answer data, teacher data) at each stage of learning of the model 110. As shown in the figure, when learning the model 110, a learning norm different from the conventional BERT learning norm is used.

[0039] Specifically, in the first stage of learning the model 110, learning is performed using the BERT learning criterion according to the prior art. The output layer in this first stage of learning is a neural network classification layer, which is replaced by the neural network regression layer 114. The learning data used in the first stage of learning is a large amount of unsupervised text data. In the first stage of learning, learning is performed to minimize the criterion of the language model using the same method as BERT.

[0040] In the second stage of learning, the model 110 is trained using a control learning model. The output layer in the second stage of learning is a recurrent layer (neural network recurrent layer 114). That is, since the output layer in the second stage is different from the output layer in the first stage, the output layer of the model is replaced with a recurrent layer (transfer learning). The input data used in the second stage of learning is a large number of sentence pairs (text pairs), and the continuous index of similarity of each sentence pair is taken as the ground truth data (see also FIG. 5).

[0041] In the third stage of learning of the model 110, learning is performed using a regression-type model. The output layer in the third stage of learning is a regression layer (neural network regression layer 114). The output layer in the third stage is the same as the output layer in the second stage, but since the learning data used is different, the internal parameters of the output layer (neural network regression layer 114) are reset to their initial values ​​before the third stage of learning is performed. The input data used in the third stage of learning is sentences (text), and learning is performed using the correct answer data for the delay time (transmission delay time) of each sentence (see also FIG. 7).

[0042] Throughout the above-mentioned first to third stages, the learning of the lower layer, the word embedding layer (BERT-type word embedding layer 112), continues. That is, the internal parameter values ​​of the word embedding layer (BERT-type word embedding layer 112) continue to be updated asymptotically through the first to third stages. Meanwhile, the internal parameter values ​​of the upper layer, the output layer (neural network regression layer 114 and the neural network classification layer in the first stage), are discarded when the learning of the second stage is completed. When the learning of the third stage is completed, the internal parameter values ​​of the output layer (neural network regression layer 114) are also retained and used for inference (estimation of the transmission delay time corresponding to an unknown input sentence).

[0043] In order to execute the learning from the first stage to the third stage, the control unit 101 controls the delay estimation device 1 as follows.

[0044] That is, the control unit 101 controls the BERT-type word embedding layer 112 to perform unsupervised learning using a sentence as an input as the first stage of learning for the model 110. The control unit 101 controls the BERT-type word embedding layer 112 to maintain the values ​​of the internal parameters resulting from the first stage of learning.

[0045] Furthermore, the control unit 101 controls the BERT word embedding layer 112 and the neural network regression layer 114 to perform learning in a second stage of learning of the model 110, in which a sentence pair is used as an input and the similarity between two sentences included in the sentence pair is given as ground truth data. The control unit 101 controls the BERT word embedding layer 112 to maintain the values ​​of the internal parameters resulting from the second stage of learning. On the other hand, the control unit 101 controls the neural network regression layer 114 to discard (initialize) the values ​​of the internal parameters of the neural network classification layer in the first stage, which are results of the second stage of learning.

[0046] Furthermore, the control unit 101 controls the BERT-type word embedding layer 112 and the neural network regression layer 114 to perform learning in which a sentence is used as an input and a delay time corresponding to the sentence is given as a correct answer as a third stage of learning for the model 110. The control unit 101 then controls the BERT-type word embedding layer 112 to maintain the values ​​of its internal parameters in the third stage, and controls the neural network regression layer 114 to maintain the values ​​of its internal parameters in the third stage.

[0047] FIG. 5 is a schematic diagram showing the flow of the second-stage learning process in this embodiment. It is assumed that the first-stage learning has been completed before the second-stage learning is started. That is, at the time of starting the second-stage learning, the values ​​of the internal parameters of the BERT-type word embedding layer 112 have been adjusted by the first-stage learning. In the second-stage learning, as described above, a sentence pair is used as an input to the model. In FIG. 5, sentence A and sentence B constitute one sentence pair. A word string based on sentence A and a word string based on sentence B are input to the BERT-type word embedding layer 112, and processed by the BERT-type word embedding layer 112, the pooling layer 113, and the neural network regression layer 114. Note that the processing in this model is performed based on the values ​​of the internal parameters at that time. The processing related to sentence A and the processing related to sentence B may be performed sequentially or simultaneously in parallel. The neural network regression layer 114 outputs an output value based on sentence A and an output value based on sentence B. Each output value from the neural network regression layer 114 may be a vector. These output values ​​from the neural network regression layer 114 are passed to a similarity calculation unit 121. The similarity calculation unit 121 calculates a similarity between an output value from the neural network regression layer 114 based on sentence A and an output value from the neural network regression layer 114 based on sentence B. The similarity calculation unit 121 calculates, for example, a cosine similarity between the two vectors.

[0048] In the second stage of learning, the internal parameters of the neural network (the BERT-type word embedding layer 112 and the neural network regression layer 114) are updated using the backpropagation method based on the difference (loss) between the similarity calculated by the similarity calculation unit 121 based on sentences A and B and the correct similarity given as learning data.

[0049] In the second stage of learning, the above process is repeated for each of a large number of sentence pairs, and the internal parameter values ​​are updated (adjusted). By repeating the process for a large number of sentence pairs, it is expected that the internal parameter values ​​will be asymptotically optimized.

[0050] FIG. 6 is a schematic diagram showing an example of learning data used in the second-stage learning. As shown in the figure, the learning data used in the second-stage learning is a set of pairs of sentence pairs (pairs of sentences A and B) and their similarities (similarity between sentences A and B, correct answer data). Here, the similarity is a value between 0.0 and 1.0, and the larger the value, the higher (more similar) the similarity between sentences A and B is. Although the figure shows only two rows (two cases) of learning data, the number of learning data used in the second-stage learning is arbitrary.

[0051] FIG. 7 is a schematic diagram showing the flow of the third stage learning process in this embodiment. It is assumed that the first and second stages of learning are completed before the third stage learning is started. That is, at the time of starting the third stage learning, as a result of the first and second stages of learning, the values ​​of the internal parameters of the BERT word embedding layer 112 have been adjusted. However, when starting the third stage learning, the values ​​of the internal parameters of the neural network recurrent layer 114 are initialized. That is, the values ​​of the internal parameters of the neural network recurrent layer 114 adjusted by the second stage learning are discarded when starting the third stage learning.

[0052] In the third stage of learning, one sentence is used as input to the model. A word string based on this input sentence is input to the BERT word embedding layer 112, and processed by the BERT word embedding layer 112, the pooling layer 113, and the neural network regression layer 114. The processing in this model is performed based on the values ​​of the internal parameters at that time. The neural network regression layer 114 outputs an output value based on the input sentence. This output value corresponds to the delay time corresponding to the input sentence. In the third stage of learning, the internal parameters of the neural network (the BERT word embedding layer 112 and the neural network regression layer 114) are updated using the backpropagation method based on the difference (loss) between the value output by the neural network regression layer 114 (corresponding to the estimated value of the delay time based on the internal parameter values ​​at that time) and the correct delay time.

[0053] In the third stage of learning, the above process is repeated for each of the many input sentences, and the values ​​of the internal parameters are updated (adjusted). By repeating the process for many sentences, it is expected that the values ​​of the internal parameters will be asymptotically optimized.

[0054] The results of the third stage of training are not discarded, i.e. the internal parameters of the trained neural network are kept as they are and used for processing in the estimation mode.

[0055] Model 110, which has completed the first to third stages of learning, is expected to estimate and output a delay time corresponding to an input sentence (word string). In other words, delay estimation device 1 estimates a transmission delay time corresponding to an input sentence (viewer generated text).

[0056] The delay time estimated by the delay estimation device 1 may be used as a feature to perform clustering processing. In addition, the estimated delay time may be used to perform processing to correct the sending time of the text (processing to subtract the delay time).

[0057] [Second embodiment] Next, a second embodiment of the present invention will be described. The second embodiment is a clustering processing device configured using the delay estimation device 1 described in the first embodiment. Note that the following description may omit the matters already described in the previous embodiment. Here, the description will focus on matters unique to this embodiment.

[0058] 8 is a block diagram showing a schematic functional configuration of the clustering processing device according to this embodiment. As shown in the figure, the clustering processing device 2 includes a delay estimation device 1, a transmitted text set supply unit 211, and a clustering processing unit 213.

[0059] The originating text set supplying unit 211 supplies a set of originating texts (viewer-generated texts) to be clustered to the delay estimation device 1. The viewer-generated texts are associated with information on the originating time.

[0060] The delay estimation device 1 has the functions described in the first embodiment. That is, the delay estimation device 1 estimates and outputs a delay time corresponding to input data. In this embodiment, the delay estimation device 1 passes the distributed representation obtained corresponding to the input text and the delay time, which is the estimated value, to the clustering processing unit 213.

[0061] The clustering processing unit 213 performs clustering processing based on the distributed representation (text feature), the sending time, and the delay time (time feature) received from the delay estimation device 1. As a result of the clustering processing, the clustering processing unit 213 assigns cluster identification information to each item of the input data (see also FIG. 11).

[0062] That is, the clustering processing unit 213 performs clustering processing based on the distributed representation output from the BERT-type word embedding layer 112 of the delay estimation device 1 based on the input text, the sending time of the text, and the delay time of the sending time output from the neural network regression layer 114 of the delay estimation device 1 based on the text. The clustering processing unit 213 can use, for example, the k-means method as a clustering method.

[0063] The clustering processing device may be implemented in the following modified example: In this modified example, a correction unit corrects the transmission time.

[0064] 9 is a block diagram showing a schematic functional configuration of a clustering processing device according to a modified example of this embodiment. As shown in the figure, the clustering processing device 3 includes a delay estimation device 1, a transmitted text set supply unit 311, a correction unit, and a clustering processing unit 313.

[0065] The originating text set providing unit 311 provides a set of originating texts (viewer generated texts) to be clustered. The viewer generated texts are associated with information on the originating time.

[0066] The delay estimation device 1 has the functions described in the first embodiment. That is, the delay estimation device 1 estimates and outputs a delay time corresponding to input data. In this modification, the delay estimation device 1 passes a distributed representation corresponding to the input text to the clustering processing unit 313. In addition, the delay estimation device 1 passes information on the transmission time of the input text and information on the delay time, which is the estimation result, to the correction unit 312.

[0067] The correction unit 312 outputs the corrected sending time by correcting the text sending time passed from the delay estimation device 1 based on the delay time of the sending time passed from the delay estimation device 1. Specifically, the correction unit 312 calculates the corrected sending time by subtracting the delay time (estimated value) from the sending time before correction.

[0068] The clustering processing unit 313 performs clustering processing based on the distributed representation (distributed representation corresponding to the original text) passed from the delay estimation device 1 and the corrected sending time passed from the correction unit 312. The corrected sending time is calculated based on the information of the sending time before correction and the delay time (estimated value). In other words, the clustering processing unit 313 also performs clustering processing based on the distributed representation output from the BERT-type word embedding layer 112 of the delay estimation device 1 based on the input text, the sending time of the text, and the delay time (estimated value) of the sending time output from the neural network regression layer 114 of the delay estimation device 1 based on the text. The clustering processing unit 313 can use, for example, the k-means method as a clustering method.

[0069] FIG. 10 is a schematic diagram for explaining the clustering process performed by the clustering processing device 2 or 3 according to this embodiment (including the modified example).

[0070] [A] in FIG. 10 is a graph showing the relationship between the broadcast time (distribution time) of content and the time transition of the frequency (density) of occurrence of viewer generated text. Here, pairs of individual viewer generated texts occurring in relation to the broadcast time of content and information on their sending time are the targets of clustering processing. Note that in the clustering processing of this embodiment (including modifications), the delay times of individual viewer generated texts estimated by delay estimation device 1 are also processed. In other words, in the clustering processing of this embodiment (including modifications), the target feature space is the direct sum space of text features and time features. That is, it is as shown in the following formula (1).

[0071]

number

[0072] In formula (1), F is a feature space to be subjected to the clustering process. text is a text feature. Also, F time is a time feature. The operator on the right side of equation (1) represents an operation to take the direct sum of the feature space and the feature space.

[0073] [B] in FIG. 10 shows a schematic diagram of the result of clustering individual viewer-generated texts in feature space F. In the illustrated example, as a result of the clustering process, a large number of viewer-generated texts are classified into three clusters. In the figure, each circle corresponds to one viewer-generated text. These circles are shaded in different clusters by hatching patterns. Each cluster corresponds to a scene in the content on which the text is based. As described above, in this embodiment, clustering is performed using a feature that combines text features and time features.

[0074] FIG. 11 is a schematic diagram showing an example of the result of the clustering process according to this embodiment (including the modified example). The input data on which the clustering process is based is the data described with reference to FIG. 3 (first embodiment). In the data of the clustering process result in FIG. 11, a cluster number is assigned to each line as output data. In this example, the cluster numbers are 1, 2, and 3. In this example, the text in the first and second lines is classified into cluster 1. The text in the third line is classified into cluster 2. The data in the fourth line is classified into cluster 3. Note that in practice, the number of clusters is not limited to 3 and is arbitrary.

[0075] FIG. 12 is a block diagram showing an example of the internal configuration of each device in the first embodiment and the second embodiment (including modified examples). Each device (delay estimation device 1, clustering processing devices 2 and 3) can be realized using a computer. As shown in the figure, the computer is configured to include a central processing unit 901, a RAM 902, an input / output port 903, input / output devices 904 and 905, etc., and a bus 906. The computer itself can be realized using existing technology. The central processing unit 901 executes instructions included in a program read from the RAM 902 or the like. The central processing unit 901 writes data to the RAM 902, reads data from the RAM 902, and performs arithmetic operations and logical operations according to each instruction. The RAM 902 stores data and programs. Each element included in the RAM 902 has an address and can be accessed using the address. Note that RAM is an abbreviation for "random access memory." The input / output port 903 is a port through which the central processing unit 901 exchanges data with an external input / output device or the like. The input / output devices 904 and 905 exchange data with the central processing unit 901 via the input / output port 903. The bus 906 is a common communication path used inside the computer. For example, the central processing unit 901 reads and writes data from and to the RAM 902 via the bus 906. Also, for example, the central processing unit 901 accesses the input / output port 903 via the bus 906.

[0076] In addition, at least some of the functions of the delay estimation device 1, the clustering processing device 2, and the clustering processing device 3 in the above-mentioned embodiment (including the modified example) can be realized by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to realize the function. In addition, the "computer system" here includes hardware such as an OS and peripheral devices. In addition, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, an optical magnetic disk, a ROM, a CD-ROM, a DVD-ROM, a USB memory, and a storage device such as a hard disk built into a computer system. In other words, the "computer-readable recording medium" may be a non-transitory computer-readable recording medium. Furthermore, the "computer-readable recording medium" may include a medium that temporarily and dynamically holds a program, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in that case. In addition, the above-mentioned program may be a program for realizing a part of the above-mentioned function, and may further be a program that can realize the above-mentioned function in combination with a program already recorded in the computer system.

[0077] [Effects of the embodiment] The advantages of each of the above embodiments are as follows.

[0078] By using the delay estimation device 1 of the first embodiment, it is possible to obtain an effect that it is possible to estimate a delay time corresponding to a text.

[0079] By using the clustering processing device 2 of the second embodiment, the delay time estimated by the delay estimation device 1 can be used as one of the feature quantities when clustering texts. This makes it possible to improve the accuracy of clustering.

[0080] [Experimental Results] An experiment was conducted to verify the effects of the above embodiment. In the experiment, viewer-generated text generated in real time by a viewer viewing content was used. The correct answer was a case where the text contents were clustered by a human being according to scenes in the content, and the results when using various models were compared. The results of the experiment are described below.

[0081] FIG. 13 shows a comparison of regression performance by various models. The five models compared are A: word occurrence count by word segmentation, B: distributed representation by Word2Vec, C: distributed representation by BERT, D: distributed representation by BERT with contrast learning, and E: distributed representation by BERT with contrast learning (model of this embodiment). The models of the transmission delay time are support vector regression for A to D, and neural network regression for E. As shown in the figure, the coefficient of determination in E (this embodiment) is 0.358, which is superior to other models. The coefficient of determination indicates the quality of prediction accuracy in regression analysis, and the higher the value, the better the performance. The upper limit of the coefficient of determination is 1.

[0082] FIG. 14 shows a comparison of clustering performance by various models. The models compared are the same five types as those shown in FIG. 13. As shown in the figure, the adjusted Rand index in E (this embodiment) is 0.3529, and a superior result is obtained compared to other models. The feature amount including the delay time element is clustered using the K-means algorithm and compared with the correct answer data. The adjusted Rand index is a method for measuring accuracy in a clustering problem without considering differences in class labels, and is corrected so that the obvious value is 0.

[0083] These experimental results show that the method according to this embodiment is superior to a combination of various conventional models, including deep learning models, in terms of coefficient of determination and clustering performance.

[0084] That is, according to the above embodiment, when analyzing viewer generated text that is thought to have been transmitted during real-time viewing of content, the analysis can be performed more efficiently (automatically) while taking into account the transmission delay time than if it were performed manually. In other words, the analysis productivity for a given cost is improved.

[0085] Although several embodiments of the present invention have been described in detail above with reference to the drawings, the specific configurations are not limited to these embodiments, and designs that do not deviate from the gist of the present invention are also included. [Industrial Applicability]

[0086] The present invention can be used, for example, to analyze text generated according to situations occurring along a time axis (one example is a scene in a content). Also, as one example, the present invention can be used in a process of clustering such text. However, the scope of use of the present invention is not limited to the examples given here. [Explanation of symbols]

[0087] 1 Delay estimator 2,3 Clustering processing device 101 Control section 110 Model 111 Word String Input Section 112 BERT-type word embedding layer 113 Pooling Layer 114 Neural Network Regression Layer 115 Transmission delay time output section 121 Similarity calculation unit 211 Outgoing Text Collection Supply Unit 213 Clustering Processing Unit 311 Outgoing Text Collection Supply Unit 312 Correction Section 313 Clustering Processing Unit 901 Central Processing Unit 902 RAM 903 Input / Output Ports 904,905 Input / Output Devices 906 Bus

Claims

1. A model that inputs a text and the time when the text was sent, and outputs the delay time of the sent time; A delay estimator comprising:

2. the text is related to a scene in the content to be distributed; The delay time is a delay time from the time the scene is distributed to the time the text is sent. The delay estimator according to claim 1 .

3. The model is configured by connecting a BERT word embedding layer, which is an input layer, and a neural network regression layer, which is an output layer. The BERT word embedding layer receives the text and the time of the text's publication, The neural network regression layer inputs the information output from the BERT word embedding layer and outputs the delay time of the transmission time. The delay estimator according to claim 1 .

4. A control unit that controls the learning of the model; Further equipped with The control unit is As a first stage of learning of the model, unsupervised learning is performed on the BERT word embedding layer using a sentence as an input, and values ​​of internal parameters of the BERT word embedding layer in the first stage are maintained; As a second stage of learning of the model, a sentence pair is used as an input, and learning is performed for the BERT word embedding layer and the neural network regression layer, in which the similarity between the two sentences included in the sentence pair is given as correct answer data, and values ​​of internal parameters of the BERT word embedding layer in the second stage are maintained, while values ​​of internal parameters of the neural network regression layer in the second stage are discarded; As a third stage of learning of the model, a learning process is performed for the BERT word embedding layer and the neural network regression layer, in which a sentence is used as an input and a delay time corresponding to the sentence is given as a correct answer, and values ​​of internal parameters of the BERT word embedding layer in the third stage are maintained, and values ​​of internal parameters of the neural network regression layer in the third stage are maintained. Control so that The delay estimator according to claim 3.

5. A delay estimating device according to claim 3; a clustering processing unit that performs clustering processing based on a distributed representation output from a BERT-type word embedding layer of the delay estimation device based on an input text, a sending time of the text, and a delay time of the sending time output from the neural network regression layer of the delay estimation device based on the text; A clustering processing device comprising:

6. a correction unit that corrects the sending time of the text based on a delay time of the sending time, and outputs a corrected sending time; Further equipped with The clustering processing unit performs a clustering process based on the distributed representation and the corrected transmission time. The clustering processing device according to claim 5 .

7. A delay estimator according to any one of claims 1 to 4, A program that makes a computer function as a

8. The clustering processing device according to claim 5 or 6, A program that makes a computer function as a

Images