Information Processing Systems

Abstract

To enable effective evaluation of viewers. [Solution] An information processing system characterized by comprising a viewing history memory unit that stores the viewing history of videos by viewers, a score determination unit that determines the score of a subject based on the viewing history corresponding to a viewer who has achieved a specific result and the viewing history corresponding to a target viewer who is different from the viewer, and an output unit that outputs the score of the subject.

Description

[Technical Field]

[0001] The present invention relates to an information processing system. [Background technology]

[0002] Personalized videos are being provided (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-092517 Summary of the Invention [Problem to be solved by the invention]

[0004] Measuring personalized video viewership is difficult.

[0005] The present invention has been made in view of the above background, and has as its object to provide a technique that can effectively evaluate viewers. [Means for solving the problem]

[0006] The main invention of the present invention for solving the above problem is an information processing system comprising: a viewing history memory unit that stores the viewing history of videos by viewers; a score determination unit that determines the score of a subject based on the viewing history corresponding to a relevant person among the viewers who has achieved a specific result and the viewing history corresponding to a target person who is a viewer other than the relevant person; and an output unit that outputs the score of the subject.

[0007] Other problems and solutions disclosed in this application will be made clear in the section on preferred embodiments of the invention and the drawings. [Effects of the Invention]

[0008] According to the present invention, viewers can be evaluated effectively. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the overall configuration of an information processing system. [Figure 2] FIG. 2 illustrates an example of a hardware configuration of a management server 2. [Figure 3] FIG. 2 illustrates an example of the software configuration of a management server 2. [Figure 4] FIG. 10 is a diagram illustrating the operation of the management server 2. DETAILED DESCRIPTION OF THE INVENTION

[0010] <System Overview> An information processing system according to an embodiment of the present invention will be described below.

[0011] 1 is a diagram showing an example of the overall configuration of an information processing system. The information processing system of this embodiment is configured to include a management server 2. The management server 2 is communicably connected to a user terminal 1 via a communication network. The communication network is, for example, the Internet, and is constructed using a public telephone line network, a mobile phone line network, a wireless communication path, Ethernet (registered trademark), etc.

[0012] The user terminal 1 is a computer operated by a user, and may be, for example, a smartphone, a tablet computer, or a personal computer.

[0013] The management server 2 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

[0014] <Administration Server> FIG. 2 is a diagram illustrating an example of the hardware configuration of the management server 2. Note that the illustrated configuration is an example, and other configurations may also be used. The management server 2 includes a CPU 201, a memory 202, a storage device 203, a communication interface 204, an input device 205, and an output device 206. The storage device 203 stores various data and programs, and is, for example, a hard disk drive, a solid state drive, or a flash memory. The communication interface 204 is an interface for connecting to a communication network, and is, for example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone network, a wireless communication device for wireless communication, or a USB (Universal Serial Bus) connector or an RS232C connector for serial communication. The input device 205 is used to input data, and is, for example, a keyboard, a mouse, a touch panel, a button, a microphone, or the like. The output device 206 is used to output data, and is, for example, a display, a printer, a speaker, or the like. Each functional unit of the management server 2 described below is realized by the CPU 201 reading a program stored in the storage device 203 into the memory 202 and executing it, and each storage unit of the management server 2 is realized as part of the storage area provided by the memory 202 and the storage device 203.

[0015] 3 is a diagram illustrating an example of the software configuration of the management server 2. The management server 2 includes a viewer information storage unit 231, a viewing history storage unit 232, a viewing history acquisition unit 211, a common element identification unit 212, a weight determination unit 213, a score determination unit 214, and an output unit 215.

[0016] <Storage section> The viewer information storage unit 231 manages information about viewers (hereinafter, "viewer information"). The viewer information may include information for identifying the viewer (e.g., viewer ID), viewer attributes (e.g., name, age, gender, email address, residential area, occupation, company name, educational background, annual household income, family composition, hobbies and interests), a match flag, etc. The attributes may be registered based on information entered by the viewer when registering as a member (e.g., name, email address, address, company name), information obtained through a questionnaire survey, information obtained from an external database, etc. The match flag is a flag value indicating whether the viewer has achieved a specific result (e.g., taken a specific action or received a specific evaluation). A specific result is, for example, a conversion. Conversions may include, for example, purchasing a specific product or service, requesting information about a specific product or service, or a job seeker being hired by a recruiter (receiving a job offer).

[0017] The viewing history storage unit 232 stores information about the viewer's viewing history of videos (hereinafter, "viewing history information"). The viewing history information may include, for example, information identifying the viewer (e.g., a viewer ID), information identifying the video viewed by the viewer (e.g., a video ID), the viewing start date and time, the viewing end date and time, the viewing duration (which may be the time from the viewing start date and time to the viewing end date and time), the viewing progress rate (the proportion of the viewed portion to the entire video), and information about the viewer's actions on the video (action information). An action on a video includes tapping (or clicking) a predetermined area on the video display screen. In this embodiment, the video is assumed to be a so-called interactive video. That is, when a predetermined area set in a video is tapped, related information related to that area can be displayed on the video, a web page displaying related information can be displayed, or the playback process can be branched to play another video. A tag is set in the predetermined area, and the action information may include information identifying the tapped area, or a tag assigned to the tapped area and the date and time the area was tapped. Furthermore, information identifying the displayed related information may be included in the action information. Multiple actions may be performed during viewing, and the viewing history may include multiple action information. Note that actions are not limited to taps. For example, actions may include liking a video, posting a comment on a video, sharing a video on social media, rewinding a video, fast-forwarding a video, changing the playback speed of a video, or bookmarking a video. Information related to these actions (information identifying the action and the date and time the action was performed) may be included in the viewing history as action information. This action information may be used by the common element identification unit 212 to identify a common element among the relevant person, in order to identify viewers with a specific action pattern. Furthermore, the score determination unit 214 may be used to highly evaluate subjects who have performed actions similar to the relevant person, in order to determine the subject's score.

[0018] The viewing history storage unit 232 may also store aggregated information about viewing by each viewer. The aggregated information may include information identifying the viewer, the viewer's total viewing time (the total viewing time of all videos), the total viewing time in a predetermined period (day, week, month, etc.), the distribution of viewed videos by genre, the distribution of viewed videos by length, the number of times each action was performed, etc.

[0019] The viewer's total viewing time is the cumulative total time a viewer has watched all videos, and can be used as an indicator of the viewer's total platform usage. The total viewing time can be used by the common element identification unit 212 to identify common elements among the persons and to identify patterns among those who have a high viewing time. The score determination unit 214 can also be used to evaluate subjects who have a similar total viewing time to the person and to determine the score of the subject.

[0020] The total viewing time for a predetermined period is an indicator of the concentration of a viewer's video viewing activity within a specific period, such as daily, weekly, or monthly. This information can be used by the common element identification unit 212 to identify common elements among the relevant persons, in order to identify the relevant person's pattern of concentrated viewing during a specific period. Furthermore, when the score determination unit 214 determines the subject's score, this information can be used to evaluate subjects who have a similar viewing pattern to the relevant person. For example, if the relevant person tends to concentrate their viewing on weekends, the score of subjects who have a similar pattern can be evaluated highly.

[0021] The viewing history information also includes the viewing time for each video for each viewer. The viewing time for each video is the cumulative time a viewer has watched a particular video, and is recorded as the total time across multiple viewing sessions. When the score determination unit 214 determines a score for a subject, if the subject has watched a particular video for a long time, the viewing time for each video can be used to give a higher score to subjects who have similarly watched that video for a long time.

[0022] The information for identifying the viewed videos may include IDs and metadata of each video viewed by the viewer. This information may be used by the common element identification unit 212 to identify a specific video or group of videos commonly viewed by the relevant person when identifying common elements of the relevant person. Furthermore, this information may be used by the score determination unit 214 to highly evaluate subjects who watch similar videos to the relevant person when determining a score for the subject. For example, it is possible to identify a pattern of a relevant person who purchases a specific product after watching a video about that product, and to highly score subjects who watch similar videos as potential purchasers.

[0023] The viewing history storage unit 232 may also store aggregated information regarding viewing time for each viewer and each video. The viewing time for each video is information indicating the amount of time a viewer spent viewing each specific video. For example, if viewer A spent a total of 30 minutes viewing video X, 45 minutes viewing video Y, and 15 minutes viewing video Z, the following information may be stored as viewer A's viewing time for each video: video X: 30 minutes, video Y: 45 minutes, and video Z: 15 minutes. The viewing time for each video may be the cumulative time when the same video is viewed multiple times, or may be recorded as a history of viewing time for each viewing session. The viewing time for each video may also include viewing states (playing, paused, fast-forwarding, etc.) for each elapsed time since the start of viewing, and viewing time for each specific segment within the video (e.g., the beginning, middle, or end). The viewing time information for each video is an important indicator for analyzing a viewer's level of interest and concentration in a specific video and can be used as a factor when the score determination unit 214 determines a score.

[0024] <Functional section> The viewing history acquisition unit 211 acquires information about video viewing by multiple viewers. The viewing history acquisition unit 211 can collect viewing data from various sources. The viewing history acquisition unit 211 collects raw data about viewers' video viewing activities.

[0025] The viewing history acquisition unit 211 can acquire a viewing log by, for example, calling an API of a video distribution platform. The viewing history acquisition unit 211 can collect event data (playback start, pause, completion, etc.) from a video player executed on the user terminal 1. The viewing history acquisition unit 211 can collect access logs from mobile applications and websites, for example. The viewing history acquisition unit 211 can collect external viewing data by coordinating with a third-party system, for example.

[0026] The viewing history acquisition unit 211 can also convert the collected raw data into an analyzable format. For example, the viewing history acquisition unit 211 can structure the raw viewing data, remove noise, and generate a consistent data set. For example, the viewing history acquisition unit 211 can perform data cleansing (processing incomplete records and removing duplicate data), data normalization (unifying data formats from different sources), session identification (identifying and classifying continuous viewing), and the like. Furthermore, the viewing history acquisition unit 211 can, for example, calculate viewing time and aggregate pure viewing time by distinguishing between pure viewing time and background playback. For example, the viewing history acquisition unit 211 can link viewing sessions and related actions. For example, the viewing history acquisition unit 211 can detect and delete temporary outliers. For example, the viewing history acquisition unit 211 may perform anonymization processing of viewer IDs.

[0027] The viewing history acquisition unit 211 can register viewing history information including the acquired viewing data in the viewing history storage unit 232. The viewing history acquisition unit 211 can register in the viewing history storage unit 232 aggregated data aggregated based on the processed data set.

[0028] The common element identification unit 212 identifies common elements in the viewing histories of multiple viewers who have achieved specific results in the past (hereinafter, "pertinent persons"). The common elements can be elements of the viewing histories (including aggregate information based on the viewing histories; the same applies hereinafter) that are common to the pertinent persons.

[0029] The common element identification unit 212 can identify, as common elements, elements common to multiple relevant persons in the viewing history corresponding to the relevant person, such as videos viewed by the relevant person, types of videos, viewing time, viewing time zone, viewing frequency, whether or not an action was taken, and the content of the action. Note that a combination of multiple elements, or a result obtained by applying one or multiple elements to a predetermined function, may be adopted as the common element.

[0030] The common element identification unit 212 can identify the common element using a large language model (LLM). For example, the common element identification unit 212 can cause the LLM to identify the common element by providing the LLM with a prompt including the viewing history corresponding to the relevant person and an instruction to create a common element of the viewing history.

[0031] Specific examples of prompts that the common element identification unit 212 gives to the LLM include the following. "The following data is the video viewing history of viewers who achieved a specific result (such as purchasing a product, requesting information, or receiving a job offer). From this viewing history, extract common elements among viewers who achieved that result. Possible common elements include (1) whether or not they watched a specific video ID, (2) the percentage of videos in a specific genre they watched, (3) viewing time patterns, (4) trends in viewing time zones, and (5) whether or not they performed a specific action. Output the extracted common elements in JSON format. Include the element name and description for each common element. [Viewing history data] {Viewer ID: 001, Viewed video: [Video ID: A001, Viewing time: 15 minutes, Viewing time: 8 PM, Action: Tap the tag "More information"], [Video ID: B002, Viewing time: 8 minutes, Viewing time: 9 PM, Action: None], ...} {Viewer ID: 002, Viewed video: [Video ID: A001, Viewing time: 12 minutes, Viewing time: 7 PM, Action: Tap tag "More information"], [Video ID: C003, Viewing time: 20 minutes, Viewing time: 10 PM, Action: Like], ...} ..."

[0032] The above prompts are just examples and can be modified as appropriate depending on the format of the viewing history data and the purpose of the analysis. For example, it is possible to create prompts that are specialized for a particular industry or application. The prompts can also include specific instructions on how to extract common elements (e.g., "Please extract elements that are common to at least 80% of the applicable users") or detailed specifications regarding the output format (e.g., "Please output the top 10 common elements in order of importance").

[0033] The weight determination unit 213 determines the weight of the common element. The weight determination unit 213 can determine the weight so as to highly evaluate viewers with a viewing pattern similar to that of the relevant person. Note that the weight determination unit 213 may be omitted and the weight of all common elements may be set to 1.

[0034] The weight determination unit 213 can, for example, provide the LLM with a prompt including the viewing history, the identified common elements, and instructions to weight the common elements, causing the LLM to create weights for the common elements.

[0035] Specific examples of prompts that the weight determination unit 213 gives to the LLM include the following. The data below shows the video viewing history of viewers who achieved a specific result, along with common factors extracted from it. Please assign weights between 0 and 1 to these common factors based on their correlation with the result and their importance. A higher weight indicates that the factor had a stronger influence on the result. Please also briefly explain the reason for the weighting. [Viewing history data] {Viewer ID: 001, Viewed video: [Video ID: A001, Viewing time: 15 minutes, Viewing time: 8 PM, Action: Tap the tag "More information"], [Video ID: B002, Viewing time: 8 minutes, Viewing time: 9 PM, Action: None], ...} {Viewer ID: 002, Viewed video: [Video ID: A001, Viewing time: 12 minutes, Viewing time: 7 PM, Action: Tap tag "More information"], [Video ID: C003, Viewing time: 20 minutes, Viewing time: 10 PM, Action: Like], ...} ... Extracted Common Elements 1. Watch video A001 2. Tap the "More Information" tag 3. Viewing at night (7 PM - 10 PM) 4. Viewing time of 10 minutes or more ... The weighting results should be output in JSON format as follows: { "Common Elements": [ {"Element name": "Element 1", "Weight": 0.X, "Reason": "Explanation of reason"}, {"Element name": "Element 2", "Weight": 0.Y, "Reason": "Explanation of reason"}, ... ] }"

[0036] It should be noted that the weight determination unit 213 may also have the function of the common element identification unit 212. For example, the weight determination unit 213 can identify the viewing history corresponding to the relevant person and the common elements of the viewing history in order to calculate a score indicating the likelihood of producing a result, and cause the LLM to identify the common elements and create weights for the common elements by providing the LLM with a prompt including an instruction to weight the common elements.

[0037] Specific examples of prompts that the weight determination unit 213 gives to the LLM to simultaneously identify and weight common elements are as follows: "The data below is the video viewing history of viewers who achieved a specific result (such as purchasing a product, requesting information, or receiving a job offer). From these viewing histories, (1) extract common elements among the viewers who achieved that result, and (2) assign a weight between 0 and 1 based on the degree to which each common element influenced the result. These common elements and weights will be used to calculate a score indicating the likelihood that a new viewer will achieve a similar result. [Viewing history data] {Viewer ID: 001, Viewed video: [Video ID: A001, Viewing time: 15 minutes, Viewing time: 8 PM, Action: Tap the tag "More information"], [Video ID: B002, Viewing time: 8 minutes, Viewing time: 9 PM, Action: None], ...} {Viewer ID: 002, Viewed video: [Video ID: A001, Viewing time: 12 minutes, Viewing time: 7 PM, Action: Tap tag "More information"], [Video ID: C003, Viewing time: 20 minutes, Viewing time: 10 PM, Action: Like], ...} ... Common factors could be watching a particular video, taking a particular action, viewing time patterns, viewing time trends, etc. For each common factor, analyze how much it influences your results and determine its weight. The results should be output in JSON format as follows: { "Common elements and weights": {"Element name": "Element 1", "Element description": "Description", "Weight": 0.X, "Weighting reason": "Explanation of reason"}, {"Element name": "Element 2", "Element description": "Description", "Weight": 0.Y, "Weighting reason": "Explanation of reason"}, ... ] }"

[0038] These example prompts provide a basic structure, and can be customized to reflect the specific content of the viewing history data and the purpose of the analysis. Analysis constraints (e.g., "Extract only elements that are seen by at least 70% of the participants") and instructions that take into account specific industry knowledge (e.g., "For recruitment videos, please consider differences in viewing patterns by job type") can also be added to the prompts. Furthermore, the level of detail and specificity of the prompts can be adjusted to reflect the characteristics and capabilities of the LLM.

[0039] The score determination unit 214 determines a score for a viewer to be analyzed (hereinafter, "subject") based on the viewing history of the subject. The score can be determined so that the higher the probability that the subject will achieve a specific result, the higher the score.

[0040] The score determination unit 214 can determine the score according to the number of common elements in the viewing history of the subject. For example, the number of common elements included in the viewing history of the subject can be used as the score.

[0041] The score determination unit 214 can calculate the score by multiplying the common elements included in the viewing history of the subject by the weight determined by the weight determination unit 213.

[0042] The score determination unit 214 can calculate the score using, for example, the following formula: Score = Σ(Wi × Ii) Here, Wi is the weight for common element i, and Ii is an index that is 1 if common element i is included in the subject's viewing history and 0 if it is not. For example, if the common elements identified are "watching video A for more than 5 minutes" (weight = 3), "tapping on product information in video B" (weight = 5), and "watching videos more than 3 times a week" (weight = 2), and the subject satisfies all of these common elements, the score will be 3 + 5 + 2 = 10. On the other hand, if the subject only satisfies "watching video A for more than 5 minutes" and "watching videos more than 3 times a week," the score will be 3 + 0 + 2 = 5.

[0043] The score determination unit 214 can also give partial scores depending on the degree to which the common elements are satisfied. For example, the following formula can be used: Score = Σ(Wi × Fi) Here, Fi is a function that expresses the degree to which common element i is satisfied with a value between 0 and 1. For example, if the common element is "watching video A for 10 minutes or more," Fi = 1 if the target person watches video A for 15 minutes, Fi = 0.5 if they watch it for 5 minutes, and Fi = 0 if they do not watch it.

[0044] The score determination unit 214 can also calculate the score taking into account the correlation between the common elements. For example, the following formula can be used: Score=Σ(Wi × Ii) + Σ(Cjk × Ij × Ik) Here, Cjk is the correlation coefficient for the combination of common elements j and k. For example, an additional score can be given when two common elements, "watching video A" and "watching video B," are simultaneously satisfied.

[0045] The score determination unit 214 can also normalize the score to a predetermined range, such as from 0 to 100. For example, the following formula can be used: Normalized Score = 100 × Score / Max_Score where Max_Score is the theoretically possible maximum score (the score when all common elements are filled with their maximum value).

[0046] The score determination unit 214 can also determine a score based on the degree of match between a video corresponding to the viewing history of the relevant person and a video corresponding to the viewing history of the target person. The degree of match is, for example, an index representing the similarity between a set of videos viewed by the relevant person and a set of videos viewed by the target person. The score determination unit 214 can use, for example, a Jaccard coefficient obtained by dividing the number of elements in the intersection of the set of videos viewed by the relevant person and the set of videos viewed by the target person by the number of elements in the union of the set of videos viewed by the relevant person and the set of videos viewed by the target person as the degree of match. The score determination unit 214 can also use a value obtained by dividing the number of elements in the intersection of the set of videos viewed by the relevant person and the set of videos viewed by the target person by the number of elements in the set of videos viewed by the relevant person as the degree of match. Furthermore, the score determination unit 214 can calculate the degree of match between the videos viewed by the relevant person and the videos viewed by the target person based on the similarity of video metadata (e.g., genre, tags, cast, producer, etc.). For example, the similarity between the distribution of genres of videos viewed by the relevant person and the target person can be calculated using an index such as cosine similarity or Euclidean distance, and used as the degree of match. The score determination unit 214 can also use the similarity between the viewing time patterns (e.g., viewing time distribution by time period) of the relevant person and the target person as the degree of match. Furthermore, the score determination unit 214 can also use the similarity between the action patterns of the relevant person and the target person with respect to videos (e.g., frequency of tapping on an area with a specific tag) as the degree of match. The score determination unit 214 may use one of these degrees of match, or may use a combination of multiple degrees of match. When multiple degrees of match are combined, a weighted average can be calculated by assigning weights to the degrees of match.

[0047] When determining the score based on the degree of match, the score determination unit 214 can use the degree of match directly as the score, for example. Alternatively, the score determination unit 214 can obtain the score by multiplying the degree of match by a predetermined coefficient. Furthermore, the score determination unit 214 can obtain the score by applying a predetermined function (e.g., a sigmoid function or a step function) to the degree of match. For example, a high score can be assigned when the degree of match exceeds a predetermined threshold, and a low score can be assigned when the degree of match is equal to or less than the threshold. Alternatively, the score determination unit 214 can calculate the degree of match with the target person for each of multiple relevant persons, and use statistics such as the average, maximum, or median of the calculated scores as the score.

[0048] The output unit 215 outputs the subject's score.

[0049] <Operation> FIG. 4 is a diagram illustrating the operation of the management server 2.

[0050] The management server 2 uses LLM to identify common elements in the viewing histories of multiple relevant individuals (S301), determines weights for the common elements (S302), and calculates the target individual's score from the common elements included in the target individual's viewing history (S303).

[0051] As described above, the information processing system of this embodiment can calculate the viewer's score. Furthermore, the information processing system of this embodiment can highly evaluate viewers whose viewing history includes elements that are the same as elements common to the viewing history of a person who has achieved results in the past. In other words, it can highly evaluate viewers whose viewing pattern is the same as that of the person. Therefore, it is possible to effectively evaluate viewers.

[0052] Although the present embodiment has been described above, the above embodiment is intended to facilitate understanding of the present invention and is not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit thereof, and equivalents thereof are also included in the present invention.

[0053] For example, the processing by each of the functional units of the management server 2 described above may be performed by any of the functional units. Also, a different functional unit that performs part of the processing by each of the functional units described above may be added. Also, the functional units of the management server 2 may be distributed across multiple computers.

[0054] Furthermore, the information stored in each storage unit of the management server 2 may be stored in any of the storage units. That is, the information stored in the above-mentioned multiple storage units may be stored in one storage unit, or part of the information stored in one of the above-mentioned storage units may be stored in another storage unit.

[0055] <Variation 1> The method of calculating the score is different in Modification 1. For example, the management server 2 according to Modification 1 can include a machine learning model learning unit and a feature extraction unit.

[0056] The machine learning model learning unit uses the viewing history of the relevant person as training data to train a machine learning model. The machine learning model learning unit identifies viewers for whom a relevant person flag is set from the viewer information storage unit 231, and acquires the viewing history of the relevant person from the viewing history storage unit 232. The machine learning model learning unit extracts features from the acquired viewing history, and trains a machine learning model for predicting whether or not the person is a relevant person.

[0057] The machine learning model training section can use various types of machine learning models, such as random forests, gradient boosting (XGBoost, LightGBM, CatBoost, etc.), support vector machines, and neural networks (multilayer perceptrons, convolutional neural networks, recurrent neural networks, etc.).

[0058] The machine learning model learning unit can extract various features from the viewing history and use them for learning. For example, the following features can be extracted: Total number of videos watched Total viewing time · Viewing time or number of views by video genre · Viewing time or number of views by video length Viewing time or number of views by time of day Viewing time or number of views by day of the week Completion rate of videos watched (percentage of videos watched to the end) -Number of actions taken while watching the video Binary features indicating whether a specific video or a video of a specific genre was watched - Average, maximum and minimum continuous viewing time Average, maximum, and minimum viewing intervals Viewing frequency (daily, weekly, monthly) These elements are used to extract common elements in the above-described embodiment.

[0059] The machine learning model training unit uses the extracted features and relevant flags (correct labels) to train the machine learning model. During training, cross-validation can be performed to prevent overfitting, and hyperparameter tuning can be performed.

[0060] The feature extraction unit extracts features from the viewing history of the target person. The feature extraction unit extracts the same features from the viewing history of the target person as the features used by the machine learning model learning unit during learning. The feature extraction unit acquires the viewing history of the target person from the viewing history storage unit 232 and calculates features from the acquired viewing history.

[0061] The score determination unit 214 can input the feature amount extracted by the feature extraction unit into a machine learning model to predict the score of the subject. The score determination unit 214 can obtain the probability that the subject will be a match as an output of the machine learning model, and can use this probability as a score.

[0062] For example, when a decision tree-based model such as random forest or gradient boosting is used, the score determination unit 214 can use the probability value (ranging from 0 to 1) obtained from the model as the score as is. Also, when a neural network is used, a probability value ranging from 0 to 1 can be obtained by applying a sigmoid function to the output layer.

[0063] If necessary, the score determination unit 214 can also convert the obtained probability values ​​to any scale, such as a scale from 0 to 100, or categorize the scores by setting a specific threshold.

[0064] The machine learning model training unit can retrain the model periodically or whenever data on a new person is added.

[0065] The machine learning model training unit can also perform ensemble learning, which trains multiple machine learning models and combines their prediction results. For example, by training different types of models such as random forests, gradient boosting, and neural networks, and then taking the average or majority vote of their prediction results, more stable prediction results can be obtained.

[0066] Furthermore, by analyzing the importance of features, the machine learning model training section can identify which viewing behaviors are strongly associated with the likelihood of becoming a target. For example, random forests and gradient boosting can calculate the importance score for each feature. This information can be used to analyze viewer behavior and develop effective content strategies.

[0067] Furthermore, the score determination unit 214 can generate not only a predicted score but also an explanation for the score (for example, "because many videos of a particular genre are viewed" or "because the viewing completion rate is high"). <Variation 2> In the second modification, time-series information on viewing patterns is utilized to analyze temporal changes in viewing patterns and treat these as common elements.

[0068] The common element identification unit 212 extracts a viewing pattern based on a time series from the viewing history of the relevant person and identifies this as a time series common element. The time series common element includes, for example, a sequence pattern in which a specific video A is viewed followed by a specific video B, a pattern in which a video of a specific genre is viewed followed by another video of a specific genre, or a pattern in which videos of a specific category are viewed at a specific time period.

[0069] The common element identification unit 212 arranges the viewing history data of the relevant person acquired from the viewing history storage unit 232 in chronological order, and can perform the following analysis.

[0070] (1) Sequence mining: The common element identification unit 212 extracts frequently occurring sequence patterns from the viewing history of the relevant person. For example, it identifies a sequential pattern of video IDs (video A → video B → video C) or a sequential pattern of video categories (category X → category Y). The common element identification unit 212 can extract frequently occurring sequence patterns using sequence pattern mining algorithms such as PrefixSpan, SPADE, GSP, and CMSPADE.

[0071] (2) Time Interval Analysis: The common element identification unit 212 analyzes the time interval between video viewings. For example, it identifies a pattern in which video B is viewed within 24 hours of viewing video A, or a pattern in which videos of a specific category are viewed consecutively on the weekend. The common element identification unit 212 uses viewing date and time information to calculate the distribution of viewing intervals and can extract characteristic time interval patterns.

[0072] (3) Viewing Continuity Analysis: The common element identification unit 212 analyzes the viewing continuity for a specific video category or a specific content series. For example, it identifies a pattern of continuing to watch videos from a specific series multiple times within a certain period of time. The common element identification unit 212 can quantify the continuity and regularity of viewing and extract persistent viewing patterns.

[0073] (4) Viewing transition probability analysis: The common element identification unit 212 calculates the transition probability from one video to another. For example, it identifies a pattern in which there is a high probability of viewing video B after viewing video A. The common element identification unit 212 can construct a transition probability matrix between videos using a probability model such as a Markov model.

[0074] The common element identification unit 212 identifies time-series common elements common to the relevant individuals from the analysis results. The time-series common elements are expressed in the following format, for example. Sequence pattern: {Video A, Video B, Video C} (watch in this order) Time interval pattern: {Video A, within 24 hours, Video B} (Watch Video A and then Video B within 24 hours) Category transition pattern: {Category X, Category Y, Category Z} (view categories in this order) Viewing time pattern: {Weekday evening, Category X}, {Weekend morning, Category Y} (viewing a specific category at a specific time)

[0075] The common element identification unit 212 can also identify time-series common elements using a large-scale language model (LLM). The common element identification unit 212 provides the LLM with viewing history data of the relevant user organized in chronological order and a prompt including an instruction such as "Please extract characteristic patterns taking into consideration the viewing order and time intervals," thereby causing the LLM to identify time-series common elements.

[0076] The weight determination unit 213 determines weights for the time-series common elements identified by the common element identification unit 212. The weight determination unit 213 can determine weights in consideration of, for example, the following factors. Sequence length: longer sequence patterns are given higher weights Frequency of occurrence of sequences: Patterns that appear frequently among the relevant individuals are given a higher weight. Specificity of time intervals: patterns with distinctive time intervals are given higher weights Proximity to conversion: Patterns that occur immediately before a conversion (specific result) are given a higher weight.

[0077] The weight determination unit 213 can also determine the weights of the time-series common elements using the LLM. The weight determination unit 213 can provide the LLM with a prompt including the viewing history data of the relevant person, the identified time-series common elements, and an instruction such as "Please determine the weights based on the degree of influence that each time-series pattern has on conversion," and cause the LLM to determine the weights.

[0078] The score determination unit 214 determines a score for a subject based on the common time-series elements included in the viewing history of the subject. The score determination unit 214 analyzes the viewing history of the subject in chronological order and detects patterns that match or are similar to the common time-series elements extracted from the subject. The score determination unit 214 can calculate the score for the subject by summing or weighted averaging the weights corresponding to each detected common time-series element.

[0079] The score determination unit 214 can calculate the score using, for example, the following formula: Score = Σ(wi × Mi) Here, wi represents the weight of time-series common element i, and Mi represents the degree of match of time-series common element i in the viewing history of the subject (a value of 1 for a perfect match, or 0 to 1 for a partial match).

[0080] The score determination unit 214 can take the following factors into consideration when calculating the degree of coincidence of the time-series common elements. - Exact Sequence Match: When the viewing order is exactly the same Partial sequence match: When the viewing order is partially the same · Time interval similarity: When the time interval between viewings is similar Category-level match: When the specific videos are different but the viewing order of the categories is the same

[0081] By utilizing the time series information of viewing history, it is possible to analyze not just "which videos were viewed" but also temporal viewing patterns such as "in what order they were viewed" and "at what time intervals." This allows for more precise analysis of viewer behavior and more accurately identifies viewers who are likely to achieve specific results.

[0082] For example, if it is known that there is a strong tendency to purchase a product after watching a product introduction video, then a usage video, and then a user review video, subjects who show a similar viewing pattern can be given a higher score. Also, if it is known that a pattern of watching business-related videos on weekday evenings and product comparison videos on weekends is associated with a specific outcome, subjects who show that pattern can be appropriately evaluated.

[0083] Furthermore, since the time interval between viewings can also be taken into account, it is possible to analyze temporal characteristics such as "watching related videos intensively in a short period of time" or "watching videos of a specific category regularly over a long period of time." This allows for the evaluation of the strength and continuity of a viewer's interest, enabling more accurate scoring.

[0084] <Variation 3> The third modification relates to a method of defining a plurality of relevant person groups and analyzing which group the target person's viewing pattern is most similar to. The management server 2 according to the third modification may further include a relevant person group storage unit and a group similarity calculation unit.

[0085] The relevant person group storage unit stores information about multiple relevant person groups. A relevant person group is a group of relevant people who have achieved different results. Different results include, for example, purchasing different products, different types of conversions, applying for different services, or applying for different jobs. For each group, the relevant person group storage unit stores information that identifies the group (e.g., group ID), information that identifies the result corresponding to the group (e.g., product ID, conversion type ID, etc.), and information that identifies relevant people belonging to the group (e.g., a list of viewer IDs).

[0086] In Modification 3, common element identification unit 212 can identify common elements for each relevant person group. Common element identification unit 212 acquires information about relevant people belonging to each group from the relevant person group storage unit, acquires the viewing history of the relevant people for each group from viewing history storage unit 232, and can identify common elements for each group.

[0087] For example, the common element identification unit 212 can provide the LLM with a prompt for each relevant group, including viewing histories corresponding to the relevant people belonging to the relevant group and an instruction to create a common element of the viewing histories, thereby causing the LLM to identify the common element. At this time, the prompt can include information identifying the result corresponding to the relevant group (e.g., "a group of customers who purchased product A," "a group of customers who subscribed to service B," etc.). This allows the LLM to more appropriately identify common elements related to specific results.

[0088] In Modification 3, the weight determination unit 213 can determine the weight of the common element for each relevant group of users. For example, the weight determination unit 213 can provide a prompt including the viewing history, the identified common element, and an instruction to weight the common element to the LLM for each relevant group of users, to cause the LLM to create the weight of the common element.

[0089] The group similarity calculation unit calculates the similarity between the viewing history of the target person and the common elements of each corresponding group. The group similarity calculation unit can calculate the similarity based on the number of common elements of each corresponding group included in the viewing history of the target person, the total value of the common elements taking weights into account, etc.

[0090] The group similarity calculation unit can calculate the similarity Sg between the viewing history of the target person and the corresponding person group g, for example, using the following formula. Sg = Σ(Wg,i × I(eg,i,subject)) Here, eg,i is the common element i of the relevant group g, Wg,i is the weight of the common element eg,i, and I(eg,i, subject) is an indicator function that is 1 if the common element eg,i is included in the viewing history of the subject, and 0 if it is not included.

[0091] The group similarity calculation unit can also calculate the probability that the subject belongs to each relevant group. For example, the group similarity calculation unit can calculate the probability Pg that the subject belongs to relevant group g using the following formula. Pg = Sg / Σ(Sh) Here, Σ(Sh) is the sum of the similarities Sh for all relevant groups h.

[0092] In Modification 3, the score determination unit 214 can determine the score of the subject based on the similarity and probability calculated by the group similarity calculation unit. For example, the score determination unit 214 can set the similarity corresponding to the relevant group with the highest similarity as the score. Alternatively, the score determination unit 214 can set the score as the sum of values ​​obtained by multiplying the similarity of each relevant group by the importance or value of the result corresponding to the relevant group.

[0093] The output unit 215 can output the similarity between the subject and each relevant group and the probability that the subject belongs to each relevant group, in addition to the subject's score. For example, the output unit 215 can output, for each subject, the relevant groups and their similarities in descending order of similarity, along with the score.

[0094] The method of defining groups of relevant users is not limited to the above-mentioned example. For example, even if the result is the same (e.g., the purchase of the same product), the groups of relevant users can be divided according to the attributes of the relevant users (e.g., age group, gender, region, etc.). The groups of relevant users can also be divided according to the period until the result is achieved (e.g., a group with a short period from watching the video to purchasing the product and a group with a long period). Furthermore, the groups of relevant users can also be divided according to the degree of the result (e.g., the amount of purchase, the number of applications, etc.).

[0095] Furthermore, the method for calculating group similarity is not limited to the above example. For example, common similarity indices such as cosine similarity or Euclidean distance can be used. Furthermore, a machine learning model (e.g., logistic regression, random forest, neural network, etc.) can be used to predict the probability that a subject belongs to each corresponding group.

[0096] Furthermore, the method for identifying common elements of the relevant group is not limited to the above-mentioned example. For example, a viewing pattern characteristic of the relevant group can be extracted using a statistical method (e.g., correlation analysis, principal component analysis, etc.). Furthermore, the viewing history of the relevant group and a non-relevant group can be compared to extract elements characteristic of the relevant group.

[0097] <Variation 4> In the fourth modification, by combining information other than the viewing history, it is possible to perform a more accurate score calculation. The management server 2 according to the fourth modification can include a behavior history storage unit.

[0098] The behavioral history storage unit stores the behavioral history of viewers. The behavioral history may include, for example, information identifying the viewer (viewer ID), website browsing history, application usage history, purchase history, search history, and activity history on social media. Website browsing history may include the URL of the web page viewed, the date and time of viewing, the viewing duration, and the number of views. Application usage history may include the identification information of the application used, the date and time of use, the usage duration, the frequency of use, and the operations performed within the application. Purchase history may include the identification information of purchased products and services, the purchase date and time, the purchase amount, and the purchase location. Search history may include search keywords, the search date and time, and click information for search results. Activity history on social media may include the content of posts, the date and time of posting, interaction information such as likes and comments, and follow / follower information.

[0099] The common element identification unit 212 can combine and analyze the viewing history, attribute information, and behavior history to extract a characteristic pattern of the relevant person. For example, the common element identification unit 212 can input the viewing history, attribute information, and behavior history of the relevant person to identify a characteristic pattern common to the relevant person. In addition to the common elements extracted from the viewing history, the characteristic pattern can include common attribute values ​​in the attribute information and common behavior patterns in the behavior history.

[0100] The common element identification unit 212 can extract feature patterns using a large-scale language model (LLM). The common element identification unit 212 can cause the LLM to extract feature patterns by, for example, providing the LLM with a prompt including the viewing history, attribute information, and behavioral history of the relevant person, as well as an instruction to extract feature patterns from this information. An example of the prompt could be, "Please analyze the following relevant person data (viewing history, attribute information, behavioral history) and extract feature patterns common to the relevant person. The feature patterns should include viewing patterns such as the types of videos viewed, viewing times, and viewing frequency, attribute characteristics such as age group and gender, and behavioral patterns such as website browsing and app usage. Also, please rate the importance of each feature pattern on a scale of 0 to 1."

[0101] The common element identification unit 212 can also extract feature patterns using a machine learning algorithm. For example, it can use data on relevant and non-relevant individuals as training data to train a machine learning model such as random forest, gradient boosting, or neural network, and calculate the importance of each feature. It can also use a clustering algorithm to cluster the data on relevant individuals and extract features from each cluster.

[0102] The common element identification unit 212 provides the extracted feature patterns and the importance (weight) of each feature pattern to the weight determination unit 213. The weight determination unit 213 can determine the weight of the feature patterns extracted from the attribute information and behavior history in addition to the weight of the common elements of the viewing history.

[0103] The score determination unit 214 determines the score of the subject based on the viewing history, attribute information, and behavior history of the subject. The score determination unit 214 can calculate the score based on, for example, the number of feature patterns included in the subject's data and the weight of each feature pattern. The score determination unit 214 can calculate the score using, for example, the following formula: Score = Σ(weight of feature pattern i × presence of feature pattern i in the subject data) Here, the presence of a feature pattern may be a binary value (0 or 1) indicating whether or not the feature pattern exists in the subject's data, or a continuous value (values ​​from 0 to 1) indicating the degree of similarity or agreement with the feature pattern.

[0104] Furthermore, the score determination unit 214 can also calculate a score by weighting and adding sub-scores calculated from the viewing history, attribute information, and behavior history. Score = w1 × viewing history subscore + w2 × attribute information subscore + w3 × behavioral history subscore Here, w1, w2, and w3 are the weights of each subscore, which can be set so that w1 + w2 + w3 = 1. Each subscore is calculated based on the feature patterns extracted from the respective data source.

[0105] In Variation 4, by combining and analyzing not only viewing history but also attribute information and behavioral history, it is possible to perform more accurate score calculations. For example, if viewers of a certain age group or gender share a common pattern of watching specific videos and browsing specific websites, subjects who match this pattern can be highly evaluated. Furthermore, by extracting characteristics that cannot be captured by viewing history alone (for example, interest in a specific product or willingness to purchase) from attribute information and behavioral history, it becomes possible to perform a more multifaceted evaluation.

[0106] Although the example of using both attribute information and behavior history has been described in Modification 4, only one of them may be used. Also, information other than attribute information and behavior history (for example, location information, device information, weather information, season information, etc.) may be combined and analyzed.

[0107] In addition, in the fourth modification, an example of extracting feature patterns using LLM or a machine learning algorithm has been described, but feature patterns may also be extracted using a rule-based approach or a statistical method. For example, features that show a statistically significant difference between relevant and non-relevant individuals may be extracted.

[0108] <Disclosures> The present disclosure also includes the following configurations. [Item 1] a viewing history storage unit that stores a viewing history of a video by a viewer; a score determination unit that determines a score for the target person based on the viewing history corresponding to the relevant person among the viewers who has achieved a specific result and the viewing history corresponding to the target person who is the viewer other than the relevant person; an output unit that outputs the score of the subject; An information processing system comprising: [Item 2] Item 1, an information processing system according to item 1, a weight determination unit that determines common elements and weights of the common elements to be used in calculating the score by providing a prompt to a large-scale language model, the prompt including the viewing history corresponding to the person and an instruction to determine elements of the viewing history and weights for the elements to be used in calculating the score based on the viewing history; the score determination unit determines the score of the subject based on the common elements and the weights related to the viewing histories corresponding to the subject; An information processing system characterized by: [Item 3] Item 1, an information processing system according to item 1, a common element identification unit that provides a prompt including the viewing history corresponding to the relevant person and an instruction to indicate a common element in the viewing history to a large-scale language model to identify the common element; a weight determination unit that determines the common elements and the weights by providing a prompt to the large-scale language model, the prompt including the viewing history, the common elements, and an instruction to determine weights of the common elements; Equipped with the score determination unit determines the score of the subject based on the common elements and the weights related to the viewing histories corresponding to the subject; An information processing system characterized by: [Item 4] Item 1, an information processing system according to item 1, the score determination unit determines the score according to a degree of coincidence between the video corresponding to the viewing history corresponding to the relevant person and the video corresponding to the viewing history corresponding to the target person; An information processing system characterized by: [Item 5] Item 1, an information processing system according to item 1, The viewing history includes, for each of the viewers, at least one of a total viewing time for the videos, a total viewing time for a predetermined period, information identifying the videos viewed, a viewing time for each of the videos, and information identifying actions taken on the videos; An information processing system characterized by: [Explanation of symbols]

[0109] 1. User terminal 2 Management Server

Claims

1. a viewing history storage unit that stores a viewing history of a video by a viewer; a score determination unit that determines a score for the target person based on the viewing history corresponding to the relevant person among the viewers who has achieved a specific result and the viewing history corresponding to the target person who is the viewer other than the relevant person; a weight determination unit that determines common elements and weights of the common elements to be used in calculating the score by providing a prompt to a large-scale language model, the prompt including the viewing history corresponding to the relevant person and instructions for determining elements of the viewing history and weights for the elements to be used in calculating the score based on the viewing history; an output unit that outputs the score of the subject; Equipped with the score determination unit determines the score of the subject based on the common elements and the weights related to the viewing histories corresponding to the subject; An information processing system characterized by:

2. a viewing history storage unit that stores a viewing history of a video by a viewer; a common element identification unit that provides a prompt including the viewing history corresponding to the relevant person and an instruction to indicate a common element in the viewing history to a large-scale language model to identify the common element; a weight determination unit that determines the common elements and the weights by providing a prompt to the large-scale language model, the prompt including the viewing history, the common elements, and an instruction to determine weights of the common elements; a score determination unit that determines a score for the target person based on the viewing history corresponding to the relevant person among the viewers who has achieved a specific result and the viewing history corresponding to the target person who is the viewer other than the relevant person; an output unit that outputs the score of the subject; Equipped with the score determination unit determines the score of the subject based on the common elements and the weights related to the viewing histories corresponding to the subject; An information processing system characterized by:

3. a viewing history storage unit that stores a viewing history of a video by a viewer; a score determination unit that determines a score for the target person according to a degree of coincidence between the viewing history corresponding to the relevant person among the viewers who has achieved a specific result and the viewing history corresponding to the target person who is the viewer other than the relevant person; an output unit that outputs the score of the subject; Equipped with The viewing history includes, for each of the viewers, at least one of the total viewing time for the videos, the total viewing time for a predetermined period, information specifying the videos viewed, or the viewing time for each of the videos, and at least one of tapping or clicking a predetermined area on the display screen of the video, displaying related information related to the area of ​​the video that was tapped or clicked, setting a like for the video, sharing the video on SNS, rewinding the video, fast-forwarding the video, changing the playback speed of the video, and registering the video as a bookmark; An information processing system characterized by:

4. storing a video viewing history of a viewer; determining common elements and weights for the common elements to be used in calculating the score by providing a large-scale language model with prompts including the viewing history corresponding to the person and instructions for determining elements of the viewing history and weights for the elements to be used in calculating a score for the person based on the viewing history; determining a score for the target person based on the viewing history corresponding to the relevant person among the viewers who has achieved a specific result and the viewing history corresponding to the target person who is the viewer different from the relevant person; outputting the subject's score; The computer executes In the step of determining the score, the computer determines the score of the subject based on the common elements and the weights related to the viewing histories corresponding to the subject; An information processing method comprising:

5. storing a video viewing history of a viewer; providing a prompt to a large-scale language model, the prompt including the viewing history corresponding to the person and an instruction to indicate a common element in the viewing history, to identify the common element; providing a prompt to the large-scale language model, the prompt including the viewing history, the common elements, and instructions for determining weights of the common elements, to determine the common elements and the weights; determining a score for the target person based on the viewing history corresponding to the relevant person among the viewers who has achieved a specific result and the viewing history corresponding to the target person who is the viewer different from the relevant person; outputting the subject's score; The computer executes In the step of determining the score, the computer determines the score of the subject based on the common elements and the weights related to the viewing histories corresponding to the subject; An information processing method comprising:

6. storing a video viewing history of a viewer; determining a score for the target person according to a degree of agreement between the viewing history corresponding to the relevant person among the viewers who has achieved a specific result and the viewing history corresponding to the target person who is the viewer other than the relevant person; outputting the subject's score; The computer executes The viewing history includes, for each of the viewers, at least one of the total viewing time for the videos, the total viewing time for a predetermined period, information specifying the videos viewed, or the viewing time for each of the videos, and at least one of tapping or clicking a predetermined area on the display screen of the video, displaying related information related to the area of ​​the video that was tapped or clicked, setting a like for the video, sharing the video on SNS, rewinding the video, fast-forwarding the video, changing the playback speed of the video, and registering the video as a bookmark; An information processing method comprising:

7. storing a video viewing history of a viewer; determining common elements and weights for the common elements to be used in calculating the score by providing a large-scale language model with prompts including the viewing history corresponding to the person and instructions for determining elements of the viewing history and weights for the elements to be used in calculating a score for the person based on the viewing history; determining a score for the target person based on the viewing history corresponding to the relevant person among the viewers who has achieved a specific result and the viewing history corresponding to the target person who is the viewer different from the relevant person; outputting the subject's score; A program for causing a computer to execute the above, In the step of determining the score, causing the computer to determine the score for the subject based on the common elements and the weights associated with the viewing histories corresponding to the subject; A program characterized by.

8. storing a video viewing history of a viewer; providing a prompt to a large-scale language model, the prompt including the viewing history corresponding to the person and an instruction to indicate a common element in the viewing history, to identify the common element; providing a prompt to the large-scale language model, the prompt including the viewing history, the common elements, and instructions for determining weights of the common elements, to determine the common elements and the weights; determining a score for the target person based on the viewing history corresponding to the relevant person among the viewers who has achieved a specific result and the viewing history corresponding to the target person who is the viewer different from the relevant person; outputting the subject's score; A program for causing a computer to execute the above, In the step of determining the score, causing the computer to determine the score for the subject based on the common elements and the weights associated with the viewing histories corresponding to the subject; A program characterized by.

9. storing a video viewing history of a viewer; determining a score for the target person according to a degree of agreement between the viewing history corresponding to the relevant person among the viewers who has achieved a specific result and the viewing history corresponding to the target person who is the viewer other than the relevant person; outputting the subject's score; A program for causing a computer to execute the above, The viewing history includes, for each of the viewers, at least one of the total viewing time for the videos, the total viewing time for a predetermined period, information specifying the videos viewed, or the viewing time for each of the videos, and at least one of tapping or clicking a predetermined area on the display screen of the video, displaying related information related to the area of ​​the video that was tapped or clicked, setting a like for the video, sharing the video on SNS, rewinding the video, fast-forwarding the video, changing the playback speed of the video, and registering the video as a bookmark; A program characterized by.

Images