server
The server system integrates diverse data types and personalized machine learning models to enhance the accuracy and relevance of video distributor searches on live streaming platforms, addressing the inefficiencies of existing search methods.
Patent Information
- Application Number
- JP2023070727
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-24
- Publication Date
- 2024-11-06
- Estimated Expiration
- 2043-04-24
AI Technical Summary
Existing live streaming platforms lack an efficient method to search for video distributors that integrates various data types such as video, audio, broadcaster profiles, and user interactions, leading to inaccurate search results.
A server system utilizing a distributor information holding unit, video data unit, conversion unit, and multiple machine learning models to generate introduction texts with different personalities, enhancing search accuracy by integrating diverse data types and user preferences.
Improves the accuracy and relevance of video distributor searches by generating personalized search results based on user preferences and broadcaster data, enhancing user engagement.
Smart Images

Figure 2024156346000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a server. [Background technology]
[0002] With the development of IT technology, the way information is exchanged has also changed. In the Showa era, one-way information transmission such as newspapers and television was the norm. In the Heisei era, mobile phones and personal computers became widespread and Internet communication speeds improved significantly, leading to the rise of instant two-way communication services such as chat services, and on-demand video distribution services became more popular as storage costs decreased. Now, in the Reiwa era, with the increasing functionality of smartphones and further improvements in network speeds such as 5G, services that enable real-time communication through video, especially live streaming services, are rapidly gaining recognition. Live streaming services are seeing an increase in users, especially young people, as they allow everyone to share the same fun time even when they are far apart.
[0003] In a live distribution platform, a broadcaster can start a live distribution whenever he / she likes, and a viewer can watch a live distribution of a broadcaster that he / she is interested in whenever he / she likes. Patent Document 1 discloses a technology that recommends live streaming content using machine learning. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] Special Publication No. 2020-521207 [Non-patent literature]
[0005] [Non-Patent Document 1] "How to give ChatGPT the personality of King Gilgamesh?", Takayuki Fukatsu, URL: https: / / note.com / fladdict / n / neff2e9d52224 [Non-Patent Document 2] "A demo of an AI responding to the contents of an image in text", ExaWizards, Inc., URL: https: / / techblog.exawizards.com / entry / 2019 / 02 / 15 / 175416 [Non-Patent Document 3] "A summary of methods for measuring similarity between texts", @shimi7o, URL: https: / / qiita.com / shimi7o / items / b3bc64e2fbe1103c7db9 Summary of the Invention [Problem to be solved by the invention]
[0006] Information about live streamers includes a wide variety of data, such as the video data (images, audio) of the live stream, the streamer's profile, data showing follower relationships, comment history, viewing history, tags, etc. If such a wide variety of data could be searched together, the accuracy of streamer searches could be improved.
[0007] The present disclosure has been made in consideration of these problems, and has as its purpose to provide technology that can improve searches for video distributors. [Means for solving the problem]
[0008] One aspect of the present invention relates to a server, which includes: a distributor information storage unit that stores information on distributors of videos distributed on a video distribution platform; a video data storage unit that stores video data including images and audio of videos distributed by the distributors; a conversion unit that converts the video data stored in the video data storage unit into text data; a plurality of machine learning models, each of which receives the text data and the information on the distributor stored in the distributor information storage unit as input and outputs a text introducing the distributor, each of which has a different personality; a text storage unit that stores an ID identifying the distributor and a plurality of texts introducing the distributor output by the plurality of machine learning models in association with each other; a request receiving unit that receives a search request from a user terminal of a user of the video distribution platform via a network; a search unit that executes a search in the text storage unit based on the search request; and a providing unit that provides the user terminal with the results of the search executed by the search unit via the network.
[0009] In addition, any combination of the above components, or mutual substitution of the components or expressions of the present invention between an apparatus, a method, a system, a computer program, a recording medium storing a computer program, etc., are also valid aspects of the present invention. Effect of the Invention
[0010] According to the present invention, searching for video distributors can be improved. [Brief description of the drawings]
[0011] [Figure 1] FIG. 1 is a schematic diagram illustrating a configuration of a live distribution system according to an embodiment of the present disclosure. [Diagram 2] 2 is a block diagram showing the functions and configuration of the user terminal of FIG. 1. [Diagram 3] 2 is a block diagram showing the functions and configuration of the server in FIG. 1. [Figure 4] 4 is a data structure diagram showing an example of a stream DB in FIG. 3. [Diagram 5]4 is a data structure diagram showing an example of a user DB in FIG. 3. [Figure 6] 4 is a data structure diagram showing an example of a gift DB in FIG. 3. [Figure 7] 4 is a data structure diagram showing an example of a personality setting DB of FIG. 3. [Figure 8] 4 is a data structure diagram showing an example of the testimonial DB of FIG. 3. [Figure 9] 13 is a flowchart showing a series of processes related to a distributor search in a server. [Figure 10] FIG. 13 is a representative screen diagram of a search reception screen displayed on the display of a user terminal of an active user. [Figure 11] FIG. 13 is a representative screen diagram of a search result screen displayed on the display of a user terminal of an active user. [Figure 12] 1 is a block diagram showing an example of a hardware configuration of an information processing device according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] Hereinafter, the same or equivalent components, members, processes, and signals shown in each drawing will be given the same reference numerals, and duplicated explanations will be omitted as appropriate. Also, some of the members that are not important for the explanation will be omitted in each drawing.
[0013] In a live streaming system according to an embodiment, data on live streaming performed by a broadcaster and information about the broadcaster themselves are input into a generative machine learning model such as a Generative Pre-trained Transformer (GPT). The machine learning model outputs a sentence introducing the broadcaster (hereinafter, an introduction). A personality is set for the machine learning model, and different introductions are output even with the same input if the personality is different. The live streaming system uses the generated introduction to search for broadcasters.
[0014] Information related to a live stream distributor is diverse, including image data and audio data of the live stream, comments, gifts, tags indicating the content, the distributor's own profile, and follower relationships. In the live stream system according to the present embodiment, such information is collected together in the form of an introductory text and becomes the subject of a search. This makes searches more efficient and improves search accuracy. Furthermore, different introductory texts are generated according to the personality set in the machine learning model, so that search results can be provided that match the user's mood and preferences.
[0015] FIG. 1 is a schematic diagram showing a configuration of a live distribution system 1 according to an embodiment of the present disclosure. The live distribution system 1 provides an interactive live distribution service in which a distributor (also called a live streamer) LV and a viewer (also called an audience) AU (AU1, AU2, ...) can communicate in real time. As shown in FIG. 1, the live distribution system 1 includes a server 10, a user terminal 20 on the distributor side, and a user terminal 30 (30a, 30b, ...) on the viewer side. In addition to distributors who distribute live distribution and viewers who watch live distribution, there are also users who log in to the live distribution platform but do not distribute or watch. Such users are called active users. The distributors, viewers, and active users may be collectively called users. The server 10 may be configured by one or more information processing devices connected to a network NW. The user terminals 20 and 30 may be, for example, mobile terminals such as smartphones, tablet terminals, laptop PCs, recorders, portable game consoles, and wearable devices, or may be stationary devices such as desktop PCs. The server 10, the user terminal 20, and the user terminal 30 are connected to each other so as to be able to communicate with each other via various wired or wireless networks NW.
[0016] The live distribution system 1 involves a distributor LV, a viewer AU, and an administrator (not shown) who manages the server 10. The distributor LV is a person who transmits content in real time by recording and filming his / her own songs, talks, performances, fortune telling, game commentary, etc. on his / her own user terminal 20 and uploading the content to the server 10 as is. The administrator provides a platform for live distribution of content on the server 10, and also mediates or manages real-time exchanges between the distributor LV and the viewer AU. The viewer AU accesses the platform with the user terminal 30, selects desired content, and views it. During the live distribution of this content, the viewer AU performs operations to make comments, cheer, or request fortune telling through the user terminal 30, and the distributor LV providing the content responds to such comments, cheers, or requests, and the reactions are transmitted to the viewer AU by video and / or audio, thereby establishing two-way communication.
[0017] In this specification, "live distribution" may mean a data transmission mode that realizes a state in which content recorded on a user terminal 20 of a distributor LV is played back and made viewable on a user terminal 30 of a viewer AU in substantially real time, or may mean distribution itself realized by such a transmission mode. Live distribution may be realized using existing live distribution technologies such as HTTP Live Streaming, Common Media Application Format, Web Real-Time Communications, Real-Time Messaging Protocol, and MPEG DASH. Live distribution includes a transmission mode in which a viewer AU can view content with a predetermined delay while a distributor LV is recording the content. The amount of delay is at least as large as the amount of communication between the distributor LV and the viewer AU. However, live distribution is distinguished from so-called on-demand distribution in which the entire recorded data of the content is temporarily stored on a server, and the data is provided to a user from the server at any time thereafter in response to a request from the user.
[0018] In this specification, "moving image data" refers to data including image data (also called video data) generated by the imaging function of the user terminals 20 and 30, and audio data (also called audio data) generated by the audio input function of the user terminals 20 and 30. The moving image data is played back on the user terminals 20 and 30, allowing users to view the content. In this embodiment, it is assumed that processing such as compression, decompression, encoding, decoding, transcoding, and other processing is performed to change the format, size, and specifications of the data from when the moving image data is generated on the distributor's user terminal until it is played back on the viewer's user terminal. Since the contents (e.g., moving images and audio) represented by the moving image data before and after such processing are substantially unchanged, this embodiment will be described assuming that the moving image data after such processing is the same as the moving image data before such processing. In other words, when moving image data is generated on the distributor's user terminal and then played back on the viewer's user terminal via the server 10, the moving image data generated on the distributor's user terminal, the moving image data passing through the server 10, and the moving image data received and played on the viewer's user terminal are all the same moving image data.
[0019] As used herein, "stream duration" is a parameter associated with a live stream and refers to the length of time that the live stream lasts. Stream duration is calculated regardless of whether the live stream has an audience. In this specification, the "total broadcast time" is a parameter associated with a broadcaster, and is the time obtained by adding up the broadcast time of all live broadcasts performed by the broadcaster in question during a specified period of time.
[0020] In the example of Fig. 1, a broadcaster LV is live-streaming a talk. The user terminal 20 of the broadcaster LV records and records the image and audio of the broadcaster LV while he or she is talking, generating video data, and transmits the video data to the server 10 via the network NW. The user terminal 20 also displays the recorded video image VD of the broadcaster LV on the display of the user terminal 20, thereby enabling the broadcast content by the broadcaster LV to be confirmed.
[0021] The user terminals 30a, 30b of the viewers AU1, AU2 who have requested the platform to watch the live broadcast of the broadcaster LV each receive video data related to the live broadcast via the network NW, and play the received video data to display the videos VD1, VD2 on the display and output audio from the speaker. The videos VD1, VD2 displayed on each user terminal 30a, 30b are substantially identical to the video VD captured by the user terminal 20 of the broadcaster LV, and the audio output from each user terminal 30a, 30b is also substantially identical to the audio recorded by the user terminal 20 of the broadcaster LV.
[0022] The recording and recording in the user terminal 20 of the distributor LV and the playback of the video data in the user terminals 30a and 30b of the viewers AU1 and AU2 are performed substantially simultaneously. When one viewer AU1 inputs a comment on the content of the talk of the distributor LV to the user terminal 30a, the server 10 displays the comment in real time on the user terminal 20 of the distributor LV and also on the user terminals 30a and 30b of each viewer AU1 and AU2. When the distributor LV reads the comment and develops a talk that overlaps the content, the video and audio of the talk are output on the user terminals 30a and 30b of each viewer AU1 and AU2, and it is recognized that a conversation between the distributor LV and the viewer AU1 has been established. In this way, the live distribution system 1 realizes live distribution that enables two-way communication, not one-way.
[0023] Fig. 2 is a block diagram showing the functions and configuration of the user terminal 20 in Fig. 1. The user terminal 30 has the same functions and configuration as the user terminal 20. Each block shown in Fig. 2 and the following block diagrams can be realized in hardware terms by elements and mechanical devices such as a computer CPU, and in software terms by a computer program, etc., but here, functional blocks realized by the cooperation of these are depicted. Therefore, those skilled in the art who have read this specification will understand that these functional blocks can be realized in various ways by combining hardware and software.
[0024] The distributor LV and the viewer AU download and install a live distribution application program (hereinafter, referred to as a live distribution app) according to this embodiment from a download site to the user terminals 20 and 30 via the network NW. Alternatively, the live distribution app may be pre-installed in the user terminals 20 and 30. When the live distribution app is executed by the user terminals 20 and 30, the user terminals 20 and 30 communicate with the server 10 via the network NW to realize various functions. Hereinafter, functions realized by the user terminals 20 and 30 (processors such as CPUs) executing the live distribution app will be described as functions of the user terminals 20 and 30. These functions are actually functions that the live distribution app causes the user terminals 20 and 30 to realize. Note that in other embodiments, these functions may be realized by a computer program written in a programming language such as HTML (HyperText Markup Language), which is transmitted from the server 10 to a web browser of the user terminals 20 and 30 via the network NW and executed by the web browser.
[0025] The user terminal 20 includes a distribution unit 100 that generates video data recording the user's image and voice and provides it to the server 10, a viewing unit 200 that acquires the video data from the server 10 and plays it, and a non-distribution processing unit 400 that processes requests from active users. The user activates the distribution unit 100 when distributing, the viewing unit 200 when viewing, and the non-distribution processing unit 400 when searching for a live broadcast to watch, viewing a distributor's profile, or viewing an archive. A user terminal on which the distribution unit 100 is active is a user terminal on the distributor side, that is, a user terminal on the generating side of video data, a user terminal on which the viewing unit 200 is active is a user terminal on the viewer side, that is, a user terminal on the playing side of video data, and a user terminal on which the non-distribution processing unit 400 is active is a user terminal of an active user.
[0026] The delivery unit 100 includes an imaging control unit 102, an audio control unit 104, a video transmission unit 106, a delivery-side UI control unit 108, and a delivery-side communication unit 110. The imaging control unit 102 is connected to a camera (not shown in FIG. 2) and controls imaging by the camera. The imaging control unit 102 acquires image data from the camera. The audio control unit 104 is connected to a microphone (not shown in FIG. 2) and controls audio input by the microphone. The audio control unit 104 acquires audio data from the microphone. The video transmission unit 106 transmits video data including the image data acquired by the imaging control unit 102 and the audio data acquired by the audio control unit 104 to the server 10 via the network NW. The video transmission unit 106 transmits the video data in real time. That is, the generation of video data by the imaging control unit 102 and the audio control unit 104 and the transmission of the generated video data by the video transmission unit 106 are substantially performed simultaneously.
[0027] The distribution side UI control unit 108 controls the UI for the distributor. The distribution side UI control unit 108 is connected to a display not shown in FIG. 2, and displays a moving image on the display by playing back the moving image data to be transmitted by the moving image transmission unit 106. The distribution side UI control unit 108 is connected to input means such as a touch panel, a keyboard, and a display not shown in FIG. 2, and acquires input from the distributor via the input means. The distribution side UI control unit 108 superimposes a predetermined frame image on the moving image. The frame image includes various user interface objects (hereinafter simply referred to as objects) for receiving input from the distributor, comments input by the viewer, and information acquired from the server 10. The distribution side UI control unit 108 accepts, for example, a tap input on an object by the distributor.
[0028] The delivery-side communication unit 110 controls communication with the server 10 during live delivery. The delivery-side communication unit 110 transmits the contents of the input by the distributor acquired by the delivery-side UI control unit 108 to the server 10 via the network NW. The delivery-side communication unit 110 receives various types of information associated with the live delivery from the server 10 via the network NW.
[0029] The viewing unit 200 includes a viewing side UI control unit 202 and a viewing side communication unit 204. The viewing side communication unit 204 controls communication with the server 10 during live distribution. The viewing side communication unit 204 receives video data related to the live distribution in which the distributor and the viewers participate from the server 10 via the network NW.
[0030] The viewer-side UI control unit 202 controls the UI for the viewer. The viewer-side UI control unit 202 is connected to a display and a speaker (not shown in FIG. 2), and plays the received video data to display a video image on the display and output audio from the speaker. The output of an image on the display and audio from the speaker can be collectively referred to as "video data being played". The viewer-side UI control unit 202 is connected to input means (not shown in FIG. 2), such as a touch panel, a keyboard, or a display, and acquires input by the viewer through the input means. The viewer-side UI control unit 202 superimposes a predetermined frame image on the image of the video data acquired from the server 10. The frame image includes various objects for receiving input from the viewer, comments input by the viewer, and information acquired from the server 10. The viewer-side communication unit 204 transmits the contents of the input by the viewer acquired by the viewer-side UI control unit 202 to the server 10 via the network NW.
[0031] The non-broadcast processing unit 400 includes a non-broadcast UI control unit 402 and a non-broadcast communication unit 404. The non-broadcast UI control unit 402 controls a UI for active users. For example, the non-broadcast UI control unit 402 generates a live broadcast selection screen that displays a list of live broadcasts that can be currently participated in and accepts the selection of a live broadcast by an active user, and displays it on the display. The non-broadcast UI control unit 402 generates a profile screen of an arbitrary user and displays it on the display. The non-broadcast UI control unit 402 generates a search screen that accepts search keywords for searching for broadcasters, and displays it on the display. The non-broadcast UI control unit 402 generates a search result display screen including search results for broadcasters, and displays it on the display. The non-broadcast UI control unit 402 plays back an archive generated by recording and filming past live broadcasts.
[0032] The non-broadcast communication unit 404 controls communication with the server 10 outside of live broadcasting. The non-broadcast communication unit 404 receives information for generating a live broadcast selection screen, broadcaster search results, information for generating a profile screen, and archive data from the server 10 via the network NW. The non-broadcast communication unit 404 transmits the contents of input by active users to the server 10 via the network NW.
[0033] Fig. 3 is a block diagram showing the functions and configuration of the server 10 in Fig. 1. The server 10 includes a distribution information providing unit 302, a relay unit 304, a gift processing unit 308, a payment processing unit 310, a stream DB 314, a user DB 318, a gift DB 320, a distributor search unit 322, a testimonial generating unit 324, a personality setting DB 326, and a testimonial DB 328.
[0034] 4 is a data structure diagram showing an example of the stream DB 314 in FIG. 3. The stream DB 314 holds information on a live stream currently being performed and information on live streams (archives) performed in the past. In particular, the stream DB 314 holds video data including images and audio of videos distributed by a distributor. The stream DB 314 holds, in association with each other, a stream ID that identifies a live stream on a live streaming platform provided by the live streaming system 1, a distributor ID that is a user ID that identifies a distributor of the live stream, a viewer ID that is a user ID that identifies a viewer of the live stream, a start time of the live stream, an end time of the live stream (in the case of an archive), a distribution content tag that represents the content of the live stream, image data of the live stream, audio data of the live stream, a history of comments posted in the live stream, and a history of gifts used in the live stream.
[0035] In the live streaming platform provided by the live streaming system 1 according to the present embodiment, when a user performs live streaming, the user becomes a streamer, and when the same user watches a live stream by another user, the user becomes a viewer. Therefore, the distinction between streamer and viewer is not fixed, and a user ID registered as a streamer ID at one time may be registered as a viewer ID at another time.
[0036] The content tag of the live broadcast may be a tag specified by the broadcaster when the broadcaster starts the live broadcast, or may be a tag obtained by a model generated by machine learning analyzing the live broadcast in real time.
[0037] FIG. 5 is a data structure diagram showing an example of the user DB318 in FIG. 3. The user DB318 holds information about users. In particular, the user DB318 holds information about distributors of videos distributed on the live distribution platform. The user DB318 includes a user ID that identifies a user, points owned by the user, rewards given to the user, the level of the user, the age range of the user, the gender of the user, the hair color of the user, a region indicating the area where the user uses the live distribution app, a user tag attached to the user, a registration date that is the date on which the user registered on the live distribution platform, a cumulative distribution time that is the total distribution time of the user, a profile image registered by the user on the live distribution platform, user IDs of other users followed by the user, user IDs of other users following the user, a history of the user viewing live distributions and profiles of other distributors, a history of the user's comments, the frequency of the user's login, the user's monthly consumption points, and the user's self-introduction.
[0038] Points are electronic value circulated within the live streaming platform. Users purchase points by credit card or other payment methods. Rewards are electronic value defined within the live streaming platform and are an indicator for determining the amount of money that a broadcaster receives from the administrator of the live streaming platform. On the live streaming platform, when a viewer gives a gift to a broadcaster during or outside of a live streaming, the viewer's points are consumed and the broadcaster's reward increases accordingly.
[0039] The level is an index indicating the performance of the user as a broadcaster on the live streaming platform. In another embodiment, the level may be an index indicating the performance of the user as a viewer on the live streaming platform, or may be an index indicating the performance of the user as both a broadcaster and a viewer. The level may increase or decrease depending on the number of live broadcasts, the broadcast time of the live broadcast, the number and / or amount of gifts given, the number and / or amount of gifts received, the number of comments, etc. Alternatively, the level may be evaluated and determined by an administrator based on reviews of the broadcaster, user satisfaction, and comments in the broadcast. Alternatively, the level may be automatically determined by a predetermined rule or a machine learning model.
[0040] Fig. 6 is a data structure diagram showing an example of the gift DB 320 in Fig. 3. The gift DB 320 holds information about gifts that can be used by viewers in live distribution. A gift is electronic data that has the following characteristics. -Can be purchased with points or money, or given free of charge. Something that viewers can give to the streamer. Giving a gift to a streamer is also called using a gift or throwing a gift. In some cases, the purchase and use of the gift occur simultaneously as a set, while in other cases the viewer can use the gift at any time after purchasing it. When a viewer gives a gift to a streamer, the streamer will receive a corresponding reward. When a gift is used, an effect associated with the gift may occur. For example, an effect corresponding to the gift may appear on the live streaming room screen.
[0041] The gift DB320 stores a gift ID that identifies a gift, a given reward that is given to a broadcaster when the gift is given to the broadcaster, and a value point that is a value to be paid when the gift is used, in association with each other. A viewer can give a gift to a broadcaster by paying the value points of a desired gift while watching a live broadcast. The payment of the value points may be made by an appropriate electronic payment means, for example, the viewer may pay the value points to the administrator. Alternatively, payment may be made by bank transfer or credit card. The relationship between the given reward and the value points can be arbitrarily set by the administrator. For example, the given reward may be set to the value points. Alternatively, the value points may be set to points obtained by multiplying the given reward by a predetermined coefficient such as 1.2, or the value points may be set to points obtained by adding a predetermined commission point to the given reward.
[0042] FIG. 7 is a data structure diagram showing an example of the personality setting DB326 in FIG. 3. In this embodiment, generative machine learning models (hereinafter, sometimes simply referred to as models) such as GPT-3, GPT-3.5, and GPT-4 provided by openAI are used as machine learning models that generate introductions. A personality can be set for a model by inputting constraints and guidelines for action into the prompts of the model (for example, see Non-Patent Document 1). The personality setting DB326 holds a personality ID that specifies the personality to be set for the model and a prompt corresponding to the personality in association with each other. In this embodiment, five prompts corresponding to five types of personalities are provided, but the number and contents of personalities can be set arbitrarily.
[0043] Fig. 8 is a data structure diagram showing an example of the introduction DB 328 of Fig. 3. The introduction DB 328 stores the broadcaster ID of the broadcaster to be introduced, the personality ID specifying the personality set in the model, the catch phrase output by the model, and the introduction output by the model in association with each other. In this embodiment, there are five models set by five different prompts, so five introductions are associated with one broadcaster ID.
[0044] Returning to FIG. 3, the introductory sentence generation unit 324 generates a plurality of introductory sentences for each distributor by referring to the stream DB 314 and the user DB 318, and registers the generated introductory sentences in the introductory sentence DB 328. The introductory sentence generation unit 324 includes a plurality of models each having a different personality. The introductory sentence generation unit 324 includes a conversion unit 338, a first introductory sentence generation model 340, a second introductory sentence generation model 342, a third introductory sentence generation model 344, a fourth introductory sentence generation model 346, and a fifth introductory sentence generation model 348.
[0045] The converter 338 converts the video data stored in the stream DB 314 into text data. A known image analysis technique may be used to convert image data into text data expressing the contents of the image data (see, for example, Non-Patent Document 2). A known Speech to Text (STT) technique may be used to convert audio data into text.
[0046] The first introduction sentence generation model 340 is a model in which a personality specified by the personality ID "CH1" in the personality setting DB 326 in Fig. 7 is set. The first introduction sentence generation model 340 is generated by inputting a prompt corresponding to the personality ID "CH1" in the personality setting DB 326 in Fig. 7 into a default model. This model may be generated manually by an administrator of the live distribution platform, or automatically by the introduction sentence generation unit 324 referring to the personality setting DB 326.
[0047] The first introductory sentence generation model 340 receives as input text data converted from video data of a broadcaster's live broadcast and information about the broadcaster stored in the user DB 318, and outputs a catchy copy and an introductory sentence for the broadcaster.
[0048] The second introduction sentence generation model 342, the third introduction sentence generation model 344, the fourth introduction sentence generation model 346, and the fifth introduction sentence generation model 348 are models in which personalities specified by the personality IDs "CH2", "CH3", "CH4", and "CH5" in the personality setting DB 326 in Fig. 7 are set. The second introduction sentence generation model 342, the third introduction sentence generation model 344, the fourth introduction sentence generation model 346, and the fifth introduction sentence generation model 348 all receive text data converted from video data of a live stream by a streamer and information about the streamer stored in the user DB 318 as input, and output a catchphrase and introduction text for the streamer.
[0049] When the distributor search unit 322 receives a request to search for a distributor from the user terminal of an active user, it searches the introduction DB 328 and returns the search results to the user terminal. The distributor search unit 322 includes a search request receiving unit 330, a job posting text generation model 332, a search unit 334, and a providing unit 336.
[0050] The search request receiving unit 330 receives a search request from a user terminal of an active user of the live distribution platform via the network NW. The search request includes the user ID of the active user who made the request, the personality ID of the personality specified by the active user, and a keyword to be searched for (hereinafter referred to as a search keyword) input by the active user.
[0051] The search unit 334 executes a search on the testimonial DB 328 based on the received search request.
[0052] The providing unit 336 provides the results of the search executed by the searching unit 334 to the requesting user terminal via the network NW.
[0053] The distributor search unit 322 allows for distributor searches in the following four modes. (1) Keyword search, no personality setting (2) Keyword search and personality settings available (3) Sentence matching search, no personality setting (4) Sentence matching search, personality setting available If personality settings are enabled, search results may differ if the personality is different, even if the search keywords are the same.
[0054] (1) Keyword search, no personality setting mode The search unit 334 searches the testimonials stored in the testimonial DB 328 using the search keyword included in the received search request as a key, and obtains a predetermined number of distributor IDs with the highest matching scores as search results. The technology for searching text using a search keyword as a key is well known, and will not be described in detail in this specification.
[0055] The search unit 334 may generate new search keywords by estimation from the search keywords included in the search request. For example, if the search keywords included in the search request are "guitar beginner" and "XX" (artist name), "guitar beginner", "guitar playing introduction", "guitar practice method", "guitar chords", "guitar chords for XX songs", and "online guitar lessons" are generated by word estimation and added to the search keywords. In addition, if estimation with adjectives is used, "guitar lessons for beginners", "how to learn guitar in a fun way", "guitar lessons to learn slowly", "guitar lessons from a reliable source", "XX guitar TAB score", and "XX songs recommended for beginners" are generated and added to the search keywords.
[0056] The providing unit 336, for each broadcaster ID acquired by the search unit 334, refers to the stream DB 314 to acquire information on the live broadcast currently being performed by the broadcaster identified by the broadcaster ID (hereinafter referred to as live broadcast information) and information on archives of past live broadcasts performed by the broadcaster ID (hereinafter referred to as archive information). The providing unit 336 generates a search result list including the broadcaster ID acquired by the search unit 334 and the live broadcast information and archive information acquired for the broadcaster ID. The providing unit 336 transmits the generated search result list to the requesting user terminal via the network NW.
[0057] (2) Keyword search and personality setting mode The search unit 334 identifies the introductions that correspond to the personality ID included in the received search request from among the introductions stored in the introduction DB 328. The search unit 334 searches the introductions identified above using the search keyword included in the received search request as a key, and obtains a predetermined number of distributor IDs from those with the highest matching scores as search results. The subsequent processing is similar to that of the mode (1).
[0058] (3) Sentence matching search, no personality setting The job posting text generation model 332 receives the search keyword included in the search request and the information of the active user who made the request as input, and outputs a text expressing the desire of the active user (hereinafter referred to as a search text). The job posting text generation model 332 can generate the following two types of text as a search text. (A) A profile of the broadcaster that the active user requesting the broadcaster would like (B) A statement from the active user requesting the streamer describing the type of person they are looking for
[0059] Regarding (A), the job posting text generation model 332 refers to the user DB 318 and obtains information corresponding to the user ID included in the search request as information of the active user who made the request. The job posting text generation model 332 generates a profile of a distributor that the active user would like to see, based on the search keywords included in the search request and the information of the active user who made the request. The profile includes a user ID, a profile picture, and a self-introduction. An example of a profile generated in this way is shown below. "User ID: HHH Profile picture: "Illustration of a woman holding a guitar against a bright pink background." Self-introduction: I'm a high school girl who loves music! I'm sensitive to trends and always trying new music! I also love playing the guitar! Let's talk about music together. My favorite music genres are rock, pop, and EDM. My hobbies are listening to music, singing while playing the guitar, going to concerts, and talking about music on social media. My favorite artists are XX and YY.
[0060] Based on the generated profile, the job posting text generation model 332 may generate a guess and a simple description of the type of person that the active user would like as a distributor, and add the guess and simple description to the profile. An example of the guess and simple description generated in this way is shown below. XX: A pop country singer-songwriter known for catchy melodies and story-telling lyrics. YY: You are a young singer-songwriter with a unique style, personal lyrics, and a dark atmosphere. KK: You're a popular singer-songwriter known for acoustic songs using guitar, and your melodies and lyrics are memorable. LL: She's a pop singer with an impressive high range and powerful vocals, and her emotional lyrics are also appealing. PP: A pop R&B singer with funky beats and rhythmic melodies, her performances are also fun.
[0061] Regarding (B), the job posting text generation model 332 generates a text that describes the type of person the active user is looking for in a distributor, based on the search keywords included in the search request and the information of the active user who made the request. An example of such a text is shown below. "I enjoy playing the guitar, but I'm worried that I'm not getting any better. I want to get better, but I don't know what to do. I'd like to know how to practice more efficiently and how to improve my skills. Anyway, I want to play XX's song, but it's so difficult that I'm about to give up. Please let me know if there are any easier methods or tips." "I'm a beginner and I'm having trouble figuring out how to practice guitar. I want to play XX's song, but it's difficult and I'm about to give up. I'd like to know how to play it more easily but also well. I especially want to play LL and YM. I'd like some advice on practice methods for beginners and tips on remembering chord progressions. Also, please let me know if you have any recommendations for sites where I can take guitar lessons online."
[0062] The search unit 334 executes a search by comparing the search text output by the job posting text generation model 332 with the introductory text stored in the introductory text DB 328. The search unit 334 calculates the similarity between the search text and each of the introductory texts stored in the introductory text DB 328. The calculation of the similarity between the texts may be realized based on, for example, the number of important words that match, the number of similar words that appear, or TF-IDF (term frequency - inverse document frequency), or may be realized using a comparison technique such as that described in Non-Patent Document 3. The search unit 334 obtains a predetermined number of distributor IDs as search results in order of the calculated similarity. The processing of the provision unit 336 is similar to that of the mode (1).
[0063] (4) Sentence matching search, personality setting available A personality specified by the personality ID included in the received search request (one of the personality IDs "CH1", "CH2", "CH3", "CH4", and "CH5" in the personality setting DB 326 in FIG. 7) is set in the job recruitment text generation model 332. The job recruitment text generation model 332 is generated by inputting a prompt corresponding to the personality ID included in the search request into a default model. When the distributor search unit 322 receives a search request, it refers to the personality setting DB 326 and automatically generates the job recruitment text generation model 332. Subsequent processing is similar to that in mode (3).
[0064] Alternatively, the search unit 334 may calculate the similarity between the search text and each of the introductory texts corresponding to the personality ID included in the search request among the introductory texts stored in the introductory text DB 328. The administrator can appropriately set whether to reflect the personality setting in the generation of the search text or to apply it to the narrowing down of the introductory texts.
[0065] The non-broadcast UI control unit 402 of the user terminal that has issued the search request generates a search result screen based on the received search result list, and displays it on the display of the user terminal. When the non-broadcast UI control unit 402 accepts the selection of a live broadcast by the active user on the search result screen, it generates a broadcast request including the stream ID of the selected live broadcast, and transmits it to the server 10 via the network NW. The broadcast information providing unit 302 starts providing the live broadcast identified by the stream ID included in the received broadcast request to the requesting user terminal. The broadcast information providing unit 302 updates the stream DB 314 so that the viewer ID of the stream ID includes the user ID of the active user of the requesting user terminal. As a result, the active user becomes a viewer of the selected live broadcast.
[0066] The relay unit 304 relays the transmission of video data from the user terminal 20 of the distributor to the user terminal 30 of the viewer in the live distribution started by the distribution information providing unit 302. The relay unit 304 receives a signal indicating a user input by the viewer during the live distribution, i.e., during the playback of the video data, from the viewing side communication unit 204. The signal indicating the user input may be an object designation signal indicating the designation of an object displayed on the display of the user terminal 30, and the object designation signal includes the viewer ID of the viewer, the distributor ID of the distributor performing the live distribution that the viewer is viewing, and an object ID that identifies the object. If the object is a gift icon, the object ID is a gift ID. In that case, the object designation signal is a gift use signal indicating the use of a gift by the viewer for the distributor. Similarly, the relay unit 304 receives a signal indicating a user input by the distributor during the playback of the video data, for example, an object designation signal, from the distribution side communication unit 110 of the distribution unit 100 of the user terminal 20.
[0067] The gift processing unit 308 updates the user DB 318 so as to increase the distributor's reward according to the reward for the gift specified by the gift ID included in the gift use signal. The gift processing unit 308 refers to the gift DB 320 and specifies the reward corresponding to the gift ID included in the received gift use signal. The gift processing unit 308 updates the user DB 318 so as to add the specified reward to the reward corresponding to the distributor ID included in the gift use signal.
[0068] In response to receiving the gift use signal, the payment processing unit 310 processes the payment of the value of the gift by the viewer. The payment processing unit 310 refers to the gift DB 320 and identifies the value points of the gift identified by the gift ID included in the gift use signal. The payment processing unit 310 updates the user DB 318 so as to subtract the identified value points from the points of the viewer identified by the viewer ID included in the gift use signal.
[0069] The operation of the live distribution system 1 configured as above will now be described. 9 is a flowchart showing a series of processes related to a distributor search in the server 10. The server 10 converts video data of a distributor's live distribution stored in the stream DB 314 into text data (S202). The server 10 repeats the following processes of steps S204 and S206 for each introductory sentence generation model. The server 10 inputs the text data and information on the distributor into the introductory sentence generation model (S204). The server 10 registers the catch phrase and introductory sentence output by the introductory sentence generation model in the introductory sentence DB 328 (S206).
[0070] The server 10 receives a search request from a user terminal via the network NW (S208). The server 10 inputs the search keyword and the information of the search requester included in the search request to the job posting text generation model 332 (S210). The server 10 executes a search for a distributor by comparing the search text output by the job posting text generation model 332 with the introductory text stored in the introductory text DB 328 (S212). The server 10 transmits the information of the distributor obtained as a result of the search and the information of the live distribution and archives being performed by the distributor to the user terminal of the search requester (S214).
[0071] 10 is a representative screen diagram of a search reception screen 600 displayed on the display of the user terminal of an active user. The search reception screen 600 has a keyword input area 602 that accepts keywords input by the active user in a free input format, a personality designation area 604 that accepts personality designation by the active user in a pull-down format, and a search button 606.
[0072] The active user inputs a desired keyword into keyword input area 602, selects a desired personality in personality designation area 604, and presses search button 606. When non-broadcast communication unit 404 of the user terminal detects the pressing of search button 606, it generates a search request including the keyword input into keyword input area 602, the personality ID of the personality selected in personality designation area 604, and the active user's user ID, and transmits the search request to server 10 via network NW.
[0073] Fig. 11 is a representative screen diagram of a search result screen 610 displayed on the display of a user terminal of an active user. The non-broadcast communication unit 404 receives from the server 10 a search result list corresponding to the search request transmitted via the search reception screen 600 of Fig. 10. The non-broadcast UI control unit 402 generates the search result screen 610 based on the received search result list and displays it on the display. The search result screen 610 includes a broadcaster ID 612 included in the search result list, a thumbnail 614 representing a live broadcast currently being performed by the broadcaster identified by the broadcaster ID, and a thumbnail 616 representing an archive of past live broadcasts by the broadcaster.
[0074] In the above-mentioned embodiment, examples of the storage unit are a hard disk and a semiconductor memory. Furthermore, based on the description in this specification, it will be understood by those skilled in the art who have read this specification that each unit can be realized by a CPU (not shown), an installed application program module, a system program module, a semiconductor memory that temporarily stores the contents of data read from a hard disk, or the like.
[0075] According to the live distribution system 1 according to the present embodiment, an introduction is generated for each distributor and registered in the introduction DB 328. When searching for a distributor, the introduction stored in the introduction DB 328 is used. By configuring the system to search for an introduction that consolidates a wide variety of information related to the distributor, such as video data and a profile, it is possible to improve the accuracy of the search and / or the speed of the search.
[0076] In addition, in the live distribution system 1 according to the present embodiment, multiple introductions are generated for the same distributor using different models with different personalities. This makes it possible to provide search results that are more in line with the desires of active users who search for distributors. Furthermore, the accuracy of search results can be further improved by allowing active users to specify their desired personality.
[0077] The hardware configuration of the information processing device according to the present embodiment will be described with reference to Fig. 12. Fig. 12 is a block diagram showing an example of the hardware configuration of the information processing device according to the present embodiment. The illustrated information processing device 900 can realize, for example, each of the server 10 and the user terminals 20 and 30 in the present embodiment.
[0078] The information processing device 900 includes a CPU 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903. The information processing device 900 may also include a host bus 907, a bridge 909, an external bus 911, an interface 913, an input device 915, an output device 917, a storage device 919, a drive 921, a connection port 925, and a communication device 929. The information processing device 900 further includes an imaging device (not shown) such as a camera. The information processing device 900 may also have a processing circuit such as a DSP (Digital Signal Processor) or an ASIC (Application Specific Integrated Circuit) instead of or in addition to the CPU 901.
[0079] The CPU 901 functions as an arithmetic processing device and a control device, and controls the overall operation or a part of the operation in the information processing device 900 according to various programs recorded in the ROM 902, the RAM 903, the storage device 919, or the removable recording medium 923. For example, the CPU 901 controls the overall operation of each functional unit included in the server 10 and the user terminals 20 and 30 in this embodiment. The ROM 902 stores programs and arithmetic parameters used by the CPU 901. The RAM 903 temporarily stores programs used in the execution of the CPU 901 and parameters that change appropriately during the execution. The CPU 901, the ROM 902, and the RAM 903 are mutually connected by a host bus 907 constituted by an internal bus such as a CPU bus. Furthermore, the host bus 907 is connected to an external bus 911 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 909.
[0080] The input device 915 may be, for example, a device operated by a user, such as a mouse, a keyboard, a touch panel, a button, a switch, or a lever, or may be a device that converts a physical quantity into an electric signal, such as a sound sensor such as a microphone, an acceleration sensor, a tilt sensor, an infrared sensor, a depth sensor, a temperature sensor, or a humidity sensor. The input device 915 may be, for example, a remote control device that uses infrared or other radio waves, or an external connection device 927 such as a mobile phone that supports the operation of the information processing device 900. The input device 915 includes an input control circuit that generates an input signal based on information input by the user or a sensed physical quantity and outputs the input signal to the CPU 901. The user operates the input device 915 to input various data to the information processing device 900 or to instruct a processing operation.
[0081] The output device 917 is configured with a device capable of visually or audibly notifying the user of acquired information. The output device 917 can be, for example, a display such as an LCD, a PDP, or an OELD, an audio output device such as a speaker or a headphone, or a printer device. The output device 917 outputs the results obtained by the processing of the information processing device 900 as a video such as text or an image, or as a sound such as audio.
[0082] The storage device 919 is a data storage device configured as an example of a storage unit of the information processing device 900. The storage device 919 is configured, for example, with a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage device 919 stores programs and various data executed by the CPU 901, and various data acquired from the outside.
[0083] The drive 921 is a reader / writer for a removable recording medium 923 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and is built into or externally attached to the information processing device 900. The drive 921 reads out information recorded on the attached removable recording medium 923 and outputs the information to the RAM 903. The drive 921 also writes information to the attached removable recording medium 923.
[0084] The connection port 925 is a port for directly connecting a device to the information processing device 900. The connection port 925 can be, for example, a Universal Serial Bus (USB) port, an IEEE 1394 port, a Small Computer System Interface (SCSI) port, or the like. The connection port 925 can also be an RS-232C port, an optical audio terminal, a High-Definition Multimedia Interface (HDMI) (registered trademark) port, or the like. By connecting an external connection device 927 to the connection port 925, various types of data can be exchanged between the information processing device 900 and the external connection device 927.
[0085] The communication device 929 is, for example, a communication interface configured with a communication device for connecting to a network NW. The communication device 929 can be, for example, a communication card for a wired or wireless LAN (Local Area Network), Bluetooth (registered trademark), or WUSB (Wireless USB). The communication device 929 may also be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various communications. The communication device 929 transmits and receives signals, for example, between the Internet and other communication devices using a predetermined protocol such as TCP / IP. The communication network NW connected to the communication device 929 is a network connected by wire or wirelessly, for example, the Internet, a home LAN, infrared communication, radio wave communication, or satellite communication. The communication device 929 realizes a function as a communication unit.
[0086] An imaging device (not shown) such as a camera is a device that captures real space and generates a captured image using an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) and various members such as a lens for controlling the formation of a subject image on the imaging element. The imaging device may capture a still image or a moving image.
[0087] The above describes the configuration and operation of the live streaming system 1 according to the embodiment. This embodiment is merely an example, and it will be understood by those skilled in the art that various modifications are possible in the combination of each component and each process, and that such modifications are also within the scope of the present disclosure.
[0088] Although the embodiment has been described using a live streaming platform as an example, the present invention is not limited to this and the technical ideas of the embodiment can also be applied to other video streaming platforms such as VOD (Video On Demand).
[0089] In the embodiment, a case has been described in which one of the characteristics set in the introductory text generation model is set in the recruitment text generation model 332, but this is not limited to this, and for example, the recruitment text generation model may have a characteristic not included in the characteristics of the introductory text generation model, or may not have a characteristic.
[0090] In an embodiment, the introductory sentence generation model may generate supplementary information for the introductory sentence. Examples of the supplementary information are as follows: List of related songs and introductions XX(Artist name) "ABCDE" is a country pop song that has a catchy melody and a structure based on music theory. The chord progression in the chorus is G, D, Em, C, which gives the listener a sense of stability and uplift. "HHKKLL" is a song that incorporates elements of funk and R&B in addition to a pop melody. The song's structure makes heavy use of the IV-vi-IV chord progression, making it a song that leaves a strong impression on the listener. "PPQQQ" is a song that incorporates modern elements such as synthesizer sounds and beatboxing in addition to its pop melody. The song is structured with a IV-vi-IV chord progression, which gives the listener a comfortable impression. The above songs are some of the songs that XX created using music theory. Her songs are not only pleasant and impressive to the listener, but also full of interesting elements from a music theory perspective. "CCDDE" is famous for its diverse chord progressions and music theory that are incorporated into the guitar riffs and solos. "FFG" This song is characterized by a guitar solo that uses complex harmonies and scales, and is often used as an example of the application of music theory. "HKLL" This song features a melodic intro and guitar solo, and is often used to explain harmony theory and melody structure. "Beginner List: "TT" … Other lists: "ZA"
[0091] In the embodiment, information published on an external platform such as a social networking site may be used as user information.
[0092] In the embodiment, a case has been described in which a search request is generated by specifying a search keyword, but this is not limited to this, and the technical idea of this embodiment can also be applied, for example, to a case in which a distributor is recommended to a user.
[0093] The conversion rates from gift points to given rewards in the embodiments are merely examples, and may be set appropriately by, for example, an administrator of the live distribution system.
[0094] The technical ideas of the embodiments may be applied to virtual live streaming and live commerce, in which an avatar that moves in sync with the streamer's movements is used instead of an image of the streamer.
[0095] In the processing procedures described in this specification, particularly in the processing procedures described using flow diagrams and flowcharts, it is possible to omit some of the steps that make up the processing procedures, to add steps that are not explicitly stated as steps that make up the processing procedures, and / or to rearrange the order of the steps, and processing procedures in which such omissions, additions, or changes in order have been made are included within the scope of this disclosure as long as they do not deviate from the spirit of this disclosure.
[0096] At least a part of the functions realized by the server 10 may be realized by a device other than the server 10, for example, the user terminals 20 and 30. At least a part of the functions realized by the user terminals 20 and 30 may be realized by a device other than the user terminals 20 and 30, for example, the server 10. For example, the superimposition of a predetermined frame image onto an image of video data performed at a viewer's user terminal may be performed at the server 10 or at a distributor's user terminal.
Claims
1. A distributor information storage unit that stores information on distributors of videos distributed on a video distribution platform; a video data storage unit that stores video data including images and audio of the video distributed by the distributor; a conversion unit that converts the video data stored in the video data storage unit into text data; A plurality of machine learning models, each of which receives the text data and the information of the distributor stored in the distributor information storage unit as input and outputs a text introducing the distributor, each of which has a different personality; A text storage unit that stores an ID that identifies the distributor and a plurality of texts that introduce the distributor output by the plurality of machine learning models in association with each other; a request receiving unit that receives a search request from a user terminal of a user of the video distribution platform via a network; a search unit that executes a search in the text storage unit based on the search request; a providing unit that provides the results of the search executed by the search unit to the user terminal via the network.
2. Further, another machine learning model is provided that receives as input a search target keyword included in the search request and information about the user of the user terminal, and outputs a sentence expressing a request of the user, The server according to claim 1 , wherein the search unit performs a search by comparing a sentence output by the other machine learning model with a sentence stored in the sentence storage unit.
3. The server according to claim 2 , wherein the other machine learning model is set to have the same personality as one of the plurality of machine learning models.
4. The server according to claim 1 , wherein the video distributed on the video distribution platform is distributed in real time by the distributor.
Citation Information
Patent Citations
Using machine learning to recommend live stream content
JP2020521207A
Extracting a portion of a document, such as a web page
US10503806B2
Cited By
Distribution system, distribution method, and distribution program
JP7784594B1
