Content distribution system, content distribution method, and content distribution program

Abstract

A content distribution system of one embodiment has at least one processor. The at least one processor displays content on a plurality of user terminals, and acquires, from two or more user terminals among the plurality of user terminals, line-of-sight data indicating a line-of-sight position of a user who visually confirms the content. The acquired two or more pieces of line-of-sight data are analyzed to calculate a distribution of line-of-sight positions of the two or more users in the content, and a mode of imparting a special effect to the content displayed on at least one target terminal among the plurality of user terminals is determined on the basis of the calculated distribution.

Description

Technical Field

[0001] One aspect of the present disclosure relates to a content publishing system, a content publishing method, and a content publishing program. Background Art

[0002] There are known technologies for distributing content using information related to the line of sight of viewers. For example, Patent Document 1 discloses a dynamic image distribution system that sends a dynamic image based on material data used to generate the dynamic image, obtains event information related to the line of sight from viewers of the dynamic image during the reproduction of the dynamic image, and operates the material data based on the event information. Patent Documents 2 and 3 also disclose technologies for distributing content using information related to the line of sight of viewers.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2020-120304

[0006] Patent Document 2: Japanese Patent Application Publication No. 2016-126773

[0007] Patent Document 3: Japanese Patent Application Publication No. 2020-202575 Summary of the invention

[0008] Problems to be solved by the invention

[0009] Mechanisms that increase the appeal of content to viewers are desirable.

[0010] Means for solving problems

[0011] A content publishing system according to one aspect of the present disclosure includes at least one processor. The at least one processor displays content on multiple user terminals, obtains line of sight data indicating line of sight positions of users who visually confirm the content from two or more user terminals among the multiple user terminals, analyzes the obtained two or more line of sight data to calculate the distribution of line of sight positions of two or more users in the content, and determines a method of assigning special effects to the content displayed on at least one target terminal among the multiple user terminals based on the calculated distribution.

[0012] In this aspect, the distribution of the gaze positions of two or more users in the content is obtained by analyzing two or more gaze data, and the method of giving special effects to the content is determined based on the distribution. Through this mechanism, the method of giving special effects is determined based on the reactions of multiple viewers to the content, thereby increasing the attractiveness of the content to the viewers.

[0013] Effects of the Invention

[0014] According to one aspect of the present disclosure, it is possible to improve the attractiveness of content to viewers. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a diagram showing an example of application of the content distribution system according to the embodiment.

[0016] Figure 2 This is a diagram showing an example of a hardware configuration associated with the content distribution system according to the embodiment.

[0017] Figure 3 This is a diagram showing an example of a functional configuration associated with the content distribution system according to the embodiment.

[0018] Figure 4 This is a sequence diagram showing an example of content distribution processing.

[0019] Figure 5 This is a diagram showing an example of content before adding special effects.

[0020] Figure 6 This is a diagram showing an example of content after adding special effects.

[0021] Figure 7 This is a diagram showing another example of content to which special effects are added. DETAILED DESCRIPTION

[0022] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same reference numerals are given to the same or equivalent elements, and repeated descriptions are omitted.

[0023] [System Overview]

[0024] The content publishing system of the implementation method is a computer system that publishes content to users. Content refers to information that is provided by a computer or a computer system and can be recognized by humans. Electronic data representing content is called content data. There is no limitation on the form of content, and content can be expressed by images (for example, photos, videos, etc.). Content can be used for various forms of information transmission or communication, for example, it can be used in various scenarios or purposes such as news, education, medical care, games, chats, business transactions, speeches, seminars, training, concerts, questionnaires, etc. Content publishing refers to the processing performed to provide content to users, for example, it refers to the processing of sending information to users via a communication network.

[0025] The content publishing system provides content to users by sending content data to user terminals. In one example, the content is provided by a publisher. A publisher is a person who wants to provide information to viewers through the content publishing system and is a sender of content. A viewer is a person who wants to obtain information through the content publishing system and is a user of content. The content publishing system sends content data provided from the publisher's terminal to the viewer's terminal. The viewer terminal processes the content data and displays the content on the screen. In the present disclosure, publishers and viewers are sometimes collectively referred to as users for explanation, and publisher terminals and viewer terminals are collectively referred to as user terminals for explanation.

[0026] There is no limitation on the method of publishing content. For example, the content publishing system can publish in real time. In this case, for example, the publisher terminal processes the captured image to generate content data, and sends the content data to the content publishing system in real time. The content publishing system sends the received content data to the viewer terminal in real time. This is a method of live broadcasting on the Internet. The content data can be generated in the content publishing system. That is, the content publishing system can generate content data by processing the real-time image provided from the publisher terminal, and send the content data to the viewer terminal in real time. The content publishing system can also be used for time-shifting recording (timeshifting) that allows viewing of content within a given period after real-time publishing.

[0027] The content distribution system can also perform on-demand distribution of content that viewers can watch at any time. In this case, for example, the content distribution system can store content data generated by processing previously shot images in a storage device such as a database. The content distribution system can send the stored content data to the viewer terminal in response to the viewer's request.

[0028] The content publishing system obtains sight data from two or more user terminals, and the sight data indicates the sight position of the user who visually confirms the content. The sight position of the user is the position where the user is looking in the screen of the user terminal. The sight data is obtained by an eye tracking system. The eye tracking system can be installed in the user terminal or in another computer different from the user terminal. Alternatively, the tracking system can also be implemented by the user terminal in collaboration with other computers.

[0029] The content publishing system calculates the distribution of the sight lines of more than two users in the content. The distribution of the sight line position is information indicating where the sight lines of each user are located in the content, for example, indicating the concentration of the sight lines of the user. This distribution can show to what extent each area in the content is focused on by viewers. The publishing system analyzes more than two sight line data and calculates the distribution of the sight line positions of more than two users. The number of sight line data used to calculate the distribution of the sight line position only needs to be more than two, and more than 100 or more than 1000 sight line data can be used. The content publishing system can determine the first area in the content where the sight lines of the users are concentrated and the second area where the sight lines of the users are not concentrated compared to the first area based on the distribution of the sight line positions. The first area can also be said to be an area that contains relatively more sight lines (sight line positions) of the users, or an area that is seen by many users. The second area can also be said to be an area where the sight lines (sight line positions) of the users are less than the first area, or an area that the users can hardly see.

[0030] The content publishing system determines the way of assigning special effects to the content based on the distribution of the sight line positions. Special effects refer to the audio-visual effects generated in the user terminal. For example, special effects include audio-visual effects for guiding the user's sight line to a specific area of ​​the content displayed by the user terminal, or audio-visual effects corresponding to the concentration of the user's sight line. The way of assigning special effects refers to the information required to generate special effects in the user terminal, for example, including the composition of special effects, the display position of special effects in the content, the volume of special effects, or the output timing of special effects. The composition of special effects includes the visual composition (display composition) and auditory composition (sound composition) of special effects. Electronic data representing the way of assigning special effects is called special effect data. The user terminal of the object to which the special effects are assigned is called the object terminal. That is, the content publishing system determines the way of assigning special effects to the content displayed on the object terminal.

[0031] [System Structure]

[0032] Figure 1 This is a diagram showing an example of application of a content publishing system 1 according to an embodiment. In this embodiment, the content publishing system 1 has a server 10. The server 10 is a computer that publishes content data. The server 10 is connected to a plurality of viewer terminals 20 and a publisher terminal 30 via a communication network N. The number of viewer terminals 20 is not limited as long as it is two or more. The number of publisher terminals 30 is not limited as long as it is one or more. The server 10 is also connected to a user database 40 via the communication network N. The structure of the communication network N is not limited. For example, the communication network N may be configured to include the Internet, or may be configured to include an intranet.

[0033] The viewer terminal 20 is a computer used by the viewer. In one example, the viewer terminal 20 has a function of accessing the content distribution system 1 to receive and display content data. The type and structure of the viewer terminal 20 are not limited. For example, the viewer terminal 20 can be a portable terminal such as a high-performance portable phone (smartphone), a tablet terminal, a wearable terminal (for example, a head-mounted display (HMD), smart glasses, etc.), a laptop personal computer, a portable phone, etc. Alternatively, the viewer terminal 20 can also be a fixed terminal such as a desktop personal computer.

[0034] The publisher terminal 30 is a computer used by the publisher. In one example, the publisher terminal 30 has the function of shooting an image and the function of accessing the content publishing system 1 and sending electronic data (image data) representing the image. The type and structure of the publisher terminal 30 are not limited. For example, the publisher terminal 30 can be a shooting system with the functions of shooting, recording and sending images. Alternatively, the publisher terminal 30 can also be a portable terminal such as a high-performance portable phone (smartphone), a tablet terminal, a wearable terminal (for example, a head-mounted display (HMD), smart glasses, etc.), a laptop personal computer, a portable phone, etc. Alternatively, the publisher terminal 30 can also be a fixed terminal such as a desktop personal computer.

[0035] The viewer operates the viewer terminal 20 and logs in to the content distribution system 1, thereby the viewer can view the content. The publisher operates the publisher terminal 30 and logs in to the content distribution system 1, thereby providing the viewer with the content. In this embodiment, it is assumed that the viewer and the publisher of the content distribution system 1 have logged in. It is not necessary for the viewer to log in to the content distribution system 1. That is, the content distribution system 1 can send content data to the viewer terminal 20 of an ordinary viewer who has not logged in. In this case, the ordinary viewer who has not logged in can also view the content.

[0036] The user database 40 is a non-temporary storage device that stores data used by the content publishing system 1. The user database 40 stores attribute information of users (e.g., viewers) who use the content publishing system 1. Attribute information is various information related to the user, and may include, for example, the user's identifier (user ID), age, gender, identifiers of the content that the user has viewed (content ID), and the number of times the content has been viewed. Attribute information may also include information arbitrarily selected or set by the user. For example, attribute information may include information about a favorite person or character selected by the user, representative colors corresponding to the person or character, and the like. Attribute information may also be stored in areas other than the user database 40. For example, attribute information may be stored in the cookie area of ​​the browser of the viewer terminal 20. Attribute information may also include information indicating whether the user has logged in to the content publishing system 1.

[0037] The user database 40 may be constructed as a single database or a collection of multiple databases. The location where the user database 40 is installed is not limited. For example, the user database 40 may be installed in a computer system different from the content distribution system 1.

[0038] Figure 2 This is a diagram showing an example of a hardware configuration related to the content distribution system 1 . Figure 2 A server computer 100 that functions as a server 10 and a terminal computer 200 that functions as a viewer terminal 20 or a publisher terminal 30 are shown.

[0039] As an example, the server computer 100 has a processor 101, a main storage unit 102, an auxiliary storage unit 103, and a communication unit 104 as hardware components. The processor 101 is a computing device that executes an operating system and an application program, such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The main storage unit 102 is a device that stores programs to be executed, calculation results, etc., and is composed of, for example, a ROM (Read Only Memory) or a RAM (Random Access Memory). The auxiliary storage unit 103 is generally a device that can store a larger amount of data than the main storage unit 102, and is composed of, for example, a non-volatile storage medium such as a hard disk or a flash memory. The auxiliary storage unit 103 stores a server program P1 and various data for enabling the server computer 100 to function as a server 10. The communication unit 104 is a device that performs data communication with other computers via a communication network N, and is composed of, for example, a network card or a wireless communication module.

[0040] In this embodiment, the content distribution program is installed as a server program P1. Each functional element of the server 10 is realized by reading the server program P1 into the processor 101 or the main storage unit 102 and causing the processor 101 to execute the program. The server program P1 includes codes for realizing each functional element of the server 10. The processor 101 operates the communication unit 104 according to the server program P1 and executes the reading and writing of data in the main storage unit 102 or the auxiliary storage unit 103.

[0041] The server 10 may be composed of one or more computers. When a plurality of computers are used, these computers are connected to each other via a communication network N, thereby logically forming one server 10.

[0042] As an example, the terminal computer 200 has a processor 201, a main storage unit 202, an auxiliary storage unit 203, a communication unit 204, an input interface 205, an output interface 206, and a camera unit 207 as hardware components. The processor 201 is a computing device that executes an operating system and an application program, such as a CPU or a GPU. The main storage unit 202 is a device that stores programs to be executed, calculation results, etc., and is composed of, for example, a ROM or a RAM. The auxiliary storage unit 203 is generally a device that can store a larger amount of data than the main storage unit 202, and is composed of, for example, a non-volatile storage medium such as a hard disk or a flash memory. The auxiliary storage unit 203 stores a client program P2 and various data for enabling the terminal computer 200 to function as a viewer terminal 20 or a publisher terminal 30. The communication unit 204 is a device that performs data communication with other computers via a communication network N, and is composed of, for example, a network card or a wireless communication module. The input interface 205 is a device that receives data based on the operation or action of the user, and is composed of, for example, at least one of a keyboard, an operation button, a pointing device, a touch panel, a microphone, a sensor, and a camera. The output interface 206 is a device that outputs data processed by the terminal computer 200, and is composed of, for example, a display device such as a monitor, a touch panel, and an HMD. The camera unit 207 is a device that captures an image (video or photo) depicting the real world, and is, for example, a camera. The camera unit 207 can also function as the input interface 205.

[0043] Each functional element of the viewer terminal 20 or the publisher terminal 30 is realized by reading the corresponding client program P2 into the processor 201 or the main storage unit 202 and executing the program by the processor 201. The client program P2 includes a code for realizing each functional element of the viewer terminal 20 or the publisher terminal 30. The processor 201 operates the communication unit 204, the input interface 205, the output interface 206, or the camera unit 207 according to the client program P2, and reads and writes data from the main storage unit 202 or the auxiliary storage unit 203.

[0044] At least one of the server program P1 and the client program P2 may be provided on the basis of being non-temporarily recorded on a tangible recording medium such as a CD-ROM, a DVD-ROM, a semiconductor memory, etc. Alternatively, at least one of these programs may be provided as a data signal superimposed on a carrier wave via a communication network N. These programs may be provided separately or together.

[0045] Figure 31 is a diagram showing an example of a functional structure associated with a content distribution system 1. The server 10 has a content transmission unit 11, an acquisition unit 12, a calculation unit 13, a determination unit 14, a decision unit 15, and a special effect sending unit 16 as functional elements. The content transmission unit 11 is a functional element that receives content data sent from a publisher terminal 30 and sends the content data to a viewer terminal 20. The acquisition unit 12 is a functional element that acquires line of sight data from the viewer terminal 20. The calculation unit 13 is a functional element that analyzes the line of sight data acquired by the acquisition unit 12 and calculates the distribution of line of sight positions. The determination unit 14 is a functional element that determines a first area in the content where the user's line of sight is concentrated and a second area where the user's line of sight is not concentrated compared to the first area based on the distribution of line of sight positions calculated by the calculation unit 13. The decision unit 15 is a functional element that determines the way of giving special effects based on the distribution of line of sight positions. The special effect sending unit 16 is a functional element that sends special effect data to the viewer terminal 20.

[0046] The viewer terminal 20 has a content receiving unit 21, a line of sight determination unit 22, a line of sight data sending unit 23, a special effect receiving unit 24, and a display control unit 25 as functional elements. The content receiving unit 21 is a functional element that receives content data from the server 10. The line of sight determination unit 22 is a functional element that determines the line of sight position according to the movement of the viewer's eyes. In the present embodiment, the line of sight determination unit 22 constitutes an eye tracking system. The line of sight data sending unit 23 is a functional element that sends line of sight data indicating the line of sight position to the server 10. The special effect receiving unit 24 is a functional element that receives special effect data from the server 10. The display control unit 25 is a functional element that controls the display of the screen of the viewer terminal 20.

[0047] The publisher terminal 30 includes a content transmission unit 31 as a functional element. The content transmission unit 31 is a functional element for transmitting content data to the server 10.

[0048] [System Action]

[0049] Reference Figure 4 The operation of the content distribution system 1 will be described, and the content distribution method of this embodiment will be described. Figure 4 This is a sequence diagram showing an example of content distribution processing as a processing flow S1.

[0050] In step S101, the content transmission section 31 transmits content data to the server 10. For example, when the publisher operates the publisher terminal 30 to publish content to viewers, the content transmission section 31 transmits content data to the server 10 in response to the operation. The content transmission section 11 receives content data transmitted from the content transmission unit 31.

[0051] In step S102, the content transmission unit 11 transmits content data to a plurality of viewer terminals 20. The content transmission unit 11 transmits content data in response to a content request from the viewer terminal 20. The content request is a data signal for requesting the server 10 to reproduce content. For example, when a viewer operates the viewer terminal 20 in order to start reproducing desired content, the viewer terminal 20 generates a content request in response to the operation and transmits it to the server 10. The content transmission unit 11 transmits content data in response to the content request. The content receiving unit 21 receives the content data transmitted from the content transmission unit 11.

[0052] In step S103, the viewer terminal 20 displays the content. When the content receiving unit 21 receives the content data, the display control unit 25 processes the content data and displays the content on the screen. The viewer terminal 20 can output sound from the speaker in conjunction with the display of the content.

[0053] In step S104, the line of sight determination unit 22 determines the line of sight position of the viewer. The line of sight determination unit 22 determines the line of sight position on the screen of the viewer terminal 20 according to the movement of the viewer's eyes. The line of sight determination unit 22 does not limit the method for determining the line of sight position. As an example, the line of sight determination unit 22 can capture an image of the periphery of the viewer's eyes through the camera unit 207 of the viewer terminal 20, and determine the line of sight position according to the position of the iris with the inner corner of the user's eye as a reference point. As another example, the line of sight determination unit 22 can also use the corneal reflection method (PCCR) to determine the line of sight position of the viewer. When the corneal reflection method is used, the viewer terminal 20 can have an infrared emitting device and an infrared camera as a hardware structure. The line of sight position can be expressed, for example, using a two-dimensional coordinate system that can uniquely determine the position on the screen of the viewer terminal 20.

[0054] In step S105, the line of sight data sending unit 23 sends the line of sight data indicating the line of sight position to the server 10. The line of sight data indicating the line of sight position is not limited to data indicating the coordinate position of the line of sight on the screen. For example, the viewer terminal 20 may also obtain data indicating that the viewer is visually confirming a specific area in the content (for example, a specific object contained in the content) as line of sight data, and send the line of sight data obtained by the line of sight data sending unit 23 to the server 10. The acquisition unit 12 receives (acquires) the line of sight data sent by the line of sight data sending unit 23. The acquisition unit 12 acquires line of sight data from two or more viewer terminals 20 among a plurality of viewer terminals 20. That is, the acquisition unit 12 acquires two or more line of sight data.

[0055] In step S106, the calculation unit 13 calculates the distribution of the sight lines of two or more viewers in the content. The calculation unit 13 analyzes the two or more sight line data acquired by the acquisition unit 12 to calculate the distribution of the sight line positions. The calculation unit 13 does not limit the calculation method of the distribution of the sight line positions. For example, the calculation unit 13 can divide the content into a plurality of small areas, and calculate the number of sight lines in each small area as the distribution of the sight line positions based on the acquired sight line data.

[0056] In step S107 , the identification unit 14 identifies a first area in the content where the viewer's line of sight is focused and a second area where the viewer's line of sight is less focused than the first area. The identification unit 14 identifies the first area and the second area based on the distribution of line of sight positions calculated by the calculation unit 13 .

[0057] The determination unit 14 does not limit the determination method of the first area and the second area. For example, the determination unit 14 divides the content into multiple small areas. The determination unit 14 may determine the small area as the first area when the number of sight lines within the small area is greater than a specified first threshold. The determination unit 14 may determine the small area as the second area when the number of sight lines within the small area is less than a specified second threshold that is smaller than the first threshold. The determination unit 14 may determine a plurality of continuous small areas as one first area when all of them are the first area. Similarly, the determination unit 14 may determine a plurality of continuous small areas as one second area when all of them are the second area. The shape and size of the first area and the second area are not limited. The first area and the second area may be polygonal, circular, or asymmetrical irregular shapes.

[0058] In step S108, the decision unit 15 determines the method of adding special effects to the content based on the distribution of the eye-line positions. As the method of adding special effects, the decision unit 15 determines, for example, the composition of the special effects, the display position of the special effects in the content, the volume of the special effects, or the output timing of the special effects.

[0059] In the case where the special effect is a visual effect, the composition of the special effect may be, for example, a change in the display mode of a portion of the content. As an example, the change in the display mode may be to add a pattern (for example, a star, heart, or emoticon mark, etc.) or a character string to a portion of the content, or to surround a portion of the content with a frame line. The change in the display mode may also be to display a portion of the content brighter or darker than other portions. The change in the display mode may also be to display the content in the form of enlarging or reducing a portion of the content. The change in the display mode may also be to display the content in the form of causing a portion of the content to perform a specified action (for example, vibrate or rotate, etc.).

[0060] The composition of the special effect may also be to change the display method of the comments included in the content. The comments may be comments posted by multiple viewers via the viewer terminal 20. In this case, the content publishing system 1 includes the comments posted from a certain viewer terminal 20 in the content, and also displays the comments on other viewer terminals 20. Therefore, viewers can read not only the comments posted by themselves, but also the comments provided by other viewers. The change in the display method of the comments may be to add a pattern to the comments of the object, to change the color of the comments of the object to a specific color, or to change the font of the comments of the object to a specific font. The change in the display method of the comments may also be to display the comments of the object larger than other comments. The case of changing the display method of the comments is included in the case of changing the display method of a part of the content mentioned above.

[0061] When the special effect is an auditory effect, the special effect may be composed of generating a sound associated with the content. For example, the special effect may be composed of generating the sound of a person or character included in the content, or generating a sound related to an object included in the content (such as an engine sound when the content includes a car), or generating a sound of reading a comment included in the content.

[0062] As described above, the determination unit 15 determines the way to give special effects based on the distribution of the sight line positions. The determination unit 15 can use information representing the first area and the second area determined based on the distribution of the sight line positions to determine the way to give special effects. As an example, the determination unit 15 can determine to change the display method of the first area as the way to give special effects. In this case, the composition of the special effect can be to add a pattern or a character string to the first area, or to display the comments located in the first area larger than other comments. The composition of the special effect can also be to display information indicating the number or proportion of viewers who visually confirm the first area. The information indicating the number or proportion of viewers is not limited to a specific numerical value. For example, the icon contained in the first area is displayed in a manner that performs a prescribed action (for example, vibration or rotation, etc.), and the prescribed action indicates that the number or proportion of viewers who visually confirm the first area is large. The number or proportion of viewers who visually confirm the first area is calculated based on the distribution of the sight line positions.

[0063] As another example, the decision unit 15 may determine that the special effect is provided by generating a sound associated with the first area. In this case, the special effect may be configured by generating a sound of a person or character located in the first area, generating a sound related to an object located in the first area, or generating a sound of reading a comment located in the first area.

[0064] By giving the above-mentioned special effect related to the first area, the sight is further focused on the first area where the sight is already focused. That is, the above-mentioned special effect can be said to be an audio-visual effect for guiding the viewer's sight to the first area in the content. The above-mentioned special effect is a special effect associated with the first area where the viewer's sight is focused, and can also be said to be an audio-visual effect corresponding to the concentration of the viewer's sight.

[0065] The decision unit 15 may also decide to change the display mode of the second area as the way of adding the special effect. The composition of the special effect in this case is the same as that of the case of changing the display mode of the first area. That is, for example, the composition of the special effect may be to add a pattern or a character string to the second area, or to display the comments located in the second area larger than other comments. The composition of the special effect may also be to display information indicating the number or ratio of viewers who visually confirm the second area.

[0066] The decision unit 15 may also decide to generate a sound associated with the second area as a way of providing the special effect. The composition of the special effect in this case is the same as the case of generating a sound associated with the first area. That is, for example, the composition of the special effect may be to generate a sound of a person or character located in the second area, to generate a sound related to an object located in the second area, or to generate a sound of reading a comment located in the second area.

[0067] By giving the special effect related to the second area as described above, the line of sight is focused on the second area where the line of sight is not focused. That is, the special effect described above can be said to be an audio-visual effect for guiding the viewer's line of sight to the second area in the content. The special effect described above is a special effect associated with the second area where the viewer's line of sight is not focused, and can also be said to be an audio-visual effect corresponding to the concentration of the viewer's line of sight.

[0068] In step S109, the special effect sending unit 16 sends the special effect data indicating the method of applying the special effect to the viewer terminal 20. At this time, the special effect sending unit 16 may not send the special effect data to all viewer terminals 20, but may send the special effect data only to the viewer terminal 20 (target terminal) to which the special effect is to be applied. The special effect receiving unit 24 of the viewer terminal 20 to which the special effect data is sent receives the special effect data.

[0069] In step S110, the viewer terminal 20 adds a special effect to the content. When the special effect is a visual effect, the display control unit 25 processes the special effect data and displays the special effect on the screen. When the special effect is an auditory effect, the viewer terminal 20 outputs the special effect from the speaker.

[0070] Reference Figure 5 and Figure 6 , an example of how to impart special effects is described. Figure 5 This is a diagram showing an example of content before adding special effects. Figure 6 This is a diagram showing an example of content after adding special effects.

[0071] exist Figure 5 and Figure 6 In the example of FIG. 1 , the viewer terminal 20 displays a video of two singers performing a live concert as content C1. Content C1 includes comments contributed by a plurality of viewers. In this example, comments such as “Wave!”, “Go!”, and “Cute!” are displayed.

[0072] First, the determination unit 14 determines a first area in the content C1 where the viewer's line of sight is focused and a second area where the viewer's line of sight is less focused than the first area, based on the distribution of the line of sight positions. In this example, the determination unit 14 determines the area A1 where the comment "Wave!" is displayed and the area A3 where one singer is displayed as the first area, and determines the area A2 where the comment "Go for it!" is displayed and the area A4 where another singer is displayed as the second area.

[0073] Afterwards, the decision unit 15 determines the method of assigning special effects based on the distribution of the sight line positions. In this example, the decision unit 15 determines the following as the method of assigning special effects: the comments included in the area A2 determined as the second area are displayed larger than other comments, and a star mark is added to the comment; and a star and a heart mark are added to the area A4 determined as the second area. In addition, the decision unit 15 determines the sound of the singer (for example, a sound such as "Look here!") produced by the area A4 determined as the second area as the method of assigning special effects. The special effect sending unit 16 sends the special effect data indicating the method of assigning special effects determined by the decision unit 15 to the viewer terminal 20 (target terminal).

[0074] The special effect receiving unit 24 receives the special effect data, and the viewer terminal 20 processes the special effect data to generate special effects. In this example, the display control unit 25 displays the comments included in area A2 larger than other comments, and adds a star or other mark to area A2 and area A4. The viewer terminal 20 emits the voice of the singer in area A4 from the speaker.

[0075] Reference Figure 7 , another example of how to impart special effects is described. Figure 7 is a diagram showing another example of content after adding special effects. Figure 7 In the example of , the viewer terminal 20 displays a questionnaire image including a character string constituting the question “Which season do you like best?” and four icons constituting options of “Spring”, “Summer”, “Autumn”, and “Winter” as content C2.

[0076] First, the determination unit 14 determines a first area in the content C2 where the viewer's line of sight is focused and a second area where the viewer's line of sight is less focused than the first area, based on the distribution of the line of sight positions. In this example, the determination unit 14 determines the area B2 including the "summer" icon as the first area, and determines the areas B1, B3, and B4 including the "spring", "autumn", and "winter" icons, respectively, as the second areas.

[0077] Then, the determination unit 15 determines the manner of applying the special effect according to the distribution of the sight line positions. In this example, the determination unit 15 determines the following as the manner of applying the special effect: the "summer" icon included in the area B2 determined as the first area is displayed in a vibrating manner; and the proportion of the sight lines located in each area B1 to B4 is displayed on the corresponding icon. That is, in this example, the icon to which the viewer pays special attention (the sight line is focused) is displayed in a vibrating manner. In addition, in this example, the degree of attention (the proportion of the sight line) from the viewer to each icon is displayed as a numerical value. The proportion of the sight line is calculated based on the distribution of the sight line positions. In this example, the character string "20%" indicating the proportion of the sight line located in the area B1 is displayed on the "spring" icon, the character string "40%" indicating the proportion of the sight line located in the area B2 is displayed on the "summer" icon, the character string "10%" indicating the proportion of the sight line located in the area B3 is displayed on the "autumn" icon, and the character string "30%" indicating the proportion of the sight line located in the area B4 is displayed on the "winter" icon. The special effect transmitting unit 16 transmits special effect data indicating the method of providing the special effect determined by the determining unit 15 to the viewer terminal 20 (target terminal).

[0078] The special effect receiving unit 24 receives the special effect data, and the viewer terminal 20 processes the special effect data to generate the special effect. In this example, the display control unit 25 displays the "summer" icon included in the area B2 in a vibrating manner, and displays a character string indicating the ratio of the line of sight on the corresponding icon.

[0079] [Modifications]

[0080] Various examples of the present disclosure have been described in detail above. However, the present disclosure is not limited to the above examples. Various modifications can be made to the present disclosure without departing from the scope of the present invention.

[0081] The content distribution system 1 can send content data not only to the viewer terminal 20 but also to the publisher terminal 30. The publisher terminal 30 processes the content data and displays the content on the screen. In this case, the content distribution system 1 can also send special effect data to the publisher terminal 30. The publisher terminal 30 processes the special effect data and adds special effects to the content. That is, the publisher terminal 30 can also function as the viewer terminal 20 (target terminal).

[0082] The way of assigning special effects is not limited to the above-mentioned examples. The decision unit 15 may also determine the way of assigning special effects based on the attribute information of the viewer. The decision unit 15 may, for example, determine the way of assigning special effects based on attribute information such as age or gender. Specifically, the decision unit 15 may determine the following as the way of assigning special effects: the comments included in the first area where the sight of older viewers is focused are displayed larger than other comments and displayed in a conspicuous color. In the case where the decision unit 15 determines the way of assigning special effects based on the attribute information of the viewer, the calculation unit 13 may classify the viewers into a plurality of groups based on the attribute information, and calculate the distribution of sight line positions according to each group. For example, the calculation unit 13 may calculate the distribution of sight line positions according to the age group or gender of the viewer.

[0083] When calculating the distribution of sight lines, the calculation unit 13 may weight the sight line data according to the attribute information of the viewer. For example, the calculation unit 13 may calculate the distribution of sight lines using a value obtained by multiplying the number of sight lines of viewers of a specific age group or gender by a predetermined coefficient.

[0084] The decision unit 15 may also determine the manner of providing the special effect based on the attribute information of the viewer (hereinafter referred to as the subject) using the target terminal. Specifically, when the attribute information includes information about a favorite person or character selected by the subject and a representative color corresponding to the person or character, the decision unit 15 may set the sound generated as the special effect to the sound of the person or character selected by the subject, or may set the color of the pattern added to the content as the special effect to the representative color corresponding to the person or character.

[0085] When multiple comments contributed by viewers are included in the content and displayed, the decision unit 15 may determine the following as a method of providing a special effect: only a predetermined number of comments on which the line of sight is largely focused among the multiple comments are displayed, and other comments are not displayed. In this case, the decision unit 15 determines the comments on which the line of sight is largely focused based on the distribution of the line of sight positions.

[0086] The determination unit 15 may also determine the following as the method of providing the special effect: displaying the distribution of the sight line positions on the screen of the user terminal in a form such as a heat map that the user can visually confirm. In this case, the determination unit 15 may determine the method of providing the special effect without determining the first area and the second area through the determination unit 14.

[0087] When the content distribution system 1 is used for on-demand distribution, the acquisition unit 12 acquires sight line data each time the content is viewed by each viewer, and stores the acquired sight line data in the database. The calculation unit 13 can analyze the accumulated two or more sight line data to calculate the distribution of sight line positions. That is, the timing at which the acquisition unit 12 acquires sight line data from each viewer terminal 20 does not have to be simultaneous, and can be different.

[0088] When the content distribution system 1 is used for on-demand distribution, the content distribution system 1 may store data indicating the history of comments posted by a certain viewer in a database in advance, and when other viewers view content, the comments may be included in the content and displayed based on the data.

[0089] In the case where each viewer terminal 20 displays content at a different time, the viewer terminal 20 may send data indicating the time when the line of sight data is acquired (e.g., data indicating the progress of the content) together with the line of sight data to the server 10. The case where each viewer terminal 20 displays content at a different time refers to, for example, a case where the content publishing system 1 is used for on-demand publishing, or a case where the timing of displaying the content is delayed (lagged), etc. The calculation unit 13 of the server 10 can calculate the distribution of line of sight positions according to each scene included in the content based on the line of sight data and the data indicating the time when the line of sight data is acquired. In the case where each viewer terminal 20 displays content at no different time, the viewer terminal 20 may not send data indicating the time when the line of sight data is acquired to the server 10. The case where each viewer terminal 20 displays content at no different time refers to, for example, a case where the content publishing system 1 is used for real-time publishing, or a case where the timing of displaying the content is not delayed (lagged), etc.

[0090] The processing of applying special effects may also be performed by the server 10. For example, the server 10 may change the composition of the content (such as changing it to a part of the content displayed with emphasis) according to the method of applying special effects determined by the determination unit 15, and transmit content data representing the changed content to the viewer terminal 20. In this case, the display control unit 25 of the viewer terminal 20 processes the content data and displays the content, thereby displaying the content with the special effects applied on the screen.

[0091] In the present disclosure, an expression such as "at least one processor executes a first process, executes a second process, ... executes an nth process" or an expression corresponding thereto indicates a concept including a case where the execution subject (i.e., the processor) of n processes from the first process to the nth process is changed in the middle. That is, the expression indicates a concept including both a case where all n processes are executed by the same processor and a case where the processor is changed in an arbitrary direction in the n processes.

[0092] The processing process of the method executed by at least one processor is not limited to the examples of the above-mentioned embodiments. For example, part of the above-mentioned steps may be omitted, or the steps may be executed in another order. Furthermore, any two or more of the above-mentioned steps may be combined, or part of the steps may be modified or deleted. Alternatively, other steps may be executed in addition to the above-mentioned steps.

[0093] In comparing the magnitude relationship of two numerical values, either of the two criteria of “or more” and “greater than” may be used, or either of the two criteria of “or less” and “less than” may be used.

[0094] Any part or all of the functional units described in this specification may be implemented by a program. The program mentioned in this specification may be released by recording it non-temporarily on a computer-readable recording medium, or released via a communication line such as the Internet (including wireless communication), or released in a state where it is installed on any terminal.

[0095] Based on the above description, those skilled in the art may think of additional effects or various modifications of the present disclosure, but the present disclosure is not limited to the above-mentioned embodiments. Various additions, changes and partial deletions can be made within the scope of the conceptual ideas and the main purpose of the present disclosure derived from the contents specified in the claims and their equivalents.

[0096] For example, a structure described in this specification as a single device (or component, the same below) (including a structure depicted as a single device in the drawings) may also be implemented by multiple devices. Alternatively, a structure described in this specification as a single device (including a structure depicted as a single device in the drawings) may also be implemented by a single device. Alternatively, a part or all of a unit or function included in a device (such as a server) may also be included in another device (such as a user terminal).

[0097] Not all matters described in this specification are essential requirements. For example, matters described in this specification but not described in the claims can be regarded as optional additional matters.

[0098] The applicant is only aware of the known technologies recorded in the "Prior Art Documents" column of this specification. It should also be noted that the present disclosure is not necessarily intended to solve the problems of the known technologies. The entirety of this specification should be considered to identify the problems that the present disclosure is intended to solve. For example, in the case where this specification records that a specified effect is achieved through a specific structure, it can also be said that the problem corresponding to the specified effect can be solved. However, the record of the effect is not necessarily intended to take such a specific structure as a necessary requirement.

[0099] [Note]

[0100] As can be understood from the above-mentioned various examples, the present disclosure includes the following aspects.

[0101] (Note 1)

[0102] A content publishing system, comprising at least one processor, which performs the following actions: displays content on multiple user terminals, obtains line of sight data representing line of sight positions of users who visually confirm the content from more than two of the multiple user terminals, analyzes the two or more obtained line of sight data to calculate the distribution of line of sight positions of more than two users in the content, and determines the method of assigning special effects to the content displayed on at least one target terminal among the multiple user terminals based on the calculated distribution.

[0103] (Note 2)

[0104] According to the content publishing system described in Note 1, the at least one processor performs the following actions: determining a first area in the content where the sights of the two or more users are focused and a second area where the sights of the two or more users are not focused compared to the first area based on the distribution, and changing the display method of the first area of ​​the content displayed on the at least one object terminal as the method of assigning the special effect.

[0105] (Note 3)

[0106] According to the content publishing system described in Note 1, the at least one processor performs the following actions: determining a first area in the content where the sights of the two or more users are focused and a second area where the sights of the two or more users are not focused compared to the first area based on the distribution, and changing the display method of the second area of ​​the content displayed on the at least one object terminal as the method of assigning the special effect.

[0107] (Note 4)

[0108] According to the content distribution system described in any one of Supplements 1 to 3, the multiple user terminals include a publisher terminal that publishes the content and multiple viewer terminals, and the content is distributed from the publisher terminal to the multiple viewer terminals in real time.

[0109] (Note 5)

[0110] According to the content distribution system described in any one of Supplements 1 to 3, the multiple user terminals include multiple viewer terminals, and the content is content distributed to the multiple viewer terminals on demand.

[0111] (Note 6)

[0112] According to the content publishing system described in any one of Supplements 1 to 5, the at least one processor determines the way of assigning the special effect based on the attribute information of the multiple users.

[0113] (Note 7)

[0114] A content publishing method is executed by a content publishing system having at least one processor, wherein the content publishing method includes the following steps: displaying content on multiple user terminals; acquiring line of sight data representing the line of sight positions of users who visually confirm the content from more than two user terminals among the multiple user terminals; analyzing the acquired more than two line of sight data to calculate the distribution of the line of sight positions of more than two users in the content; and determining a method of assigning special effects to the content displayed on at least one object terminal among the multiple user terminals based on the calculated distribution.

[0115] (Note 8)

[0116] A content publishing program that causes a computer to execute the following steps: displaying content on multiple user terminals; acquiring line of sight data representing the line of sight positions of users who visually confirm the content from two or more user terminals among the multiple user terminals; analyzing the two or more acquired line of sight data to calculate the distribution of the line of sight positions of two or more users in the content; and determining a method of assigning special effects to the content displayed on at least one target terminal among the multiple user terminals based on the calculated distribution.

[0117] According to Supplementary Notes 1, 7, and 8, the distribution of the gaze positions of two or more users in the content is obtained by analyzing two or more gaze data, and the method of assigning special effects to the content is determined based on the distribution. Through this mechanism, the method of assigning special effects is determined based on the reactions of multiple viewers to the content, thereby increasing the attractiveness of the content to the viewers. In one example, through this mechanism, the reactions of the viewers are reflected in the content, and interactive content is provided, thereby promoting communication between viewers and publishers and between viewers.

[0118] According to Supplementary Note 2, the display mode of the first area where the user's line of sight is focused is changed. This can provide, for example, a special effect for guiding the viewer's line of sight to the first area, and can further focus the viewer's line of sight on the first area.

[0119] According to Supplementary Note 3, the display mode of the second area where the user's line of sight is less focused than the first area is changed. Thus, for example, a special effect for guiding the viewer's line of sight to the second area can be given, and the line of sight spread over the first area can be dispersed so that the line of sight is also focused on the second area.

[0120] According to Supplementary Note 4, special effects are added to the content delivered in real time. This can increase the appeal of the content delivered in real time to viewers.

[0121] According to Supplementary Note 5, special effects are added to the content distributed on demand, thereby making it possible to increase the appeal of the content distributed on demand to viewers.

[0122] According to Supplementary Note 6, the method of applying special effects is determined based on the attribute information of the viewer, thereby further enhancing the appeal of the content to the viewer.

[0123] Description of symbols

[0124] 1: Content publishing system; 10: Server; 11: Content transmission unit; 12: Acquisition unit; 13: Calculation unit; 14: Determination unit; 15: Decision unit; 16: Special effect sending unit; 20: Viewer terminal; 21: Content receiving unit; 22: Line of sight determination unit; 23: Line of sight data sending unit; 24: Special effect receiving unit; 25: Display control unit; 30: Publisher terminal; 31: Content sending unit; 40: User database; A1, A2, A3, A4, B1, B1, B2, B3, B4: Area; C1, C2: Content; P1: Server program; P2: Client program.

Claims

1. A content publishing system having at least one processor, The at least one processor performs the following actions: Display the content on multiple user terminals. acquiring, from two or more user terminals among the plurality of user terminals, line of sight data indicating line of sight positions of users who visually check the content, The two or more acquired sight line data are analyzed to calculate the distribution of sight line positions of two or more users in the content, A method of providing a special effect to the content displayed on at least one target terminal among the plurality of user terminals is determined based on the calculated distribution.

2. The content distribution system according to claim 1, wherein: The at least one processor performs the following actions: determining, based on the distribution, a first area in the content on which the eyes of the two or more users are focused and a second area on which the eyes of the two or more users are less focused than the first area, The method of providing the special effect is determined as changing the display method of the first area of ​​the content displayed on the at least one target terminal.

3. The content distribution system according to claim 1, wherein: The at least one processor performs the following actions: determining, based on the distribution, a first area in the content on which the two or more users' eyes are focused and a second area on which the two or more users' eyes are less focused than the first area, The method of providing the special effect is determined as changing the display method of the second area of ​​the content displayed on the at least one target terminal.

4. The content distribution system according to any one of claims 1 to 3, wherein: The plurality of user terminals include a publisher terminal that publishes the content and a plurality of viewer terminals. The content is content distributed from the publisher terminal to the plurality of viewer terminals in real time.

5. The content distribution system according to any one of claims 1 to 3, wherein: The plurality of user terminals include a plurality of viewer terminals, The content is content distributed to the multiple viewer terminals on demand.

6. The content distribution system according to any one of claims 1 to 3, wherein: The at least one processor determines a method of imparting the special effect according to the attribute information of the multiple users.

7. A content publishing method, executed by a content publishing system having at least one processor, wherein: The content publishing method comprises the following steps: Displaying the content on multiple user terminals; acquiring, from two or more of the plurality of user terminals, line of sight data indicating line of sight positions of users visually checking the content; Analyzing the acquired two or more sight line data to calculate the distribution of sight line positions of two or more users in the content; as well as A method of providing a special effect to the content displayed on at least one target terminal among the plurality of user terminals is determined based on the calculated distribution.

8. A content distribution program, which causes a computer to execute the following steps: Displaying the content on multiple user terminals; acquiring, from two or more of the plurality of user terminals, line of sight data indicating line of sight positions of users visually checking the content; Analyzing the acquired two or more sight line data to calculate the distribution of sight line positions of two or more users in the content; and A method of providing a special effect to the content displayed on at least one target terminal among the plurality of user terminals is determined based on the calculated distribution.

Images