Video processing method, and video processing system

Abstract

To provide a video processing method capable of facilitating analysis of viewer evaluations of video content.SOLUTION: The video processing method includes the steps: acquiring video content delivered by a first server (S10); acquiring images for analyzing viewers of video content (an image of a viewer in front of a display device on which the video content is being played, or an image showing part of the video content posted by a viewer on a social networking service); identifying a corresponding frame in the video content that corresponds to the image for analyzing (S122); and associating the identified corresponding frames and the images for analyzing by the elapsed time in the video content being played (S123).SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to a video processing method and a video processing system. [Background technology]

[0002] Patent Document 1 discloses a data processing system. In this data processing system, the data processing device includes an acquisition unit, a calculation unit, and an output unit. The acquisition unit acquires data indicating the timing of a user's blink. The calculation unit calculates an index according to the degree of coincidence between the timing of the blinks of a plurality of users based on the difference between the timing of the blinks of the plurality of users. The output unit outputs data according to the index. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-169805 A Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a video processing method and the like that makes it easier to analyze viewers' evaluations of video content. [Means for solving the problem]

[0005] In a video processing method according to an aspect of the present disclosure, video content is acquired. In the video processing method, an analysis image for analyzing a viewer of the video content is acquired. In the video processing method, a corresponding frame in the video content that corresponds to the analysis image is identified. In the video processing method, the identified corresponding frame and the analysis image are linked by an elapsed time in the video content being played. Effect of the Invention

[0006] The present disclosure has an advantage in that it makes it easier to analyze viewers' evaluations of video content. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating an overview of a video processing method according to a comparative example. [Diagram 2] FIG. 2 is a block diagram showing an overall configuration including a video processing system according to an embodiment. [Diagram 3] FIG. 3 is a sequence diagram illustrating an example of the operation of the video processing system according to the embodiment. [Figure 4] FIG. 4 is a flowchart illustrating an example of a time analysis thread in the video processing system according to the embodiment. [Diagram 5] FIG. 5 is a flowchart illustrating an example of a viewing analysis thread in the video processing system according to the embodiment. [Figure 6] FIG. 6 is a flowchart illustrating an example of an output thread in the video processing system according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of output of evaluation information in the video processing system according to the embodiment. [Figure 8] FIG. 8 is a diagram showing another example of output of evaluation information in the video processing system according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating advantages of the video processing system according to the embodiment. [Figure 10] FIG. 10 is an explanatory diagram of an example of acquiring video content using a relay device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] [1. Findings that formed the basis of this disclosure] First, the inventor's viewpoint will be explained below.

[0009] For example, in facilities such as museums, video content may be played on a display device such as a display, and the played video content may be viewed by visitors. Hereinafter, these visitors, i.e., people who can view the video content, are referred to as "viewers." In other words, viewers are not limited to people who are viewing the video content, but also include people who are in a situation where they can view the video content but do not view it. The analysis results of the viewers' evaluation of the video content, such as which scenes in the video content the viewers were interested in, can be useful information for the video content producer when updating the video content or producing new video content. Therefore, it is conceivable to analyze the viewers' evaluation of the video content by capturing an image of the viewer in front of the display device on which the video content is played, and analyzing the captured image of the viewer.

[0010] Fig. 1 is an explanatory diagram of an overview of a video processing method of a comparative example. Fig. 1(a) is an explanatory diagram of an overview of a video processing method of a comparative example in a case where a single server 200 distributes video content and analyzes captured images of viewers.

[0011] As shown in (a) of Fig. 1, in the video processing method of the comparative example, a server 200 distributes video content to a display device 201. As a result, the video content is played on the display device 201. The server 200 also acquires an image captured by a camera 202 of a viewer A1 who is in front of the display device 201 on which the video content is being played. The server 200 then analyzes the acquired image to analyze the viewer A1's evaluation of the video content.

[0012] Specifically, the server 200 analyzes whether the viewer A1 is facing the display device 201, that is, whether the viewer A1 is watching the video content, by analyzing the face direction of the viewer A1 in the acquired image, etc. Then, the server 200 analyzes the viewer A1's evaluation of the video content by linking the elapsed time of the played video content with whether the viewer A1 is watching the video content during that elapsed time.

[0013] Here, in order to link the elapsed time in the reproduced video content with whether or not the viewer A1 is watching the video content at that elapsed time as described above, it is necessary to synchronize the time when the viewer A1 is imaged with the elapsed time in the reproduced video content. In the case shown in (a) of Fig. 1, one server 200 distributes the video content and analyzes the image captured by the camera 202, so that the start time of the reproduction of the video content can be shared in both the distribution process and the analysis process, making the above synchronization possible.

[0014] On the other hand, as shown in (b) of Figure 1, if the first server 203 that distributes the video content and the second server 204 that analyzes the captured image of the viewer A1 are different from each other, the distribution process and the analysis process cannot share the same start time, and therefore the above synchronization cannot be performed.

[0015] Therefore, in the video processing method of the comparative example, in the situation shown in (b) of Figure 1, it is not possible to link the elapsed time of the video content being played with whether or not viewer A1 is watching the video content during that elapsed time, making it difficult to analyze viewer A1's evaluation of the video content.

[0016] In view of the above, the inventors have created the present disclosure.

[0017] Hereinafter, the embodiments will be described with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, arrangement and connection forms of the components, steps, order of steps, and the like shown in the following embodiments are merely examples and are not intended to limit the present disclosure. In addition, among the components in the following embodiments, components that are not described in the independent claims will be described as optional components.

[0018] In addition, each drawing is a schematic diagram and is not necessarily a precise illustration. In addition, in each drawing, the same reference numerals are used for substantially the same configuration, and duplicated explanations may be omitted or simplified.

[0019] (Embodiment) [2. Configuration] [2-1. Overall composition] First, an overall configuration including a video processing system 100 according to an embodiment will be described. FIG. 2 is a block diagram showing an overall configuration including a video processing system 100 according to an embodiment. The video processing system 100 is a system for analyzing an evaluation of a viewer A1 on video content. In the embodiment, the video processing system 100 is realized by a second server 2. In the embodiment, the second server 2 (video processing system 100) is used together with a first server 1, a display device 3, a first camera 41 which is a display image acquisition device 4, and an image collection device 51 and a second camera 52 which are analysis image acquisition devices 5. The first server 1 and the display device 3 can communicate with each other via a network such as the Internet or a LAN (Local Area Network). In addition, the second server 2, the first camera 41, the image collection device 51, and the second camera 52 can communicate with each other via a network such as the Internet or a LAN.

[0020] The first server 1 is a server that distributes video content. In the embodiment, the first server 1 distributes the video content by transmitting a video signal including the video content to the display device 3. The video content is composed of, for example, moving images. Note that the video content may be composed of, for example, a plurality of still images. When the video content is composed of a plurality of still images, the display device 3 plays back the video content such that the displayed still images are switched over time.

[0021] The second server 2 is a server that analyzes the evaluation of the viewer A1 on the video content. The second server 2 will be described in detail in [2-2. Video Processing System] below.

[0022] The display device 3 is, for example, a display of a television receiver or the like, and plays video content included in a video signal transmitted from the first server 1. The display device 3 may be, for example, a screen that displays an image projected by a projector. The display device 3 may also be a structure other than a screen, such as a wall surface, as long as it has a display surface that displays an image projected by a projector.

[0023] The display image acquisition device 4 is a device that acquires an image (hereinafter also referred to as a "display image") displayed on the display device 3. In the embodiment, the display image acquisition device 4 is realized by a first camera 41. The first camera 41 is installed around the display device 3 so that the display device 3 falls within an imaging range, and captures an image displayed on the display device 3. The first camera 41 also outputs information including the captured image to the second server 2 (image processing system 100). Note that the communication between the first camera 41 and the second server 2 may be wired communication or wireless communication.

[0024] The analysis image acquisition device 5 is a device that acquires an analysis image for analyzing the viewer A1 of the video content. The analysis image is an image that indicates whether the viewer A1 is viewing the video content distributed by the first server 1. In the embodiment, the analysis image acquisition device 5 has an image collection device 51 and a second camera 52.

[0025] The image collection device 51 is, for example, a desktop or laptop personal computer, and is a device that executes an appropriate image collection algorithm to collect images posted on a social network service (SNS) and related to video content distributed by the first server 1. For example, the image collection device 51 collects images with metadata attached thereto, such as a hashtag, indicating that the images are related to the video content distributed by the first server 1, on the social network service. The image collection device 51 also outputs information including the collected images to the second server 2 (video processing system 100). Note that the communication between the image collection device 51 and the second server 2 may be wired communication or wireless communication.

[0026] Here, the image related to the video content distributed by the first server 1 may include, for example, an image captured by the viewer A1 of an image displayed on the display device 3. The image related to the video content may also include, for example, an image downloaded by the viewer A1 from an image distributed via a network such as the Internet at a digital museum or the like. In the latter case, it can be said that the viewer A1 is not directly looking at the display device 3 but is viewing the video content via the network. In other words, in either case, the image collected by the image collection device 51 is an image that indicates whether or not the viewer A1 is viewing the video content distributed by the first server 1.

[0027] The second camera 52, like the first camera 41, is installed around the display device 3 so that the space in front of the display device 3 falls within the imaging range, and images the viewer A1 in front of the display device 3. The second camera 52 also outputs information including the captured image to the second server 2 (video processing system 100). Note that the communication between the second camera 52 and the second server 2 may be wired communication or wireless communication.

[0028] The second camera 52 captures an image of the viewer A1 facing the display device 3 or a viewer A1 not facing the display device 3. Therefore, by analyzing the image captured by the second camera 52 with an appropriate image analysis algorithm, it is possible to analyze whether the viewer A1 is watching the video displayed on the display device 3. In other words, the image captured by the second camera 52 is an image that indicates whether the viewer A1 is watching the video content distributed by the first server 1.

[0029] [2-2.Video Processing System] Next, the configuration of the video processing system 100 will be described in detail. As already described, in the embodiment, the video processing system 100 is realized by the second server 2. The video processing system 100 includes a first acquisition unit 11, a second acquisition unit 12, a processing unit 13, an output unit 14, and a storage unit 15. The first acquisition unit 11, the second acquisition unit 12, the processing unit 13, and the output unit 14 may each be realized by a dedicated circuit, or may be realized by a processor executing a corresponding computer program stored in a memory.

[0030] The first acquisition unit 11 is a communication interface for communicating with the first camera 41 (display image acquisition device 4). The first acquisition unit 11 acquires video content. In the embodiment, the first acquisition unit 11 acquires video content distributed from the first server 1 as follows. For example, first, before the event is held, the video content is played on the display device 3, and the video displayed on the display device 3 is captured by the first camera 41. Then, the first acquisition unit 11 acquires information including the video captured by the first camera 41 from the first camera 41, thereby indirectly acquiring the video content distributed from the first server 1. The video content acquired by the first acquisition unit 11 is stored in the storage unit 15.

[0031] For example, if the video content distributed from the first server 1 is downloaded as a file and stored in advance on a recording medium, the first acquisition unit 11 may acquire the video content from the file stored on the recording medium.

[0032] The second acquisition unit 12 is a communication interface for communicating with the image collection device 51 and the second camera 52 (analysis image acquisition device 5). The second acquisition unit 12 acquires an analysis image for analyzing viewers of video content. In the embodiment, the second acquisition unit 12 acquires the analysis image by acquiring information including the image collected by the image collection device 51 from the image collection device 51. The second acquisition unit 12 also acquires the analysis image by acquiring information including the image captured by the second camera 52 from the second camera 52. The analysis image acquired by the second acquisition unit 12 is stored in the memory unit 15.

[0033] The processing unit 13 has a function of executing a process of analyzing the viewer A1's evaluation of the video content distributed by the first server 1. In the embodiment, the processing unit 13 executes the following three processes.

[0034] First, the processing unit 13 executes a process of identifying a corresponding frame in the video content that corresponds to the image for analysis. Here, when the image for analysis is an image captured by the second camera 52, that is, an image capturing the viewer A1 in front of the display device 3, the processing unit 13 identifies a frame that was being played on the display device 3 at the time when the image for analysis was captured in the video content as the corresponding frame. Specifically, when the second acquisition unit 12 acquires information including an image captured by the second camera 52, the processing unit 13 identifies a display image (frame) acquired from the display device 3 by the first acquisition unit 11 at the time when the information was acquired as the corresponding frame.

[0035] Furthermore, when the image for analysis is an image captured by the image collection device 51, that is, an image showing a part of video content posted by the viewer A1 to a social network service, the processing unit 13 identifies a frame in the video content that the image for analysis shows as a corresponding frame. Specifically, when the second acquisition unit 12 acquires information including an image collected by the image collection device 51, the processing unit 13 executes pattern matching between the image included in the information and the video content, and identifies a frame in the video content that matches the image (or has the same characteristics as the image) as the corresponding frame.

[0036] Secondly, the processing unit 13 executes a process of linking the identified corresponding frame and the analysis image with the elapsed time in the video content being played. Specifically, the processing unit 13 can calculate data linking each of all frames included in the video content with the elapsed time from the start of the playback of the video content by referring to the video content stored in the storage unit 15. Therefore, since the processing unit 13 has identified the corresponding frame corresponding to the analysis image in the first process described above, it can link the analysis image and the corresponding frame with the elapsed time in the video content being played by referring to the above data.

[0037] Thirdly, the processing unit 13 executes a process of generating evaluation information indicating the evaluation of the viewer A1 on the video content. Here, the evaluation of the viewer A1 on the video content indicates the degree of interest the viewer A1 has in the video content. For example, the evaluation information includes at least one of the number of viewers A1 directly viewing the display device 3 on which the video content is being played, the number of viewers A1 in front of the display device 3, and the number of images showing a part of the video content posted to a social networking service. In addition, the evaluation information includes the ratio of the number of viewers A1 directly viewing the display device 3 to the number of viewers A1 in front of the display device 3, etc.

[0038] For example, when the image for analysis is an image captured by the second camera 52, that is, an image capturing an image of a viewer A1 in front of the display device 3, the processing unit 13 executes an appropriate image analysis algorithm on the image to calculate the number of viewers A1 facing the display device 3, that is, the number of viewers A1 directly viewing the display device 3. Then, the processing unit 13 generates, as evaluation information, information linking the calculated number of viewers A1 with the corresponding frame corresponding to the image and the elapsed time in the video content.

[0039] Furthermore, for example, when the analysis image is an image captured by the image collecting device 51, that is, an image showing a part of a video content posted by the viewer A1 to a social network service, the processing unit 13 calculates the number of posts of the image. Then, the processing unit 13 generates, as evaluation information, information linking the calculated number of posts with the corresponding frame of the image and the elapsed time in the video content.

[0040] The evaluation information generated by the processing unit 13 is stored in the storage unit 15 for each corresponding frame, in other words, for each elapsed time in the video content.

[0041] The output unit 14 outputs the evaluation information generated by the processing unit 13 as time-series data of the elapsed time in the reproduced video content. That is, as described above, the storage unit 15 stores the evaluation information generated by the processing unit 13 for each elapsed time in the video content. Therefore, the output unit 14 can output the time-series data of the evaluation information by reading out the evaluation information stored in the storage unit 15.

[0042] For example, the output unit 14 outputs the time series data of the evaluation information by displaying the time series data of the evaluation information on a display connected to the second server 2. The output unit 14 may be, for example, a communication interface for communicating with an information processing terminal different from the second server 2. In this case, the output unit 14 outputs the time series data of the evaluation information to the information processing terminal. Then, the information processing terminal that has acquired the time series data of the evaluation information displays the time series data of the evaluation information on a display provided in the information processing terminal. The information processing terminal is, for example, a smartphone, a tablet terminal, or a desktop or laptop personal computer.

[0043] The storage unit 15 is, for example, a semiconductor memory or the like, and stores various information referenced when the video processing system 100 operates. In the embodiment, the storage unit 15 stores the video content acquired by the first acquisition unit 11. The storage unit 15 also stores the analysis image acquired by the second acquisition unit 12. The storage unit 15 also stores the evaluation information generated by the processing unit 13.

[0044] [3. Operation] The operation of the video processing system 100 according to the embodiment, that is, the video processing method according to the embodiment, will be described below. Fig. 3 is a sequence diagram showing an example of the operation of the video processing system 100 according to the embodiment. Below, a case will be described in which the analysis image is an image captured by the second camera 52, that is, an image capturing an image of a viewer A1 in front of the display device 3. Also, below, the process executed by the second server 2 (video processing system 100) will be described by dividing it into three threads: a time analysis thread, a viewing analysis thread, and an output thread.

[0045] First, the time analysis thread will be described. In the time analysis thread, the second server 2 acquires a display image from the display image acquisition device 4 (here, the first camera 41) (S11), executes time analysis using the acquired display image (S12), and outputs the time analysis result (S13).

[0046] Fig. 4 is a flowchart showing an example of a time analysis thread in the video processing system 100 according to the embodiment. As shown in Fig. 4, the first acquisition unit 11 of the video processing system 100 acquires video content by, for example, reading out video content previously stored in the storage unit 15 (S10). Next, the first acquisition unit 11 acquires a display image from the display image acquisition device 4 (here, the first camera 41) (S11).

[0047] Next, the processing unit 13 executes the following steps S121 to S123 as time analysis (S12). First, the processing unit 13 executes matching (pattern matching) between the display image acquired by the first acquisition unit 11 and the video content stored in the storage unit 15 (S121). As a result, the processing unit 13 extracts a frame corresponding to the display image (S122). In step S122, the processing unit 13 extracts a frame having the maximum score by pattern matching from among all frames of the video content as a frame corresponding to the display image. Then, the processing unit 13 links the extracted frame with the time measured by the second server 2 and the elapsed time in the video content (S123), and outputs a time analysis result obtained by linking (S13). Note that step S13 is an internal process in the processing unit 13, and corresponds to a process of giving the time analysis result from the time analysis thread to the output thread.

[0048] Thereafter, the video processing system 100 repeats steps S11 to S13. Therefore, the video processing system 100 outputs the time analysis result every time it acquires a display image. Note that the video processing system 100 may output the time analysis results for a certain period (for example, the period during which an event is held) all at once.

[0049] Returning to Fig. 3, the viewing analysis thread will now be described. In the viewing analysis thread, the second server 2 acquires an analysis image from the analysis image acquisition device 5 (here, the second camera 52) (S21), executes viewing analysis using the acquired analysis image (S22), and outputs a viewing analysis result (S23).

[0050] Fig. 5 is a flowchart showing an example of a viewing analysis thread in the video processing system 100 according to the embodiment. As shown in Fig. 5, the second acquisition unit 12 of the video processing system 100 acquires an analysis image from the analysis image acquisition device 5 (here, the second camera 52) (S21).

[0051] Next, the processing unit 13 executes the following steps S221 to S224 as the viewing analysis (S22). First, the processing unit 13 detects a person (i.e., the viewer A1) included in the analysis image acquired by the second acquisition unit 12 by executing an appropriate image analysis algorithm (S221) and detects the face of the person (S222). Then, the processing unit 13 executes a viewing judgment based on the direction of the detected person's face (S223). In the viewing judgment, if the detected person's face faces forward, that is, faces the display device 3, the processing unit 13 judges that the person is viewing the video content. On the other hand, if the detected person's face faces backward or to the side, that is, does not face the display device 3, the processing unit 13 judges that the person is not viewing the video content.

[0052] Note that processing unit 13 may detect the direction of gaze of a person (i.e., viewer A1) included in the analysis image in step S222, and perform viewing determination based on the direction of gaze of the person detected in step S223. In this case, in the viewing determination, processing unit 13 determines that the person is viewing the video content if the gaze of the detected person is directed toward display device 3, and determines that the person is not viewing the video content if not.

[0053] Then, the processing unit 13 links the viewing determination result and the time measured by the second server 2 (S224), and outputs the viewing analysis result obtained by the linking (S23). Note that step S23 is an internal process in the processing unit 13, and corresponds to a process of providing the viewing analysis result from the viewing analysis thread to the output thread.

[0054] Thereafter, the video processing system 100 repeats steps S21 to S23. Therefore, the video processing system 100 outputs the viewing analysis result every time it acquires an analysis image. Note that the video processing system 100 may output the viewing analysis result for a certain period (for example, the period during which an event is held) all at once.

[0055] Returning to Fig. 3, the output thread will now be described. In the output thread, the second server 2 acquires the time analysis result obtained in the time analysis thread (S31), acquires the viewing analysis result obtained in the viewing analysis thread (S32), generates rating information based on the time analysis result and the viewing analysis result (S33), and outputs the generated rating information (S34).

[0056] Fig. 6 is a flowchart showing an example of an output thread in the video processing system 100 according to the embodiment. As shown in Fig. 6, the processing unit 13 of the video processing system 100 acquires a time analysis result obtained in the time analysis thread (S31), and acquires a viewing analysis result obtained in the viewing analysis thread (S32). Note that steps S31 and S32 may be executed in the reverse order, or may be executed in parallel.

[0057] Next, the processing unit 13 executes the following steps S331 and S332 as a process (S33) for generating evaluation information. First, the processing unit 13 links the time analysis result and the viewing analysis result (S331). Specifically, the processing unit 13 links an analysis image in the viewing analysis result in which the time measured by the second server 2 coincides with a frame in the time analysis result by referring to the time analysis result and the viewing analysis result. The frame corresponds to the analysis image, and thus corresponds to the corresponding frame. That is, the corresponding frame is specified by step S331. Furthermore, by linking the analysis image and the corresponding frame, the determination result in the viewing analysis result and the elapsed time in the video content in the time analysis result are also linked. That is, by step S331, the specified corresponding frame and the analysis image are linked to the elapsed time in the video content being played. As a result, the determination result of whether the viewer A1 is viewing the video content is linked for each corresponding frame, in other words, for each elapsed time in the video content.

[0058] Next, the processing unit 13 generates a graph based on the data obtained in step S331 (S332). In the graph, for example, the number of viewers A1 who directly watch the display device 3 on which the video content is being played is plotted for each elapsed time of the video content. In other words, step S332 corresponds to a process of generating time-series data of the evaluation information.

[0059] Then, the output unit 14 of the video processing system 100 outputs the evaluation information (here, time-series data of the evaluation information) generated by the processing unit 13 (S34). FIG. 7 is a diagram showing an example of output of evaluation information in the video processing system 100 according to the embodiment. In FIG. 7, the vertical axis represents an evaluation value obtained by quantifying the evaluation of the viewer A1 for the video content, and the horizontal axis represents the elapsed time in the video content. In FIG. 7, the solid line represents the number of viewers A1 in front of the display device 3, the dashed line represents the number of viewers A1 directly looking at the display device 3, and the dotted line represents the number of posts to the social network service. The same applies to FIG. 8 described later. The output unit 14 outputs the time-series data of the evaluation information by, for example, displaying the graph shown in FIG. 7 on a display connected to the second server 2.

[0060] The output unit 14 may not only output the time series data of the evaluation information as shown in Fig. 7, but may also output a video of the reproduced video content as shown in Fig. 8. Fig. 8 is a diagram showing another example of output of the evaluation information in the video processing system 100 according to the embodiment. The output unit 14 outputs the time series data of the evaluation information and the video of the reproduced video content by, for example, displaying the graph and video shown in Fig. 8 on a display connected to the second server 2.

[0061] [4. Advantages, etc.] Advantages of the video processing system 100 (video processing method) according to the embodiment will be described below with reference to Fig. 9. Fig. 9 is an explanatory diagram of advantages of the video processing system 100 according to the embodiment. As already mentioned, in the video processing method of the comparative example, when the first server 203 that distributes video content and the second server 204 that analyzes the captured image of the viewer A1 are different from each other as shown in Fig. 1(b), it is not possible to link the elapsed time in the played video content with whether or not the viewer A1 is watching the video content during that elapsed time, and it is difficult to analyze the viewer A1's evaluation of the video content.

[0062] In contrast, in the video processing system 100 according to the embodiment, a corresponding frame corresponding to an analysis image in the video content is identified, and the identified corresponding frame and the analysis image are linked by the elapsed time in the reproduced video content. Therefore, as shown in Fig. 9, even if the second server 2 is different from the first server 1 that distributes the video content, the video processing system 100 according to the embodiment can link the elapsed time in the reproduced video content with whether or not the viewer A1 is watching the video content at that elapsed time, which has the advantage of making it easier to analyze the viewer A1's evaluation of the video content, such as which part of the video content the viewer A1 showed interest in.

[0063] Furthermore, in the video processing system 100 of the embodiment, even if the video content is played repeatedly, by calculating a representative value (e.g., an average value) of the evaluation of the viewer A1 for each elapsed time of the video content, it is possible to obtain an evaluation of the viewer A1 that is free from the influences of the time period when the video content is played or the surrounding environment of the display device 3 on which the video content is played.

[0064] Furthermore, the video processing system 100 according to the embodiment has an advantage in that it makes it easier to analyze the number of posts on a social network service, which can be an important indicator for video content creators.

[0065] Therefore, by making it easier to analyze the evaluations of the viewer A1 regarding the video content, there is an advantage in that it is possible to expect an improvement in the quality of feedback when updating the video content or creating new video content.

[0066] [5. Other embodiments] Although the embodiments have been described above, the present disclosure is not limited to the above-described embodiments.

[0067] For example, in the above embodiment, first acquisition unit 11 acquires video content distributed from first server 1 by acquiring information including a video captured by first camera 41 from first camera 41, but this is not limited to the above. For example, first acquisition unit 11 may acquire video content distributed from first server 1 via relay device 42, which is display image acquisition device 4, as shown in Fig. 10. Fig. 10 is an explanatory diagram of an example of acquiring video content using relay device 42.

[0068] The relay device 42 includes a receiving unit 421, a first transmitting unit 422, and a second transmitting unit 423. The receiving unit 421 is a communication interface for communicating with the first server 1. The receiving unit 421 receives video content distributed from the first server 1. The receiving unit 421 also distributes the received video content to each of the first transmitting unit 422 and the second transmitting unit 423. The first transmitting unit 422 is a communication interface for communicating with the display device 3. The first transmitting unit 422 transmits the video content received from the receiving unit 421 to the display device 3. The second transmitting unit 423 is a communication interface for communicating with the second server 2. The second transmitting unit 423 transmits the video content received from the receiving unit 421 to the second server 2.

[0069] As described above, the video content delivered from the first server 1 may be distributed to each of the display device 3 and the second server 2 by using the relay device .

[0070] In the above embodiment, the evaluation information includes information indicating whether the viewer A1 is viewing the video content, but is not limited to this. For example, the evaluation information may further include information indicating the gender, age, or facial expression of the viewer A1 viewing the video content.

[0071] In the above embodiment, the video processing system 100 is realized by a single device, but this is not limited thereto. For example, the video processing system 100 may be realized by a plurality of devices. In other words, a plurality of components included in the video processing system 100 may be distributed among two or more devices.

[0072] In the above embodiment, the video processing system 100 is realized by the second server 2, but is not limited to this. For example, the video processing system 100 may be provided in an information processing terminal such as a personal computer installed in a closed space. In other words, the video processing system 100 may be realized by cloud computing or edge computing.

[0073] In the above embodiment, the processes executed by a specific processing unit may be executed by another processing unit. The order of multiple processes may be changed, or multiple processes may be executed in parallel.

[0074] In the above embodiment, each component may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

[0075] Furthermore, each component may be realized by hardware. Each component may be a circuit (or an integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit, or a dedicated circuit.

[0076] In addition, the general or specific aspects of the present disclosure may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, etc. Also, the present disclosure may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0077] The present disclosure may also be realized as a video processing method executed by a computer such as the video processing system of the above-described embodiment. The present disclosure may also be realized as a program (computer program product) for causing a computer to execute such a video processing method, or as a computer-readable non-transitory recording medium on which such a program is recorded.

[0078] In addition, the present disclosure also includes forms obtained by applying various modifications to each embodiment that a person skilled in the art may conceive, or forms realized by arbitrarily combining the components and functions of each embodiment within the scope that does not deviate from the spirit of the present disclosure.

[0079] (summary) As described above, in the video processing method according to the first aspect, video content is acquired. In addition, in this video processing method, an analysis image for analyzing a viewer A1 of the video content is acquired. In addition, in this video processing method, a corresponding frame in the video content that corresponds to the analysis image is identified. In addition, in this video processing method, the identified corresponding frame and the analysis image are linked by the elapsed time in the played video content.

[0080] This type of video processing method has the advantage that it is possible to link the elapsed time of the video content being played with whether or not viewer A1 is watching the video content during that elapsed time, making it easier to analyze viewer A1's evaluation of the video content, such as which parts of the video content showed viewer A1's interest.

[0081] Also, for example, in the video processing method according to the second aspect, in the first aspect, the analysis image includes an image of a viewer A1 in front of the display device 3 on which the video content is being played. In the video processing method, a frame of the video content being played on the display device 3 at the time the analysis image was captured is identified as the corresponding frame.

[0082] Such a video processing method has the advantage that it is possible to determine whether or not viewer A1 is facing the display device 3 based on an image captured of viewer A1, making it easier to analyze viewer A1's evaluation of the video content, such as whether or not viewer A1 is interested in the video content.

[0083] In addition, for example, in the video processing method according to the third aspect, in the first or second aspect, the analysis image includes an image showing a part of the video content posted on a social network service by a viewer A1 of the video content. In the video processing method, a frame in the video content that is shown by the analysis image is identified as a corresponding frame.

[0084] Such a video processing method has the advantage that it becomes easier to analyze viewer A1's evaluation of video content, such as whether or not viewer A1 is interested in the video content, based on whether or not the viewer A1 has posted an image to a social networking service.

[0085] Also, for example, in a video processing method according to a fourth aspect, in any one of the first to third aspects, evaluation information indicating the viewer A1's evaluation of the video content is further output as time series data of the elapsed time in the video content being played.

[0086] Such a video processing method has the advantage that it becomes easier to grasp which parts of the video content interest the viewer A1, and therefore it becomes easier to analyze the viewer A1's evaluation of the video content.

[0087] Also, for example, in the video processing method of the fifth aspect, in the fourth aspect, the evaluation information includes at least one of the number of viewers A1 directly viewing the display device 3 on which the video content is being played, the number of viewers A1 in front of the display device 3, and the number of images showing part of the video content posted to a social networking service.

[0088] Such a video processing method has the advantage that it becomes easier to grasp which parts of the video content interest the viewer A1, and therefore it becomes easier to analyze the viewer A1's evaluation of the video content.

[0089] Also, for example, in a video processing method according to a sixth aspect, in the fourth or fifth aspect, a video obtained by playing back the video content is further output.

[0090] Such a video processing method has the advantage that the reproduced video content and the rating information can be viewed simultaneously, making it easier to understand which parts of the video content interest the viewer A1.

[0091] Also, for example, a video processing system 100 according to a seventh aspect includes a first acquisition unit 11, a second acquisition unit 12, and a processing unit 13. The first acquisition unit 11 acquires video content. The second acquisition unit 12 acquires an analysis image for analyzing a viewer A1 of the video content. The processing unit 13 identifies a corresponding frame in the video content that corresponds to the analysis image, and links the identified corresponding frame and the analysis image by the elapsed time in the played video content.

[0092] Such a video processing system 100 can link the elapsed time of the video content being played with whether or not viewer A1 is watching the video content during that elapsed time, which has the advantage of making it easier to analyze viewer A1's evaluation of the video content, such as which parts of the video content viewer A1 was interested in. [Industrial Applicability]

[0093] The present disclosure is applicable, for example, to a system for analyzing viewers' evaluations of video content. [Explanation of symbols]

[0094] 1 First Server 2 Second Server 3 Display device 4 Display image acquisition device 5 Image acquisition device for analysis 11 First acquisition part 12 Second acquisition part 13 Processing section 14 Output section 15 Storage section 41 Camera No. 1 42 Relay Device 51 Image acquisition device 52 Second Camera 100 Image Processing System 200 Servers 201 Display device 202 Camera 203 First Server 204 Second Server 421 Receiving unit 422 First Transmission Unit 423 Second Transmission Unit A1 Viewer

Claims

1. Get video content, acquiring an analysis image for analyzing a viewer of the video content; identifying a corresponding frame in the video content that corresponds to the analysis image; linking the identified corresponding frame and the analysis image with each other based on an elapsed time in the video content being played; Image processing method.

2. the analysis image includes an image of the viewer in front of a display device on which the video content is reproduced, identifying a frame of the video content that was being played on the display device at the time when the analysis image was captured as the corresponding frame; The video processing method according to claim 1 .

3. the analysis image includes an image showing a part of the video content posted by the viewer on a social network service, identifying a frame indicated by the analysis image in the video content as the corresponding frame; The video processing method according to claim 1 .

4. further outputting evaluation information indicating an evaluation of the viewer with respect to the video content as time-series data of the elapsed time in the video content being played back; The image processing method according to any one of claims 1 to 3.

5. The evaluation information includes at least one of the number of viewers directly viewing the display device on which the video content is being played, the number of viewers in front of the display device, and the number of posts of images showing a part of the video content to a social networking service. The video processing method according to claim 4.

6. further outputting a video of the reproduced video content; The video processing method according to claim 4.

7. A first acquisition unit that acquires video content; a second acquisition unit that acquires an analysis image for analyzing a viewer of the video content; A processing unit, The processing unit includes: identifying a corresponding frame in the video content that corresponds to the analysis image; linking the identified corresponding frame and the analysis image with each other based on an elapsed time in the video content being played; Video processing system.

Images