Subtitle display control system and subtitle display control method

Abstract

Provided are a subtitle display control system and a subtitle display control method capable of displaying subtitles that are more harmonious with the atmosphere of content. A subtitle display control system includes: a display unit that displays content; a feature extraction unit that extracts features of the content; a display pattern determination unit that determines the display pattern of the subtitles on the basis of the characteristics of the content; and a display control unit that displays the subtitles on the display unit in the display pattern determined by the display pattern determination unit.

Description

Technical Field

[0001] The present disclosure relates to a subtitle display control system and a subtitle display control method. Background Art

[0002] Conventionally, there has been proposed a subtitle display control system for changing the display of subtitles based on an audio signal of content.

[0003] For example, Japanese Patent Application Laid-Open No. 2007-300323 describes a subtitle display control system that determines a subtitle display method based on features of sound extracted from a sound signal and superimposes the subtitles on a video using the determined display method. Summary of the invention

[0004] However, when the display of subtitles is changed based on a sound signal, the display style of the subtitles may not always match the atmosphere of the image of the content.

[0005] The present disclosure is made in view of the above problems. An object of the present invention is to provide a subtitle display control system and a subtitle display control method capable of displaying subtitles that are more in harmony with the atmosphere of the content.

[0006] A subtitle display control system according to one aspect of the present disclosure includes: a display unit that displays content; a feature extraction unit that extracts features of the content; a display style determination unit that determines a display style of subtitles based on the features of the content; and a display control unit that displays the subtitles on the display unit in the display style determined by the display style determination unit.

[0007] A subtitle display control method according to one aspect of the present disclosure is a subtitle display control method executed by a subtitle display control system, comprising: a step of extracting content features; a step of determining a display style of subtitles based on the content features; and a step of displaying the subtitles on the display unit in the determined display style. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 This is a block diagram showing the basic structure of the subtitle display control system according to the first embodiment. Figure 2 It is shown Figure 1 A diagram of an example of a table stored in a storage unit. Figure 3 It is shown Figure 1 A diagram of an example of another table stored in a storage unit of . Figure 4 It is shown by Figure 1 A flowchart of an example of a process performed by a subtitle display control system. Figure 5This is a diagram showing an example of a table stored in the storage unit in another processing example 1. Fig. 6A The diagram is a schematic diagram showing an example of a subtitle display image on a screen. Figure 6B This is a schematic diagram showing an example of a subtitle display image outside the screen. Figure 7 This is a diagram showing an example of a table stored in the storage unit in another processing example 2. Figure 8 This is a diagram showing an example of a table stored in the storage unit in another processing example 3. Fig. 9 It is a block diagram showing the structure of a subtitle display control system according to the second embodiment. Fig.10 is a diagram showing another method of providing content in the second embodiment. Fig.11 This is a block diagram showing the configuration of a subtitle display control system according to the third embodiment. Fig.12 A diagram showing an example of an image represented by an image signal. Fig.13 It is a block diagram showing the structure of a subtitle display control system according to a fourth embodiment. DETAILED DESCRIPTION

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. In addition, in the drawings, the same reference numerals are given to the same or equivalent components, and when the descriptions related to the same or equivalent components are repeated, they will be appropriately omitted.

[0010] [First Embodiment / Configuration of Subtitle Display Control System] Figure 1 1 is a functional block diagram showing the basic structure of the subtitle display control system 100 according to the first embodiment. The subtitle display control system 100 may include Figure 1 That is, the subtitle display control system 100 can also be implemented by one device including a part or all of the functional blocks shown. Figure 1 All the functional blocks shown in the figure are thus constituted by the one device. Alternatively, the functional blocks may be respectively included in a plurality of devices. Figure 1 The functional blocks shown in the figure include the following: Figure 1 All the functional blocks shown are implemented to realize the subtitle display control system 100. In this case, the plurality of devices are connected in a manner capable of mutual information communication.

[0011] The subtitle display control system 100 includes, for example, any display device that displays content on a display unit. Examples of the display device include, but are not limited to, television receivers, monitors, displays, computers, tablet terminals, smartphones, and projectors.

[0012] One or more devices constituting the subtitle display control system 100 include, for example, a control unit, a storage unit, and a communication unit. Depending on the functions of each device, one or more devices may further include functional units other than the control unit, the storage unit, and the communication unit.

[0013] The control unit controls and manages the entire device, represented by each functional unit of the device. The control unit performs various controls, such as running a control program stored in the storage unit. For example, the control unit can be composed of a control device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit).

[0014] The storage unit is a storage medium capable of storing programs and data. The storage unit may be composed of, for example, a semiconductor memory or a magnetic memory. Specifically, the storage unit may be composed of, for example, an EEPROM (Electrically Erasable Programmable Read-Only Memory). The storage unit may store, for example, a program for causing the control unit to operate.

[0015] The communication unit performs information communication with the external device. The communication unit is configured to include an appropriate interface according to the information communication method. The device can send and receive various information with the external device via the communication unit.

[0016] like Figure 1As shown, the subtitle display control system 100 includes as functional units: a content signal receiving unit 1, a signal separation unit 2, a subtitle signal decoding unit 3, an image signal decoding unit 4, a sound signal decoding unit 5, a subtitle feature extraction unit 6, an image feature extraction unit 7, a sound feature extraction unit 8, a storage unit 9, a display style determination unit 10, a subtitle signal conversion unit 11, a subtitle signal processing unit 12, an image signal processing unit 13, a sound signal processing unit 14, a display control unit 15, a display unit 16, and a sound output unit 17. These functional units are implemented by the functional units possessed by one or more devices. For example, the content signal receiving unit 1 is implemented by the communication unit of one or more devices. For example, the storage unit 9 is implemented by the storage unit of one or more devices. For example, the signal separation unit 2, the subtitle signal decoding unit 3, the image signal decoding unit 4, the sound signal decoding unit 5, the subtitle feature extraction unit 6, the image feature extraction unit 7, the sound feature extraction unit 8, the display mode determination unit 10, the subtitle signal conversion unit 11, the subtitle signal processing unit 12, the image signal processing unit 13, the sound signal processing unit 14, and the display control unit 15 are implemented by the control unit of one or more devices. The display unit 16 and the sound output unit 17 include a display device as one of the one or more devices.

[0017] The content signal receiving unit 1 receives a content signal transmitted from an external device. The content signal is a signal related to information of the content reproduced by the subtitle display control system 100.

[0018] In this embodiment, the content includes images. For example, the content includes videos, dynamic images, or static images. Specifically, the content is a movie, a TV series, a play, an animation, and a computer game. However, the content is not limited to the content exemplified here. The content may also include sound. In this embodiment, the content includes both images and sound.

[0019] The content signal is generated, for example, in an external device that is a signal transmission source. The external device generates the content signal by, for example, multiplexing a subtitle signal, an image signal, and a sound signal. Here, the subtitle signal, the image signal, and the sound signal are signals related to information about the subtitle, image, and sound of the content, respectively. The external device transmits the content signal generated by multiplexing to the subtitle display control system 100. The subtitle display control system 100 receives the content signal transmitted by the external device in this way through the content signal receiving unit 1.

[0020] The signal separation unit 2 separates the multiplexed content signal into original signals. In this embodiment, the signal separation unit 2 separates the content signal into a subtitle signal, an image signal, and a sound signal. The separated subtitle signal, image signal, and sound signal are input to the subtitle signal decoding unit 3, the image signal decoding unit 4, and the sound signal decoding unit 5, respectively.

[0021] The subtitle signal decoding unit 3, the image signal decoding unit 4 and the audio signal decoding unit 5 decode the subtitle signal, the image signal and the audio signal respectively. The decoded subtitle signal, the image signal and the audio signal are input to the subtitle feature extraction unit 6, the image feature extraction unit 7 and the audio feature extraction unit 8 respectively.

[0022] The feature extraction unit including the subtitle feature extraction unit 6, the image feature extraction unit 7, and the sound feature extraction unit 8 extracts the features of the content. Specifically, the subtitle feature extraction unit 6, the image feature extraction unit 7, and the sound feature extraction unit 8 extract the features of the subtitle, the image, and the sound, respectively. The subtitle feature extraction unit 6, the image feature extraction unit 7, and the sound feature extraction unit 8 extract features based on, for example, a predetermined algorithm.

[0023] The subtitle feature extraction unit 6 extracts features of the subtitles. For example, the subtitle feature extraction unit 6 extracts text data of the subtitles as features of the subtitles.

[0024] In addition, for example, the subtitle feature extraction unit 6 extracts a specific string contained in the text data as a feature of the subtitle. The specific string extracted by the subtitle feature extraction unit 6 is predetermined, for example, stored in a storage unit of the device. The subtitle feature extraction unit 6 can extract the specific string by searching the specific string stored in the storage unit from the text data.

[0025] The specific character string can be appropriately determined. The specific character string can be, for example, a character string representing a specific sound or voice. As an example, the specific character string can be a character string representing a shout or a laugh. Character strings representing shouts include, for example, "ah" or "wow", etc. Character strings representing laughs include, for example, "wahaha" or "chuckle", etc. However, character strings representing shouts or laughs are not limited to the examples shown here. In addition, the specific character string is not limited to character strings representing shouts or laughs. For example, the specific character string can be an onomatopoeia including onomatopoeia and mimetic words, etc.

[0026] In addition, for example, the subtitle feature extraction unit 6 extracts the explanatory sentences related to the sound contained in the text data as the features of the subtitles. The explanatory sentences related to the sound are sentences that describe the content of the sound, such as the sound of a door opening and closing, the sound of rain, the sound of a telephone ringing, thunder, or a siren. In the text data of the subtitles, the explanatory sentences related to the sound are expressed in a specific manner. As an example, as a specific manner, brackets are used to express the explanatory sentences related to the sound. Specifically, the sound of a door opening and closing is expressed as (the sound of a door opening and closing) in the text data of the subtitles. In this case, the subtitle feature extraction unit 6 can extract the explanatory sentences related to the sound by retrieving the part expressed in a specific manner from the text data.

[0027] Furthermore, the subtitle feature extraction unit 6 is not limited to extracting description words related to specific character strings and sounds, and may extract other features that can be extracted from a subtitle signal as subtitle features.

[0028] The image feature extraction unit 7 extracts features of the image. For example, the image feature extraction unit 7 extracts color information of the image, a person or an object reflected in the image, and the like as features of the image.

[0029] The color information of an image is information related to the color contained in the image, for example, information on the color contained in the entire image or a part of the image. The image feature extraction unit 7 can extract the color information as, for example, RGB values. For example, the image feature extraction unit 7 can extract the RGB value as a feature of the image from the color signal contained in the image signal by converting the color of the entire image into RGB values. The image feature extraction unit 7 can convert the color of the image into RGB values ​​by a known method.

[0030] Color information can be represented by a plurality of colors that are distinguished in advance. For example, it is assumed that color information is distinguished into 11 colors: red, orange, yellow, green, blue, purple, pink, brown, white, gray, and black. The image feature extraction unit 7 determines which color of the image is close to among the 11 colors based on the RGB value, for example, and uses the determined color as the color information of the image. Details of the method for determining the color information of an image based on the RGB value will be described later. Figure 4 In addition, the color information does not necessarily have to be extracted as RGB values ​​or multiple colors, and may be extracted by other indicators related to hue or color, for example.

[0031] The person or object reflected in the image is information related to the person or object reflected in the image, which may be the person or object itself, or may include the action of the person or object. The image feature extraction unit 7 may extract the person or object reflected in the image using a known image recognition technology. For example, the image feature extraction unit 7 determines the person reflected in the image. The image feature extraction unit 7 may also extract the action of the mouth of the person reflected in the image. By extracting the action of the mouth, the person (speaker) who is speaking in the image can be determined. The image feature extraction unit 7 may also extract the expression of the person (e.g., speaker) reflected in the image. As a feature of the image, the image feature extraction unit 7 may also extract information that distinguishes the expression of the speaker into a cheerful expression and a dull expression. In addition, for example, the image feature extraction unit 7 extracts a specific object reflected in the image. The extracted specific object is, for example, predetermined and stored in the storage unit of the device. The specific object may be, for example, a door, rain, a telephone, thunder, or a specific vehicle (e.g., a patrol car, an ambulance, or a fire truck, etc.), but is not limited thereto. In addition, the image feature extraction unit 7 may also extract the position of the person or object reflected in the image as a feature of the image.

[0032] In addition, the image feature extraction unit 7 may also extract other features that can be extracted from the image signal as the image feature, not limited to the color information of the image, or the person or object reflected in the image.

[0033] The sound feature extraction unit 8 extracts features of the sound. For example, the sound feature extraction unit 8 extracts volume, pitch of the sound, a specific type of sound, and the like as features of the sound.

[0034] The sound feature extraction unit 8 can, for example, extract the volume and the height of the sound based on the waveform of the sound in the sound signal. As a feature of the sound, the sound feature extraction unit 8 can extract whether the sound contained in the sound signal exceeds or falls below a predetermined threshold value related to the volume or the height of the sound. The predetermined threshold value is, for example, predetermined and stored in the storage unit of the device.

[0035] The specific type of sound is, for example, a person's voice, a sound made by a specific object, etc. The sound made by a specific object may be, for example, the sound of a door opening and closing, the sound of rain, the sound of a telephone ringing, thunder, or a siren, etc., but is not limited thereto. The sound feature extraction unit 8 may, for example, use a known sound recognition technology to extract the specific type of sound. The type of sound extracted by the sound feature extraction unit 8 is, for example, predetermined and stored in the storage unit of the device.

[0036] In addition, the voice feature extraction unit 8 may extract the length of the silent period as the feature of the voice. For example, the voice feature extraction unit 8 may extract the length of the silent period based on the waveform of the voice in the voice signal.

[0037] In addition, the sound feature extraction unit 8 is not limited to the volume, the pitch of the sound, a specific type of sound, and the length of the silent period, and can also extract other features that can be extracted from the sound signal as the sound feature. In addition, the sound feature extraction unit 8 sends the sound signal to the sound signal processing unit 14.

[0038] The subtitle feature extraction unit 6 , the image feature extraction unit 7 , and the audio feature extraction unit 8 provide information on the extracted features to the display mode determination unit 10 .

[0039] The storage unit 9 stores information necessary for determining the display mode based on the features. For example, the storage unit 9 stores a table showing the correspondence between the features and the display modes. Alternatively, the storage unit 9 stores an algorithm for determining the display mode based on the features.

[0040] Here, the feature is a feature extracted by the subtitle feature extraction unit 6, the image feature extraction unit 7, and the sound feature extraction unit 8. The display style is the display style of the subtitle. The display style of the subtitle includes, for example, at least one element of the subtitle text size, font, color, and display position. In this embodiment, the subtitle display style is assumed to include all elements of the subtitle text size, font, color, and display position and is described below.

[0041] Each element related to the display style of subtitles is associated with a specific feature. For example, the size of the text corresponds to the volume as a sound feature, the font corresponds to the color information as an image feature, the color of the text corresponds to the expression of the speaker as an image feature, and the display position corresponds to the position of the person or object reflected in the image as an image feature. In addition, the correspondence between each element related to the display style of subtitles and the feature is not limited to this.

[0042] The storage unit 9 stores a table showing the correspondence between the characteristics and the display mode for each element related to the display mode. For example, in the above example, the text size corresponds to the volume as a sound feature. In this case, the storage unit 9 stores a table showing the correspondence between the volume and the text size.

[0043] Figure 2 : is a diagram showing an example of a table stored in the storage unit 9. Specifically, Figure 2 is a table showing the correspondence between volume and text size. Figure 2 In the text, the volume is classified into three stages: less than V1, greater than V1 and less than V2, and greater than V2. Here, V1 and V2 are thresholds related to the volume, and V1<V2. The values ​​of V1 and V2 are predetermined, for example. The first size, the second size, and the third size are associated with the three-stage classification of the volume as text sizes. The first size, the second size, and the third size represent the size of the text, such as the number of points of the text. Here, it is assumed that the second size is larger than the first size, and the third size is larger than the second size. As shown in Figure 2 As can be understood from the table, the first size corresponds to a volume less than V1, the second size corresponds to a volume greater than V1 and less than V2, and the third size corresponds to a volume greater than V2. In other words, the larger the volume, the larger the text size.

[0044] For example, in the above example, the font corresponds to the color information as the image feature. In this case, the storage unit 9 stores a table indicating the correspondence relationship between the color information and the font.

[0045] Figure 3 : is a diagram showing an example of another table stored in the storage unit 9. Specifically, Figure 3 is a table showing the correspondence between color information and fonts. Figure 3In , color information is divided into 11 colors: red, orange, yellow, green, blue, purple, pink, brown, white, gray and black. Fonts are associated with the color information of the 11 colors. Figure 3 In the table shown, red and gray are associated with running script, orange is associated with Maru-Gothic, yellow is associated with Meilio, green and blue are associated with Mincho, purple is associated with textbook, pink is associated with bold Maru-Gothic, brown is associated with textbook, white is associated with Kaiti, and black is associated with Kaku-Gothic.

[0046] The association between colors and fonts is performed in advance, for example, based on the things imaged according to each color and / or the images of each color itself. For example, the color red can be associated with things such as attitude, blood, lipstick, and apples. In addition, for example, the color red itself can be associated with images of heat, gorgeousness, passion, and danger. Fonts consistent with such things or images are associated with red. The same is true for other colors. Using this method, for example, the association between colors and fonts is performed in advance by the administrator or content provider of the subtitle display control system 100.

[0047] Here, use Figure 2 and Figure 3 The table showing the correspondence between the volume and the text size and the table showing the correspondence between the color information and the font are specifically described, but the same table is also stored in the storage unit 9 for the correspondence between the speaker's expression and the color of the text. For example, dark colors are associated with gloomy expressions, and warm colors are associated with cheerful expressions.

[0048] Regarding the display position of the text, for example, an algorithm that can determine that the display position of the text and the speaker have a predetermined positional relationship is stored in the storage unit 9. The predetermined positional relationship is, for example, a relationship in which the distance from the speaker is within a certain range. The predetermined positional relationship is preferably a relationship in which the distance from the speaker is close. When the speech content is displayed as subtitles by shortening the distance from the speaker, the audience can easily understand the speaker of the subtitles.

[0049] Refer again Figure 1 The display style determination unit 10 determines the display style of the subtitles based on the features of the content. In this embodiment, the display style determination unit 10 determines the display style of the subtitles based on the features extracted by the subtitle feature extraction unit 6, the image feature extraction unit 7, and the sound feature extraction unit 8. Specifically, the display style determination unit 10 can determine the display style of the subtitles by referring to the storage unit 9.

[0050] For example, the display style determination unit 10 determines the display style of the subtitles based on the characteristics of the image. As an example, the display style determination unit 10 determines the font as the display style by referring to the table stored in the storage unit 9 based on the color information as the characteristic of the image. Specifically, the display style determination unit 10 determines the font as the display style by referring to the table stored in the storage unit 9. Figure 3 The table shown in FIG. 1 determines the font corresponding to the color information as the display style of the subtitles. In the same way, the display style determination unit 10 can determine the color and display position of the text as the display style of the subtitles.

[0051] In addition, the display mode determination unit 10 may determine the display mode of the subtitles based on other characteristics. For example, the display mode determination unit 10 may determine the text size as the display mode based on the characteristics of the sound. Figure 2 The table shown determines the display style of subtitles according to the character size corresponding to the volume which is a feature of the sound.

[0052] The display style determination unit 10 may also determine the display style of the subtitles based on a combination of multiple features. For example, the display style determination unit 10 may determine the text size, font, color, and display position of the subtitles as the display style of the subtitles based on the features of the image and the features of the sound.

[0053] The subtitle signal conversion unit 11 converts the text data of the subtitles into the format determined by the display mode determination unit 10. For example, the storage unit 9 stores in advance the data of the text of all the character sizes, fonts, and colors used in the subtitle display control system 100. The subtitle signal conversion unit 11 refers to the storage unit 9 and converts the text data of the subtitles into the character size, font, and color determined by the display mode determination unit 10.

[0054] The subtitle signal processing unit 12 processes the subtitle data converted by the subtitle signal conversion unit 11 into a form that can be displayed together with the image. For example, when the subtitles are displayed superimposed on the image, the subtitle signal processing unit 12 processes the subtitle data converted by the subtitle signal conversion unit 11 into a form that can be superimposed on the image.

[0055] The image signal processing unit 13 generates an image to be displayed on the display unit 16 based on the image signal. For example, the image signal processing unit 13 processes the image signal into a format that can be displayed on the display unit 16.

[0056] The audio signal processing unit 14 processes the audio signal into a format that can be output by the audio output unit 17. For example, the audio signal processing unit 14 converts the audio signal, which is a digital signal, into an analog signal.

[0057] The display control unit 15 causes the image and subtitles to be displayed on the display unit 16. Specifically, the display control unit 15 causes the image and subtitles to be displayed on the display unit 16 based on the image signal processed by the image signal processing unit 13 and the subtitle data processed by the subtitle signal processing unit 12. At this time, the display control unit 15 causes the subtitles to be displayed on the display unit 16 in the display style determined by the display style determination unit 10. The display control unit 15 causes the subtitles to be displayed on the display unit 16 by overlapping the image, for example.

[0058] The display unit 16 is a device that displays images. The display unit 16 can be composed of a known display such as a liquid crystal display (LCD), an organic EL display (OELD), or an inorganic EL display (IELD). The display unit 16 displays various information based on the control of the display control unit 15. For example, the display unit 16 displays content including images and subtitles based on the control of the display control unit 15.

[0059] The sound output unit 17 is a device that outputs sound. The sound output unit 17 is composed of, for example, a speaker. The sound output unit 17 outputs, for example, sound of content based on the control of the sound signal processing unit 14 .

[0060] [Processing example of subtitle display control system] Next, an example of processing executed by subtitle display control system 100 will be described. Figure 4 1 is a flowchart showing an example of a process of displaying subtitles in a display style determined based on the characteristics of an image. Figure 4 An example of a process of determining the display style of subtitles based on color information that is a feature of an image and displaying the subtitles in the determined display style is shown.

[0061] exist Figure 4 In the process shown, the subtitle display control system 100 first determines the color information of the image based on the RGB value. Specifically, the subtitle display control system 100 determines the color information of the image through steps S11 to S13.

[0062] That is, the image feature extraction unit 7 extracts the color signal of one frame of the image based on the decoded image signal (step S11). The image feature extraction unit 7 converts the extracted color signal of one frame into RGB values ​​(step S12). Thus, the RGB values ​​of one frame of the image are extracted.

[0063] The image feature extraction unit 7 determines the approximate color information of the image of the frame based on the RGB value of the extracted frame (step S13). For example, the image feature extraction unit 7 determines the color information based on the classification of the color information to which the RGB value of the extracted frame approximates from the pre-classified color information. Since the color information is classified into 11 colors in the above example, the image feature extraction unit 7 determines which classification of the 11 colors the RGB value of the extracted frame approximates. For example, when the RGB value of the extracted frame is most similar to the RGB value of red among the 11 colors, red is determined as the approximate color information.

[0064] Specifically, the image feature extraction unit 7 can determine the approximate color information by referring to the storage unit 9 in step S13. For example, in the storage unit 9, predetermined RGB values ​​are stored for the distinction of 11 colors. The image feature extraction unit 7 refers to the storage unit 9 and compares the RGB value of the extracted 1 frame with the respective RGB values ​​of the distinction of the 11 colors, thereby determining the color information of the distinction having the most approximate RGB value of the RGB value of the extracted 1 frame. The color information of the distinction having the RGB value most approximate to the RGB value of the extracted 1 frame is determined as the color information approximate to the RGB value of the extracted 1 frame. In addition, the comparison of RGB values ​​can be performed by comparing the respective numerical values ​​of red (R), green (G) and blue (B), or can be performed based on the values ​​calculated by performing a predetermined numerical processing or weighting using the numerical values ​​of red (R), green (G) and blue (B). Therefore, the comparison of RGB values ​​can be performed using a variety of comparison methods.

[0065] The image feature extraction unit 7 accumulates the color information of one frame determined in step S13 (step S14). For example, the image feature extraction unit 7 stores the color information of one frame determined in step S13 in the storage unit 9. In this case, the color information may be temporarily stored at least until the font is determined in step S16.

[0066] Then, the image feature extraction unit 7 determines whether color information has been accumulated for a frame of a specified time (step S15). The specified time can be pre-selected as an appropriate value, for example, it can be selected within a range of 1 second to several seconds. However, the specified time is not limited to the range shown here. Here, it is assumed that the specified time is 1 second. That is, in step S15, the image feature extraction unit 7 determines whether color information has been accumulated for a frame of 1 second. For example, an image of 1 second is composed of images of multiple frames. Therefore, the image feature extraction unit 7 determines whether color information has been accumulated through the processing of steps S11 to S14 for images of the number of frames constituting the image of 1 second.

[0067] When the image feature extraction unit 7 determines that the color information for the predetermined time (here, 1 second) has not been accumulated ("No" in step S15), the image feature extraction unit 7 executes steps S11 to S14 for the next frame of the frame for which the color information has been accumulated in step S14. Thus, the color information for the next frame is accumulated. The image feature extraction unit 7 accumulates the color information for the number of frames for the predetermined time by repeating this operation.

[0068] When the image feature extraction unit 7 determines that the color information has been accumulated for a predetermined amount of time ("Yes" in step S15), the accumulated color information is sent to the display style determination unit 10 as a feature of the image. The display style determination unit 10 determines the display style of the subtitles based on the accumulated color information. In the present embodiment, since the color information, which is a feature of the image, corresponds to the font, which is a display style of the subtitles, the display style determination unit 10 determines the font as the display style of the subtitles based on the accumulated color information (step S16).

[0069] At this time, the display mode determination unit 10 determines the display mode of the subtitles by, for example, referring to a table stored in the storage unit 9. In this example, the display mode determination unit 10 determines the display mode of the subtitles by, for example, referring to Figure 3 The table showing the correspondence between color information and fonts as shown determines the font used as the display style of the subtitles.

[0070] The display style determination unit 10 can determine the font using a variety of determination methods based on the accumulated color information. For example, the accumulated color information contains color information of 1 second's worth of frames. The display style determination unit 10 can determine the font corresponding to the largest number of color information among the color information that stores 1 second's worth of frames as the display style of the subtitles. For example, among the accumulated color information, the number of frames determined to be red is X, the number of frames determined to be orange is Y, and the number of frames determined to be yellow is Z. The display style determination unit 10 determines the font associated with the color information having the largest number of frames among these X, Y, and Z as the display style of the subtitles. For example, when X is greater than Y and Z, the font associated with the color information having X number of frames (in Figure 3 For example, when Y is greater than X and Z, the font corresponding to orange with Y frames (in Figure 3 In the example shown, the font is round Gothic (in the example shown) as the display style of the subtitles. For example, when Z is greater than X and Y, the font associated with the yellow color with Z frames (in Figure 3 In the example shown, Meilio style is determined as the display style of the subtitles.

[0071] The display style determination unit 10 may also use other methods to determine the font. For example, a predetermined operation may be performed based on the color information of the number of frames of 1 second, and the font may be determined based on the information obtained as a result of the operation. As an example, the display style determination unit 10 weights the RGB values ​​of red, orange, and yellow based on X, Y, and Z, which are the respective number of frames, and determines the font based on the RGB value obtained as a result. In this case, for example, the following processing may be performed: color information approximate to the RGB value obtained as a result of the weighting is determined again, and the font associated with the determined color information is determined as the display style of the subtitles.

[0072] When the display style determination unit 10 determines the font, the display control unit 15 displays the subtitles on the display unit 16 in the determined font after processing by the subtitle signal conversion unit 11 and the subtitle signal processing unit 12 (step S17 ).

[0073] The display control unit 15 determines whether the display of subtitles for one sentence has been completed (step S18). The display of subtitles for one sentence refers to the display of subtitles for one screen. That is, the display of subtitles for one sentence refers to the display of subtitles in the range without switching the screen display. For example, when the screen of the subtitle is switched once, subtitles are displayed before and after the switching, and subtitles for one sentence are displayed before and after the switching.

[0074] If the display of the subtitles for one sentence is not completed (No in step S18), the display control unit 15 continues to display the same subtitles until the display of the subtitles for one sentence is completed. On the other hand, if the display of the subtitles for one sentence is completed (Yes in step S18), the process moves to step S11, and the image feature extraction unit 7 extracts the color signal for the next frame.

[0075] By repeating steps S11 to S18 in this way, the subtitle display control system 100 can determine the font based on the color information that is a feature of the image, and display the subtitles in the determined font. Figure 4 In the example shown in the flow of FIG. 1 , the font of the subtitle is determined based on the color information of the beginning of the image of the subtitle for a predetermined time (e.g., 1 second). The subtitle is displayed in the determined font, and according to step S18, the subtitle is continuously displayed in the same font without changing the font until the display of one sentence of the subtitle is completed. Therefore, it is easy to prevent the subtitle from being difficult to see due to frequent font changes.

[0076] but, Figure 4 Step S18 in the process of may not necessarily be executed. In this case, the font of the subtitle is determined based on the color information of the image at a predetermined time interval, and the subtitle is displayed in the determined font. Therefore, the font can be changed more in real time according to the change of the characteristics (color information) of the image.

[0077] In addition, Figure 4 In the above, a processing example is described in which a font of a subtitle is determined based on color information and the subtitle is displayed in the determined font. However, the processing example may also be performed in accordance with Figure 4 The method of performing the process of determining the display style of other subtitles based on other features and displaying them. For example, the subtitle display control system 100 can be based on Figure 4 The method determines the text size based on the volume as a feature of the sound, determines the color of the text of the subtitle based on the expression of the speaker as a feature of the image, determines the display position of the subtitle based on the position of the person or object reflected in the image as a feature of the image, and displays the subtitle in the determined style. In addition, the subtitle display control system 100 can determine multiple elements of the display style of the subtitle based on multiple features, respectively, and appropriately combine them to display the subtitles. For example, the subtitle display control system 100 can display subtitles using a text size determined based on the volume and a font determined based on color information. The same is true for other combinations.

[0078] Thus, in the subtitle display control system 100 according to the present embodiment, the display style determination unit 10 determines the display style of the subtitles based on the characteristics of the image, and displays the subtitles on the display unit 16 in the display style determined by the display style determination unit. Therefore, subtitles that match the atmosphere of the content based on the characteristics of the image can be displayed.

[0079] The subtitle display control system 100 may use methods other than those described in the above embodiments, or may use other methods in addition to the methods described in the above embodiments to display subtitles based on features. Several other processing examples executed by the subtitle display control system 100 are described below.

[0080] [Other processing example 1] In the subtitle display control system 100, the display style determination unit 10 can determine the display style of the subtitles based on the characteristics of the image and the characteristics of the subtitles. Here, the case where the background color of the image is used as the characteristic of the image and a specific character string is used as the characteristic of the subtitles is described.

[0081] The background color of an image is the color of the background of the image. As the background color of an image, for example, the color of the background portion from which a person is removed from the image is used. Alternatively, the color information in the above-mentioned embodiment can also be used as the background color of an image. Therefore, the background color of an image can be determined based on the RGB value of the image or its background. The background color of an image is, for example, divided into multiple colors. The background color of an image can be divided into 11 colors, for example, as in the above-mentioned embodiment. The background color of an image can also be divided based on a different standard from the 11 colors in the above-mentioned embodiment. Here, it is assumed that the background color of an image is divided into two types: a bright background color and a dark background color. As a classification standard for the bright background color and the dark background color, for example, an RGB value for the classification standard pre-stored in a storage unit is used.

[0082] The specific character string may be, for example, a character string representing a specific sound or voice. Here, it is assumed that the specific character string is a character string representing a shout and a laugh, respectively. The character strings representing the shout and the laugh are, for example, respectively predetermined and stored in the storage unit 9. For example, as a character string representing a shout, "ah" and "wow" are stored. In addition, as a character string representing a laugh, "wahaha" or "chuckle" are stored.

[0083] The storage unit 9 stores, for example, a table associating features of images and features of subtitles with display styles of subtitles. Figure 5 FIG. 2 is a diagram showing an example of a table stored in the storage unit 9 in another processing example 1. Figure 5 In the example shown, the combination of shouting and laughing as the characteristics of the subtitles and the bright background color and dark background color as the characteristics of the image determines a total of four display modes. Specifically, shouting and bright background color are associated with the first display style. Similarly, shouting and dark background color are associated with the second display style, laughter and bright background color are associated with the third display style, and laughter and dark background color are associated with the fourth display style.

[0084] The first display style to the fourth display style are specific display styles associated with subtitles, for example, defined by a combination of one or more of the text size, font and color of the subtitles. The first display style to the fourth display style are predetermined as display styles that represent the characteristics of the corresponding image and the characteristics of the subtitles, respectively. The first display style associated with the shout and the bright background color is a brisk font such as white or yellow, which reminds people of joyful cheers. The second display style associated with the shout and the dark background color is a blurred font such as red or gray, which reminds people of horror and tragic feelings. The third display style associated with laughter and a bright background color is a popular font such as light blue or pink, which reminds people of an exciting or lively atmosphere. The fourth display style associated with laughter and a dark background color is a dim font such as brown or black, which reminds people of a pursed smile, etc. However, the display style described here is an example, and the first display style to the fourth display style can be set to other appropriate display styles.

[0085] In another processing example 1, the display style determination unit 10 determines the display style of the subtitles based on the features of the image and the features of the subtitles. For example, as the features of the image, the image feature extraction unit 7 uses the RGB values ​​for the classification criteria stored in the storage unit 9 to determine whether the image has a bright background color or a dark background color. In addition, as the features of the subtitles, the subtitle feature extraction unit 6 determines whether a character string indicating shouting or laughter is included in the subtitles. The display style determination unit 10 determines the display style of the subtitles based on whether the image has a bright background color or a dark background color and whether the image contains a character string indicating shouting or laughter.

[0086] For example, when the image has a bright background color and includes a character string indicating a call, the display style determination unit 10 refers to, for example, Figure 5 For example, when the image has a dark background color and contains a character string representing a call, the display style determination unit 10 refers to, for example, Figure 5 The display style determination unit 10 determines the display style of the subtitles including the character string representing the shout to be the second display style. Similarly, for the case of including the character string representing the laugh, the display style determination unit 10 determines the display style of the subtitles to be the third display style or the fourth display style, depending on whether the background color is light or dark. When the display style determination unit 10 determines the display style, the display control unit 15 displays the subtitles on the display unit 16 in the determined display style through processing by the subtitle signal conversion unit 11 and the subtitle signal processing unit 12.

[0087] Since the first to fourth display styles are determined based on a specific character string that is a characteristic of the subtitles and a background color that is a characteristic of the image, by determining the display style of the subtitles based on the characteristics of the subtitles and the characteristics of the image and displaying the subtitles, subtitles that are more in harmony with the atmosphere of the content can be displayed.

[0088] [Other processing example 2] In the subtitle display control system 100, the display style determination unit 10 can determine the display style based on the type (genre) of the content. The type of content is information pre-assigned to the content and represents the classification of the content of the content. The type of content can be appropriately classified, for example, into "news / report", "sports", "information / wide show", "TV series", "music", "variety show", "movie", "animation / special effects", "documentary / science and education", "theater / performance" and "others". However, the classification shown here is only an example. The information on the type of content is contained in the multiplexed content signal, for example. The display style determination unit 10 is able to identify the type of content based on the information on the type contained in the content signal.

[0089] For example, a table in which display styles corresponding to graphics (images) of each type are associated with each content type is stored in advance in the storage unit 9. The display style determination unit 10 can refer to the table to determine the display style of the subtitles based on the type information. In addition, the display style of the subtitles can be set to the font of the subtitles, for example, but is not limited thereto.

[0090] Furthermore, in the subtitle display control system 100, as a method of displaying subtitles, it is also possible to configure the subtitle display control system 100 to be able to selectively select between displaying subtitles on the screen and displaying subtitles off the screen. Fig. 6A As schematically shown in FIG. 1 , the subtitles on the screen are displayed by displaying the image of the content on the entire display unit 16 (display screen) and superimposing the subtitles on the image of the content. Figure 6B As schematically shown in FIG. 1 , the subtitles are displayed off-screen in a manner that a dedicated area for displaying subtitles is provided on the display unit 16 (display screen) so that the subtitles are displayed without overlapping the content image. Fig. 6A and Figure 6B In the embodiment, the subtitles are displayed at the lower center of the display unit 16, but the display position of the subtitles is not necessarily the lower center of the display unit 16, and may be the upper part or the left and right part. For example, the subtitles may be displayed on the screen or off the screen based on a predetermined operation input by the user viewing the content.

[0091] When the display style determination unit 10 selects subtitles on the screen according to the type of content, it can add animation (text animation) to the subtitles as the display style of the subtitles. For example, a table associating the type of content with whether to add animation to the subtitles is stored in advance in the storage unit 9. The display style determination unit 10 can refer to the table to determine which type of animation to add to the subtitles.

[0092] Figure 7 2 is a diagram showing an example of a table stored in the storage unit 9 in another processing example 2. Figure 7 As shown in FIG. 1 , the type, font, and display style of the subtitle are stored in a table in association with each other. The display style of the subtitle is a part of the display style of the subtitle. Figure 7 In the example shown, for the case where the subtitles are on the screen and the case where the subtitles are off the screen, it is stored whether to display them in the style of subtitles or to display them with additional animation. Figure 7 In the case where the subtitle is recorded as "subtitle", the subtitle text (character) is displayed as a normal subtitle, for example, at a fixed position on the top, bottom, left, or right of the screen. Figure 7 When "animation" is recorded in the text of the subtitle, the text of the subtitle is displayed on the display screen with animation effects such as appropriate changes in display position, subtitle movement, and enlargement / contraction. In addition, what kind of animation is added can be predetermined, or it can be determined by a predetermined determination method based on the text of the subtitle.

[0093] For example, if the type of content is a TV series, the subtitles can be displayed as an animation near the person who speaks the language of the subtitles. This makes it possible to identify the speaker and improve the sense of presence. Also, for example, if the type of content is a variety show, the interest of the program can be increased by moving or enlarging / shrinking the subtitles as an animation. In addition to the examples shown here, depending on the type of content, by displaying the subtitles with appropriate animations, the atmosphere of the content can be more easily conveyed to the user as the viewer.

[0094] In addition, Figure 7 When "Subtitles / Animation" is recorded in the subtitles, the subtitle text is displayed as a normal subtitle at a fixed position or with an animation effect, depending on the content. For example, the subtitle text of the explanatory content is displayed as a normal subtitle, and the subtitle text expressing the character's lines is displayed with an animation effect.

[0095] In another processing example 2, the display mode determination unit 10 refers to, for example, Figure 7The display style determination unit 10 determines the font as the display style of the subtitles based on the information of the content type and the display method determined based on the user's input (i.e., subtitles on the screen and subtitles off the screen). For example, when the type is sports and the display method is subtitles on the screen, the display style determination unit 10 refers to Figure 7 , the font is determined to be "Ming Dynasty Style", and the display method is determined to be "Subtitles". For example, when the genre is variety show and the display method is to display subtitles outside the screen, the display style determination unit 10 refers to Figure 7 For example, when the type is animation / special effects and the display method is subtitles on the screen, the display style determination unit 10 refers to Figure 7 The font is determined as "Gothic" and the display mode is determined as "subtitles". When the display mode determination unit 10 determines the display mode (font and display mode), the display control unit 15 displays the subtitles on the display unit 16 in the determined display mode after processing by the subtitle signal conversion unit 11 and the subtitle signal processing unit 12.

[0096] Thus, since the subtitles are displayed in a font consistent with the graphics of the content type, subtitles that are more harmonious with the atmosphere of the content can be displayed. In addition, when the subtitles are on the screen, by adding animation to the subtitles according to the type of content, it is easier to convey the atmosphere of the content to the user as the viewer.

[0097] [Other processing example 3] In the subtitle display control system 100, the display mode determination unit 10 can replace the explanatory sentence related to the sound with a character representing the sound represented by the explanatory sentence related to the sound when the feature of the subtitle is a descriptive sentence related to the sound. As described in the above embodiment, the explanatory sentence related to the sound is extracted as a feature of the subtitle by the subtitle feature extraction unit 6, for example.

[0098] The storage unit 9 stores in advance a table that associates, for example, description sentences related to sound with characters representing the sound represented by the description sentences related to sound. The display mode determination unit 10 replaces, for example, the description sentence related to the extracted sound with characters representing the sound recorded in the table.

[0099] In addition, as a feature of the image, the display style determination unit 10 can make the display style of the text representing the replaced sound different based on whether the image contains a sound source that emits a description sentence related to the sound. For example, the display style determination unit 10 makes the display method different based on whether the sound source is included in the image. That is, the display style determination unit 10 makes the text representing the sound different whether it is displayed as a normal subtitle or with an animation. The display style determination unit 10 can, for example, refer to a specified table stored in the storage unit 9 to make the display style different.

[0100] Figure 8 2 is a diagram showing an example of a table stored in the storage unit 9 in another processing example 3. Figure 8 As shown in FIG. 1 , the table stores the description sentences related to the sound, the text representing the sound, and the display style of the subtitles in association with each other. The display style of the subtitles is a part of the display style of the subtitles. Figure 8 In the example shown, for the cases of "sound source present" indicating that the sound source is included in the image and "sound source absent" indicating that the sound source is not included in the image, it is stored whether to display it as subtitles or with animation.

[0101] In another processing example 3, the display style determination unit 10 replaces the description sentence related to the sound as a feature of the subtitle with text representing the sound, and determines the display style of the subtitle based on the description sentence related to the sound as a feature of the subtitle and the presence or absence of the sound source as a feature of the image. For example, the subtitle feature extraction unit 6 extracts the subtitle of the description sentence "rain sound" as a feature of the subtitle. In this case, the display style determination unit 10 refers to, for example, Figure 8 As shown in the table, the subtitle "Rain Sound" is replaced with a subtitle representing the sound "Whoa~". Furthermore, the image feature extraction unit 7 extracts whether "rain", which is the sound source of the rain sound, is included in the image in the image displaying the subtitle as a feature of the image. In the case where "rain" is included in the image, that is, when there is a sound source, the display style determination unit 10 determines to display the replaced subtitle "Whoa~" with an animation as the display style of the subtitle. On the other hand, in the case where "rain" is not included in the image, that is, when there is no sound source, the display style determination unit 10 determines to display the replaced subtitle "Whoa~" as a normal subtitle as the display style of the subtitle. For the subtitle "Whoa~", when an animation is added, the subtitle "Whoa~" is displayed, for example, by moving the subtitle "Whoa~" on the display screen.

[0102] For example, the subtitle feature extraction unit 6 extracts the subtitle of the explanatory sentence "telephone ringing" as the subtitle feature. In this case, the display mode determination unit 10 refers to, for example, Figure 8The table shown in FIG. 1 replaces the subtitle "Phone Ringing" with a subtitle representing the sound "Beep Beep Beep". Furthermore, the image feature extraction unit 7 extracts whether the image contains "Phone" as the sound source of the phone ringing as a feature of the image in the image displaying the subtitle. In the case where "Phone" is included in the image, that is, when there is a sound source, the display style determination unit 10 determines to display the replaced subtitle "Beep Beep Beep" with an animation as the display style of the subtitle. In this case, the subtitle "Beep Beep Beep" is, for example, arranged and displayed near the displayed phone on the display screen. On the other hand, in the case where "Phone" is not included in the image, that is, when there is no sound source, the display style determination unit 10 determines to display the replaced subtitle "Beep Beep Beep" as a normal subtitle as the display style of the subtitle. In this case, the subtitle "Beep Beep Beep" is, for example, displayed at the lower center on the display screen.

[0103] Thus, the explanation sentence related to the sound is replaced with the text representing the sound and displayed, so that the user can have a more realistic sense of presence. In addition, since the display style can be different according to the presence or absence of the sound source, the display style can be different not only based on the characteristics of the subtitles, but also based on the characteristics of the image.

[0104] In addition, Figure 8 In the table shown, in the description sentence related to each sound, not only one character representing the sound may be associated, but multiple characters representing the sound may be associated. In this case, the character representing the sound may be determined based not only on the description sentence related to the sound that is a feature of the subtitle, but also on the feature of the image.

[0105] For example, in Figure 8In the table shown, the text indicating the sound "beep, beep, beep" is associated with the explanatory sentence about the sound "siren". However, the explanatory sentence related to the sound "siren" may be associated with text indicating three sounds, such as "beep, beep, beep", "woooo ... When the identified type of vehicle is a fire truck, the display style determination unit 10 determines “Woo-Chan-Chan-Chan” as the text representing the sound, and converts the subtitle of the explanatory sentence “Siren sound” into the subtitle “Woo-Chan-Chan-Chan”.

[0106] As another example, when the display style determination unit 10 extracts a subtitle of the explanatory sentence "rain sound" as a feature of the subtitle, the display style determination unit 10 obtains the rainfall pattern in the image as a feature of the image. The display style determination unit 10 may replace the subtitle of the explanatory sentence "rain sound" with "tick-tick-tick", "tick-tick-tick", "whoosh", "whoosh-whoosh", etc. according to the rainfall pattern.

[0107] [Second embodiment] Fig. 9 2 is a block diagram showing the configuration of a subtitle display control system 200 according to the second embodiment. In the second embodiment, the same reference numerals are given to the configurations common to the first embodiment, and the description thereof will be omitted as appropriate.

[0108] like Fig. 9 As shown, unlike the subtitle display control system 100 involved in the first embodiment, the subtitle display control system 200 involved in the second embodiment does not include the content signal receiving unit 1, the signal separation unit 2, the subtitle signal decoding unit 3, the image signal decoding unit 4 and the sound signal decoding unit 5. The subtitle display control system 200 involved in the second embodiment includes a plurality of signal receiving units for respectively receiving the subtitle signal, the image signal and the sound signal of the content. Specifically, Fig. 9 As shown, the subtitle display control system 200 includes a subtitle signal receiving unit 23 , an image signal receiving unit 24 , and a sound signal receiving unit 25 .

[0109] In the second embodiment, the server 30 stores subtitle data, image data, and sound data of the content. The server 30 sends a subtitle signal, an image signal, and a sound signal, which respectively represent the subtitle data, the image data, and the sound data, to the subtitle display control system 200. The subtitle signal receiving unit 23 receives the subtitle signal sent from the server 30. The image signal receiving unit 24 receives the image signal sent from the server 30. The sound signal receiving unit 25 receives the sound signal sent from the server 30. That is, unlike the first embodiment, the subtitle display control system 200 involved in the second embodiment receives a subtitle signal, an image signal, and a sound signal, instead of a multiplexed content signal. The subtitle signal receiving unit 23 outputs the received subtitle signal to the subtitle feature extraction unit 6. The image signal receiving unit 24 outputs the received image signal to the image feature extraction unit 7. The sound signal receiving unit 25 outputs the received sound signal to the sound feature extraction unit 8. As for the subsequent processing, since it is the same as the first embodiment, the specific description is omitted.

[0110] In the second embodiment, each signal of the content may not necessarily be provided to the subtitle display control system 200 from one server 30. Each signal of the content may be provided from two or more servers. Fig.10 As shown, the subtitle signal may be provided from the first server 31, and the image signal and the sound signal may be provided from the second server 32. In this case, the first server 31 stores the subtitle data of the content, and the second server 32 stores the image data and the sound data of the content. The combination of each data stored with a plurality of data is not limited to the combination shown here, and may be any combination. In addition, each signal of the content may not necessarily be provided from two servers, and may be provided from three or more servers.

[0111] [Third Embodiment] Fig.11 1 is a block diagram showing the configuration of a caption display control system 300 according to the third embodiment. In the third embodiment, the same reference numerals are given to the configurations common to the first embodiment, and the description thereof will be omitted as appropriate.

[0112] like Fig.11 As shown, unlike the subtitle display control system 100 involved in the first embodiment, the subtitle display control system 300 involved in the third embodiment further includes an image data cutting processing unit 41 and a conversion processing unit 42 based on the subtitle display control system 100 involved in the first embodiment.

[0113] The image data cutting processing unit 41 obtains the decoded image signal from the image signal decoding unit 4. The image data cutting processing unit 41 cuts out the text included in the image shown by the image signal. For example, the image data cutting processing unit 41 uses a well-known image recognition technology to extract the text information embedded in the image to cut out the text. The image data cutting processing unit 41 extracts the text embedded in the image, so that the text is not included in the subtitles shown by the subtitle signal. That is, the image data cutting processing unit 41 cuts out the text included in the image.

[0114] As an example, assume that the image represented by the image signal is Fig.12 The image shown in FIG. 1 has the words "Autumn leaves scenery of Kyoto" embedded in the image. The image data cutting processing unit 41 Fig.12 In the image shown, the text "the scenery of red leaves in Kyoto" embedded in the image is cut out using image recognition technology. The image data cutting processing unit 41 inputs the data of the cut text (here, the text "the scenery of red leaves in Kyoto") to the conversion processing unit 42.

[0115] The conversion processing unit 42 converts the text cut by the image data cutting processing unit 41 into a subtitle signal. In this example, the conversion processing unit 42 converts the data of the text "Red leaves scenery in Kyoto" obtained from the image data cutting processing unit 41 into a subtitle signal. The conversion processing unit 42 inputs the converted subtitle signal to the subtitle feature extraction unit 6.

[0116] The processing performed by the subtitle feature extraction unit 6 is the same as that of the first embodiment. That is, the subtitle feature extraction unit 6 extracts the features of the subtitles, and the display style determination unit 10 determines the display style of the subtitles based on the features of the image. In the third embodiment, the subtitle signal sent from the conversion processing unit 42 is also processed in the same manner as the subtitle signal input from the subtitle signal decoding unit 3. Therefore, the display style determination unit 10 determines the display style of the subtitles represented by the subtitle signal converted by the conversion processing unit 42, for example, based on the features of the image. The subtitles are displayed on the display unit 16 in the display style determined by the display style determination unit 10.

[0117] In the third embodiment, the image signal processing unit 13 may remove the text cut out by the image data cutting processing unit 41 from the image represented by the image signal through image processing. Fig.12 In the example shown, in the image shown by the image signal, the text "The red leaves scenery of Kyoto" is eliminated by the image signal processing unit 13, and instead, the text "The red leaves scenery of Kyoto" converted into a subtitle signal by the conversion processing unit 42 is displayed in the display style determined by the display style determination unit 10.

[0118] [Fourth Embodiment] Fig.134 is a block diagram showing the configuration of a subtitle display control system 400 according to the fourth embodiment. In the fourth embodiment, the same reference numerals are given to the configurations common to the first embodiment, and the description thereof will be omitted as appropriate.

[0119] like Fig.13 As shown, the subtitle display control system 400 according to the fourth embodiment is based on the subtitle display control system 100 according to the first embodiment and further includes a sound data cutting processing unit 43 and a conversion processing unit 44.

[0120] The audio data cutting processing unit 43 obtains the decoded audio signal from the audio signal decoding unit 5. The audio data cutting processing unit 43 cuts the audio represented by the audio signal. For example, the audio data cutting processing unit 43 cuts the audio using a known audio recognition technology.

[0121] As an example, assume that the sound represented by the sound signal includes a narration (speech) of "What beautiful red leaves". In this case, the sound data cutting processing unit 43 cuts the sound of "What beautiful red leaves" using the sound recognition technology. The sound data cutting processing unit 43 inputs the data of the cut sound to the conversion processing unit 44.

[0122] The conversion processing unit 44 converts the sound cut by the sound data cutting processing unit 43 into a subtitle signal. In this example, the conversion processing unit 44 converts the sound data of "What beautiful red leaves" obtained from the sound data cutting processing unit 43 into a subtitle signal. The conversion processing unit 44 inputs the converted subtitle signal to the subtitle feature extraction unit 6.

[0123] The processing performed by the subtitle feature extraction unit 6 is the same as that of the first embodiment. That is, the subtitle feature extraction unit 6 extracts the features of the subtitles, and the display style determination unit 10 determines the display style of the subtitles based on the features of the image. In the fourth embodiment, the subtitle signal sent from the conversion processing unit 44 is also processed in the same way as the subtitle signal input from the subtitle signal decoding unit 3. Therefore, the display style determination unit 10 determines the display style of the subtitles represented by the subtitle signal converted by the conversion processing unit 44, for example, based on the features of the image. The subtitles are displayed on the display unit 16 in the display style determined by the display style determination unit 10.

[0124] In the above-described embodiments and other processing examples, the contents of each table stored in the storage unit 9 may be preset and stored in the storage unit 9, but may also be set by the user through a predetermined operation input. Figure 2 In the table shown, the user can freely set the relationship between volume and text size. Figure 3 In the table shown in FIG. 1 , the user can freely set the relationship between color information and fonts. In addition, for example, in Figure 8 In the table shown, the user can freely set the text representing the sound corresponding to each of the explanatory sentences about the sound. Figure 8 In the example shown, the description sentence related to the sound "telephone ringing" is associated with the text representing the sound "beep beep beep", but the user can associate the text "ring beep" as the text representing the sound instead of the text "beep beep beep". The same is true for other tables.

[0125] When the user sets the contents of each table stored in the storage unit 9, the user can set the table with contents matching the graphics while referring to actual TV broadcasts, or set the table according to the contents of preference. Alternatively, the user can search or download the font data, text color, or onomatopeia (text representing sound) of the user through the network to set the table.

[0126] In the above embodiment, the content signal is described as a signal that multiplexes the subtitle signal, the image signal, and the sound signal. However, the content signal may be a signal that multiplexes any combination of the subtitle signal, the image signal, and the sound signal. In other words, the content signal is not necessarily a signal that multiplexes all of the subtitle signal, the image signal, and the sound signal. In this case, the signal separation unit 2 may separate the multiplexed content signal into the original signal.

[0127] In addition, in the above-mentioned embodiment, regarding the association between the elements and features of the display style of the subtitles, it is explained that the text size is associated with the volume, the font is associated with the color information, the color of the text is associated with the expression of the speaker, and the display position is associated with the position of the person or object reflected in the image. However, the association between the elements and features related to the display style of the subtitles is not limited to this, and any association can be made. Therefore, the display style determination unit 10 can determine the display style associated with the feature based on the pre-set association according to the feature of the subtitle, image or sound.

[0128] The present disclosure is described based on the drawings and embodiments, but it should be noted that it is easy for those skilled in the art to make various deformations and modifications based on the present disclosure. Therefore, it should be noted that these deformations and modifications are included in the scope of the present disclosure. For example, the functions included in each functional unit or each step can be reconfigured in a manner that is not logically contradictory, and multiple functional units or steps can be combined into one, or divided.

Claims

1. A subtitle display control system, characterized in that: include: A display unit, which displays content; A feature extraction unit, which extracts features of the content; A display style determination unit that determines a display style of subtitles based on the characteristics of the content; as well as A display control unit displays the subtitles on the display unit in the display mode determined by the display mode determination unit.

2. The subtitle display control system according to claim 1, characterized in that: The device further includes a signal separation unit that separates a subtitle signal, an image signal, and a sound signal from a multiplexed received signal related to the content. The feature extraction unit extracts a feature of an image from the image signal as a feature of the content, The display mode determination unit determines a display mode of the subtitles included in the subtitle signal based on a feature of the image.

3. The subtitle display control system according to claim 2, characterized in that: The display mode determination unit determines the display mode by referring to a table indicating a preset correspondence relationship between the image feature and the display mode, based on the image feature.

4. The subtitle display control system according to claim 2, characterized in that: The signal separation unit further separates a sound signal from the received signal. The system further includes a sound feature extraction unit that extracts a feature of the sound of the content from the sound signal. The display mode determination unit determines the display mode based on a feature of the image and a feature of the sound.

5. The subtitle display control system according to claim 1, characterized in that: The display style includes at least one element of the text size, font, color, and display position of the subtitle.

6. The subtitle display control system according to claim 2, characterized in that: The display mode determination unit determines a display position as a display mode of the subtitles based on a position of a sound source as a feature of the image.

7. The subtitle display control system according to claim 2, characterized in that: The system further includes a subtitle feature extraction unit, which extracts features of subtitles of the content from the subtitle signal. The display mode determination unit determines the display mode based on a feature of the image and a feature of the subtitle.

8. The subtitle display control system according to claim 7, characterized in that: The display mode determination unit determines the display mode based on color information of the image as a feature of the image, when the subtitles include a specific character string as a feature of the subtitles.

9. The subtitle display control system according to claim 1, characterized in that: The display style determination unit determines a font as the display style based on a genre of the content.

10. The subtitle display control system according to claim 1, characterized in that: As a method of displaying the subtitles, it is possible to select one of displaying the subtitles on the screen in a manner that is superimposed on the image and displaying the subtitles outside the screen in a manner that is not superimposed on the image. The display style determination unit adds animation to the subtitles as the display style when the subtitles are selected on the screen according to the type of the content.

11. The subtitle display control system according to claim 2, characterized in that: The system further includes a subtitle feature extraction unit, which extracts features of subtitles of the content from the subtitle signal. When the feature of the subtitle includes a description sentence related to sound, the display mode determination unit replaces the description sentence related to sound with a character representing the sound represented by the description sentence related to sound.

12. The subtitle display control system according to claim 11, characterized in that: The display style determination unit makes the display style of the replaced characters representing the sound different based on whether a sound source that is a feature of the image is included in the image, the sound source being a sound represented by an explanatory sentence related to the sound.

13. The subtitle display control system according to claim 1, characterized in that: The method further includes a plurality of signal receiving units for respectively receiving a subtitle signal, an image signal, and a sound signal of the content.

14. The subtitle display control system according to claim 2 or 13, characterized in that: Also includes: an image data cutting processing unit for cutting out characters included in the image indicated by the image signal; as well as a conversion processing unit that converts the text cut out by the image data cutting processing unit into the subtitle signal, The display mode determination unit determines a display mode of subtitles represented by the subtitle signal converted by the conversion processing unit.

15. The subtitle display control system according to claim 14, characterized in that: The system further includes an image signal processing unit that removes the text cut out by the image data cutting processing unit from the image represented by the image signal through image processing.

16. The subtitle display control system according to claim 2 or 13, characterized in that: It also includes: a sound data cutting processing unit, which cuts the sound represented by the sound signal; and a conversion processing unit that converts the sound cut by the sound data cutting processing unit into the subtitle signal, wherein the subtitle signal indicates a character expressing the sound, The display mode determination unit determines a display mode of subtitles represented by the subtitle signal converted by the conversion processing unit.

17. A subtitle display control method, which is a subtitle display control method executed by a subtitle display control system, characterized in that: include: Steps for extracting features of the content; The step of determining a display style of subtitles based on the characteristics of the content; as well as The step of displaying the subtitles on a display unit in the determined display style.

Images