Advertisement provision system and advertisement provision method

The advertisement system addresses the issue of distracting viewers by using line-of-sight detection and eye attraction analysis to optimize ad display, ensuring minimal disruption during focused viewing.

JP7578904B2Active Publication Date: 2024-11-07MAZDA MOTOR CORP
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
JP2020198236
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2020-11-30
Publication Date
2024-11-07
Estimated Expiration
2040-11-30

AI Technical Summary

Technical Problem

Existing advertisement systems display overlay advertisements at inopportune times, distracting viewers who are concentrating on program content, leading to a bothersome viewing experience.

Method used

An advertisement system that utilizes line-of-sight detection to determine viewer concentration and eye attraction levels, adjusting the display of advertisements to avoid interrupting viewer focus by superimposing ads when attention is diverted from important image areas.

Benefits of technology

The system effectively provides advertisements without disturbing viewers by synchronizing display with their attention levels, enhancing ad recognition when appropriate and avoiding distraction during high concentration periods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007578904000001
    Figure 0007578904000001
  • Figure 0007578904000002
    Figure 0007578904000002
  • Figure 0007578904000003
    Figure 0007578904000003
Patent Text Reader

Abstract

To provide an advertisement provision system and an advertisement provision method capable of providing an overlay advertisement without causing a viewer to be bothered.SOLUTION: An advertisement provision system S that provides an advertisement by using a program distribution company 4 that distributes a display image including an advertisement display area A on which advertisement information can be superimposed, and a monitor 31 that displays the distributed display image includes a first camera 32 that detects the line of sight of a user who sees the image displayed on the monitor 31, a meaningful area setting unit 41 that detects a meaningful area P that displays an important portion of the image in the image, and an advertisement synthesis unit 45 that controls a second image (advertisement information) superimposed on the advertisement display area A, and the advertisement synthesis unit 45 superimposes and displays the second image on the advertisement display area A when a visual object L does not exist in the meaningful area P.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an advertisement providing system and an advertisement providing method that provide advertisement information using an image management means that delivers an image including an advertisement posting area on which advertisement information can be superimposed, and an image display means that displays the image delivered from the image management means. [Background technology]

[0002] A technology has been known for some time that involves synthesizing advertising information for real products or services (hereinafter referred to as the second image) with an advertising display area that constitutes part of video information (hereinafter referred to as the first image) of a movie or the like, such as a billboard or street vision display shown as part of the scenery in the image, to create program content, which is a new image for display, and then providing this program content to a viewer (user).

[0003] The mapping system of Patent Document 1 comprises a display for displaying the received display image, a data control means for inputting a base data stream (first image) consisting of metadata etc. and creating data to be sent to the display, a synthesis condition setting means for inputting a composite data stream (second image) consisting of advertisements etc. and setting synthesis conditions, and a synthesis means for synthesizing the composite data set by the synthesis condition setting means into the advertisement display area of ​​the base data to create a new display image (program content).

[0004] The object that a viewer's gaze is directed at (the visual target) reflects the viewer's interest and awareness, and the degree of awareness, and therefore tends to be used as an indicator of the viewer's intentions. The car navigation system of Patent Document 2 has a gaze detection means for detecting the driver's gaze, a display means for displaying a map of the direction of travel, an audible indication means for audibly indicating the direction to the driver, and a control means for controlling each of these devices, and the control means displays the map when the driver's gaze is directed toward the display means, and audibly provides direction information to the driver when the driver's gaze is not directed toward the display means. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Special table number 2016-515327 [Patent Document 2] Special table number 2014-516181 Summary of the Invention [Problem to be solved by the invention]

[0006] The mapping system of Patent Document 1 can synthesize advertising information that has been scheduled for display in advance into an advertising display area on the screen at a specified time or at a specified cycle, and can display advertising information superimposed on the advertising display area (hereinafter referred to as overlay advertising) naturally as part of the scenery when the viewer is watching program content. However, with the technology of Patent Document 1, since overlay advertisements may be displayed when a viewer is concentrating on the program content, there is a risk that the viewer who is engrossed in the program may find the overlay advertisements annoying.

[0007] Viewers can understand 10% of their impressions from visual information. 9 The viewer has a fixation mechanism that selects salient information (pop-up target) at a speed of 10000 bits / s. The viewer unconsciously (reflexively) selects bottom-up or top-down attention depending on the concentration state of consciousness. Bottom-up attention is a passive process in which we recognize and process objects that stand out because they have physical features that clearly distinguish them from other objects in our visual field, whereas top-down attention is an active process in which we determine a target feature and then intentionally select and recognize an object based on that feature.

[0008] Therefore, for example, when a viewer is concentrating on watching a soccer broadcast, which is a sports program, the viewer focuses top-down attention on the ball and the player manipulating the ball. In contrast, when advertising information is displayed in an advertising display area built into the fence, viewers will reflexively direct their attention in a bottom-up manner to the suddenly appearing overlay advertisement. Usually, advertisements are composed of display images with elements (brightness, color, direction, etc.) that are highly eye-catching (also called visual salience or saliency), and from an ergonomic point of view, highly eye-catching areas in the visual field attract the viewer's gaze reflexively. Therefore, bottom-up attention becomes even stronger. That is, it is not easy to provide overlay advertisements without causing annoyance to the viewer.

[0009] An object of the present invention is to provide an advertisement providing system, an advertisement providing method, and the like that can provide overlay advertisements without bothering the viewer. [Means for solving the problem]

[0010] The invention of claim 1 is an advertising provision system that provides advertisements using an image management means that distributes an image including an advertising posting area on which advertising information can be superimposed, and an image display means that displays the image distributed from the image management means, the system having a gaze detection means that detects the gaze of a viewer viewing the image, a meaningful area detection means that detects a meaningful area within the image that displays important parts of the image, and an advertising display control means that controls the advertising information to be superimposed on the advertising posting area, and is characterized in that the advertising display control means superimposes the advertising information on the advertising posting area when the viewer's gaze is not in the meaningful area.

[0011] This advertisement providing system has a gaze detection means for detecting the gaze of a viewer viewing the image, and can therefore determine a visual target that attracts the viewer's gaze. It has a meaningful area detection means for detecting a meaningful area within the image that displays an important part of the image, and can therefore detect a meaningful area that the creator wants the viewer to be interested in and direct their gaze to, and can visually display it. It has an advertisement display control means for controlling the advertisement information superimposed on the advertisement posting area, and can therefore display the advertisement information without requiring a separate advertisement display time that interrupts the image. The advertisement display control means superimposes and displays the advertisement information on the advertisement posting area when the viewer's gaze is not in the meaningful area, and can therefore display the advertisement information without disturbing the viewer's concentration when the viewer is distracted from the image and has a low level of concentration.

[0012] The invention of claim 2 is characterized in that, in the invention of claim 1, it has an attractiveness detection means capable of detecting the attractiveness of the semantic area, and the advertising display control means changes the display form of the superimposed advertising information in accordance with the attractiveness of the semantic area when the viewer's line of sight is present in the semantic area. According to this configuration, the display format of the advertising information can be changed according to the concentration level of the viewer.

[0013] The invention of claim 3 is characterized in that, in the invention of claim 2, the advertising display control means prohibits the display of the superimposed advertising information when the attractiveness of the semantic area is lower than a predetermined threshold. According to this configuration, when the viewer is in a state of high concentration, ie, engrossed in the image, the display of the advertising information is prohibited, so that the viewer's concentration is not disturbed.

[0014] The invention of claim 4 is characterized in that, in the invention of claim 1, it has an attractiveness detection means capable of detecting the attractiveness of each area divided in association with the displayed image, and the advertising display control means changes the display form of the superimposed advertising information according to the attractiveness of the destination when the viewer's gaze moves from the semantic area to outside the semantic area. According to this configuration, the attention level of the advertising information can be set in accordance with the concentration level of the viewer.

[0015] The invention of claim 5 is characterized in that, in the invention of claim 4, the advertisement display control means increases the attractiveness of the superimposed advertising information when the attractiveness of the destination is high compared to when the attractiveness of the destination is low. According to this configuration, when the degree of attraction of the destination is high, the gaze is reflexively drawn to the degree of attraction and moves in a random state (low concentration), so the advertisement's attention level can be increased to increase the viewer's recognition of the advertisement.When the degree of attraction of the destination is low, the gaze is intentionally moved to the target of attention and moves in a focused state (medium concentration), so the advertisement's attention level can be decreased to avoid disruption of the focused state.

[0016] The invention of claim 6 is an advertising provision method that provides advertisements using an image management means that distributes an image including an advertising display area on which advertising information can be superimposed, and an image display means that displays the image distributed from the image management means, and is characterized by having a gaze detection step that detects the gaze of a viewer viewing the image, a meaningful area detection step that detects a meaningful area within the image that displays an important part of the image, and an advertising display step that superimposes advertising information on the advertising display area when the viewer's gaze is not in the meaningful area. According to this configuration, basically the same effect as that of claim 1 can be achieved. Effect of the Invention

[0017] According to the advertisement providing system and advertisement providing method of the present invention, by determining the concentration level based on the viewer's line of sight and the degree of visual attraction, it is possible to provide an overlay advertisement without causing the viewer any annoyance. [Brief description of the drawings]

[0018] [Figure 1] 1 is a block diagram showing a functional configuration of an advertisement providing system according to a first embodiment. [Diagram 2] These are example screens from a soccer broadcast program, where (a) shows a gaze image in which the semantic area is the visual target, and (b) shows a gaze image in which the semantic area is not the visual target. [Diagram 3] 1A and 1B are example screens from a movie program, in which (a) shows a gaze image in which a semantic region is the visual target, and (b) shows a gaze image in which the semantic region is not the visual target. [Figure 4] 1A and 1B are example screens from a news program, in which (a) shows a gaze image in which a semantic region is the visual target, and (b) shows a gaze image in which the semantic region is not the visual target. [Diagram 5] 13 is a flowchart of an advertisement display control process. [Figure 6] 13 is a graph showing the relationship between eye-attractiveness and gaze frequency. [Figure 7] 13 is a graph showing the relationship between visual attraction and probability. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. The following description is an example of the application of the present invention to an advertisement providing system, and is not intended to limit the present invention, its applications, or its uses. EXAMPLES

[0020] A first embodiment of the present invention will be described below with reference to FIGS. As shown in Figure 1, the advertising provision system S is composed of main components including a video production company 1 (image management means), an advertising production company 2, a user area 3, and a program distribution company 4, each of which is connected to each other via a network 5 so that they can communicate with each other. The following description includes a description of a method for providing advertisements.

[0021] Based on requests from clients, video production company 1 produces program information (hereinafter referred to as first images), which is image content such as sports broadcasts, movies, dramas, variety shows, and news reports, and distributes it over network 5. This video production company 1 includes, for example, a storage unit 11 that stores a plurality of first images, and a video control unit 12 that extracts a specified first image from the plurality of stored first images from storage unit 11 in real time, and transmits the extracted first image after performing a predetermined processing.

[0022] Based on a client's request, the advertising production company 2 creates still image or video advertising information (hereinafter referred to as second images), which is image content that stimulates purchasing desire for products or services, and distributes it over the network 5. This advertising production company 2 includes, for example, a memory unit 21 that stores a plurality of second images, and an advertising control unit 22 that extracts a specified second image from the plurality of stored second images in real time from the memory unit 21 and transmits the extracted second image. There may be multiple video production companies 1 and advertising production companies 2, or only one of them may be multiple.

[0023] The user area 3 is, for example, the home (residence) where the user, who is the viewer, usually lives. As shown in FIG. 1, the main components of the user area 3 are a display monitor 31 (image display means) that the user watches at home, a first camera 32, a second camera 33, a directional speaker 34, and a controller 35. The monitor 31 is a liquid crystal display panel, a plasma display panel, or the like capable of displaying images (hereinafter, referred to as program content) distributed by the program distribution company 4.

[0024] The first camera 32 is attached integrally to the display surface side of the monitor 31 so as to face the user, and mainly captures the user's face area including the eyes. The second camera 33 is disposed behind or to the side of the user independently of the first camera 32, and mainly captures the surroundings (living environment) of the user. These first and second cameras 32, 33 are image sensors such as CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). The directional speaker 34 is configured to be capable of beamforming using, for example, five omnidirectional ECMs (Electret Condenser Microphones) arranged in front of the upper surface of the monitor 31.

[0025] The controller 35 processes the image information captured by the first and second cameras 32 and 33, and creates an image of the user's line of sight and an image of the user's surroundings based on the captured image information. The controller 35 controls the operation of the monitor 31 and the directional speaker 34 to display program content distributed by the program distribution company 4 to the user. As shown in FIG. 1, the controller 35 includes a line-of-sight calculation unit 36, a display control unit 37, and the like.

[0026] The line of sight calculation unit 36 ​​creates a facial image of the user by image processing of the image information captured by the first camera 32, and also creates an image of the user's surroundings by image processing of the image information captured by the second camera 33. Furthermore, the line of sight calculation unit 36 ​​detects the direction of the user's face and the direction of the black eye (pupil) based on the user's facial image, calculates the line of sight direction based on these, and creates a line of sight image in which the display image (program content) of the monitor 31 currently being viewed by the user is synthesized with a visual target L, which is the destination (gazing point) of the user's line of sight (see Figs. 2 to 4).

[0027] The display control unit 37 controls the monitor 31 to display the program content. Furthermore, when displaying a highly conspicuous advertisement in advertisement posting area A described later, the display control unit 37 is configured to be able to localize the sound source by controlling the directional speaker 34 and emphasizing the signal in the direction of advertisement posting area A using a delay device to adjust the phase of the audio signal. Using this sound source separation function, the user's line of sight is guided to the highly conspicuous advertisement (advertising posting area A).

[0028] The program distribution company 4 creates program content by combining the first image and the second image, and sequentially distributes this program content to the user area 3 (controller 35). In addition, the program distribution company 4 requests the advertising production company 2 to provide a second image corresponding to the attributes of the user area 3 (e.g., nationality, address, family composition, user age, etc.) to be distributed at a specified timing, and receives the second image in response to the request from the advertising production company 2.

[0029] As shown in FIG. 1, the program distribution company 4 includes a semantic area setting unit 41 (semantic area detection means), an attractiveness determination unit 42 (attractiveness detection means), a concentration determination unit 43, an advertisement specification setting unit 44, an advertisement synthesis unit 45, a distribution control unit 46, and the like. The advertisement specification setting unit 44 and the advertisement synthesis unit 45 correspond to an advertisement display control means.

[0030] The meaningful area setting unit 41 sets an area that displays an important part of an image in a first image received from the video production company 1, that is, a so-called meaningful area P (P1, P2). The semantic area P is the area that the creators want the viewer to be interested in and focus their attention on. Therefore, the semantic domain P includes a first semantic domain P1 that is formed spontaneously due to the attributes of the program, and a second semantic domain P2 that is intentionally created in advance by the creator.

[0031] The first semantic area P1 is formed spontaneously on the spot by the behavior of the participants (performers). As shown in Figures 2(a) and 2(b), for example, in the case of a soccer broadcast program, the ball, the player who manipulates the ball, and the players who stay near the ball are set as the setting conditions of the semantic area, and the spatial area where people and objects that satisfy these conditions exist is set as the semantic area P1. In the case of the semantic area P1, the area where the viewer is desired to take an interest and look at is not known in advance, so the semantic area is determined over time. In addition, Fig. 2(a) is a gaze image of a scene where allies and enemies are jumbled together to fight for the ball, and Fig. 2(b) is a gaze image of a scene where players are waiting for a corner kick. The user's gaze is concentrated on the semantic area P1 in Fig. 2(a), but the user's gaze is not concentrated on the semantic area P1 in Fig. 2(b).

[0032] The second semantic domain P2 is created intentionally in advance by the creator. As shown in Figures 3(a) and 3(b), for example, in the case of a movie program, the screenwriter or director has set an area where the viewer is expected to take an interest and look at the screenplay, so a semantic area P2 consisting of the actor's facial expressions and actions exists in advance in the first image. Note that Figure 3(a) is a gaze image of a dining scene at a restaurant, and Figure 3(b) is a gaze image of a scene where a heartbroken actor hangs his head. The user's gaze is focused on the semantic area P2 in Figure 3(a), but the user's gaze is not focused on the semantic area P2 in Figure 3(b).

[0033] As shown in Fig. 4(a) and Fig. 4(b), for example, in the case of a news report program, a semantic area P2 consisting of a newscaster and a monitor screen exists due to the intention of the producer. Fig. 4(a) is a gaze image of a scene in which a favorite newscaster of the user explains news of interest, and Fig. 4(b) is a gaze image of a scene in which a newscaster who is not of interest to the user explains news of no interest to the user. The user's gaze is concentrated on the semantic area P2 in Fig. 4(a), but the user's gaze is not concentrated on the semantic area P2 in Fig. 4(b). Hereinafter, when there is no need to distinguish between semantic area P1 and semantic area P2, they will be collectively referred to simply as semantic area P.

[0034] The visual attraction determination unit 42 determines the visual attraction of each divided area in the first image received from the video production company 1 in association with the displayed image. The divided area is, for example, a structural area in the image, a spatial area, or a divided area related to subsequent processing such as a semantic area P and other areas. Visual attraction is also called visual salience or saliency, and is defined as the ease of attracting the user's attention, determined only by physical visual stimuli (brightness, color, edge direction, etc.). If factors such as the user's interests and intentions are not taken into consideration, then, from an ergonomic perspective, the user will naturally focus on areas with high visual attraction within the field of view.

[0035] The visual attraction determination unit 42 calculates the visual attraction of the first image and the user's peripheral image as a scalar quantity, which is not dependent on the features, and expresses each of them as a two-dimensional visual attraction map. The method of creating this visual attraction map is a known technique, and therefore a detailed description thereof will be omitted.

[0036] The concentration level determination unit 43 classifies the user's mental state into four states: a distracted state, an absent-minded state, a focused state, and an engrossed state, and determines the concentration level based on these states. The concentration level determination unit 43 determines the concentration level using the line-of-sight image received from the controller 35 and the semantic area P set by the semantic area setting unit 41 . When the visual target L (user's gaze point) is not present in the meaningful area P, the user is in a distracted state and is determined to have a low level of concentration. Specifically, the time ratio (e.g., 40% or more) that the visual target L stays in the meaningful area P during the judgment time is used as the criterion, and when the time that the visual target L stays in the meaningful area P is less than 40% of the judgment time, it is determined that the visual target L does not stay in the meaningful area P.

[0037] The concentration level determination unit 43 determines the concentration level of the user using the semantic area P set by the semantic area setting unit 41 and the degree of attraction of each area determined by the attention level determination unit 42 . Specifically, when the first attractiveness of the semantic area P where the visual target L is staying is equal to or less than the first threshold, the user is in an immersive state and is determined to have a high degree of concentration. When the user directs his or her gaze to the semantic area P that has a lower attractiveness than other areas, the user is in an immersive state concentrating on the program.

[0038] In order to classify the states other than the distracted state and the engrossed state more finely, the concentration level determination unit 43 determines whether the state is absent-minded or focused based on the degree of attraction (second degree of attraction) of the destination of the gaze. When the user's line of sight moves from the meaningful area P to a location other than the meaningful area P, the concentration level determination unit 43 determines the user's psychological state using the second degree of attraction of the destination. This is because visual attention and saccadic eye movement are closely related to the user's level of concentration.

[0039] Even if the visual target L (point of gaze) is in the semantic area P, it does not necessarily mean that the user is aware of it (is interested in it). Even if the visual target L is present, it may not be the object of attention. Therefore, when the user's gaze moves from the semantic area P, the state up until that point is determined using the degree of attention. Specifically, when the second attractiveness of the destination to which the user's gaze has moved is equal to or less than the second threshold (first threshold < second threshold), the state of focus (medium concentration) is determined. When the user deliberately focuses on an object with low attractiveness, it is assumed that the user has intentionally directed his or her gaze to the object of interest. When the second degree of attraction is greater than the second threshold value, a absentminded state (low degree of concentration) is determined. It is assumed that gazing at a highly attractive object indicates that the gaze is unconsciously directed toward the semantic area P before moving.

[0040] The advertisement specification setting unit 44 changes the degree of attraction of the second image to be combined with the first image in accordance with the user's degree of concentration. When the degree of concentration is medium, the advertisement specification setting unit 44 sets the degree of attraction of the second image to low, and when the degree of concentration is low, the advertisement specification setting unit 44 sets the degree of attraction of the second image to high. When the eye-catching degree of the second image is to be high, for example, the luminance or color brightness of the advertisement characters is increased, or the movement of the advertisement is enlarged. This avoids disruption of the user's viewing when the degree of concentration is medium, and encourages the user to purchase while attracting the user's gaze to the monitor 31 when the degree of concentration is low. Note that when the degree of concentration is high, the second image is not composited with the first image, and therefore the eye-catching degree of the second image is not set.

[0041] The advertisement combining unit 45 performs a predetermined process on the second image with the set degree of visibility, and combines it with the advertisement posting area A of the first image to generate a display image (program content). This advertisement synthesis unit 45 forms an overlay advertisement in which the second image is superimposed as a part of the first image in advertisement display area A that constitutes a partial area of ​​the first image. As shown in Figures 2(b), 3(b), and 4(b), the advertisement display area A constitutes a partial area of ​​the first image. Specifically, in Figure 2, the advertisement display area A is a plain (green) fence of a soccer field, in Figure 3, it is a decorative single-color board attached to the wall of a restaurant, and in Figure 4, it is set as a display board of a program title installed on the wall of a newsroom. The distribution control unit 46 distributes the program content generated by the advertisement synthesizing unit 45 to the controller 35 (user area 3).

[0042] Next, an example of an advertisement display control process will be described with reference to the flowchart shown in Fig. 5. In the figure, Si (i = 1, 2, ...) indicates each step. First, the necessary information is acquired as appropriate (S1), and then the process moves to S2. In S2, while the user is viewing a predetermined program content, a gaze image of the user and an image of the user's surroundings are sequentially created, and the process proceeds to S3.

[0043] In S3, a semantic area P of the image for display that the user will view is set, and the process moves to S4. In S4, the attention degree for each area of ​​the display image viewed by the user and the image surrounding the user is calculated, and the process proceeds to S5.

[0044] In S5, at a predetermined judgment time, it is judged whether or not the user's visual target L is outside the semantic region P. If the judgment result of S5 is that the user's visual target L is outside the semantic region P, the user is not interested in the program content, and the process proceeds to S6. If the judgment result of S5 is that the user's visual target L is in the semantic region P, the user is presumed to be interested in the program content, and the process proceeds to S11.

[0045] In S6, it is determined whether the user's visual target L was outside the semantic region P in the previous time as well. If the result of the determination in S6 is that the user's viewing target L was also outside the semantic region P the previous time, the user is determined to be uninterested in the program content and to have a low concentration level because the user continues to view outside the semantic region P (S7). If the result of the determination in S6 is that the previous viewing target L was within the semantic region P and has now fallen outside the semantic region P for the first time, the process proceeds to S10.

[0046] In S10, it is determined whether the previous visual target L is within the meaningful area P and whether the second visual attraction of the current visual line movement destination is greater than the second threshold value. When the result of the determination in S10 is that the second degree of attraction is greater than the second threshold, the gaze that had been absentmindedly gazing at the semantic area P is reflexively guided to the high degree of attraction, and the process proceeds to S7. If the result of the judgment in S10 is that the second degree of attraction is equal to or less than the second threshold, the gaze, which was focused on the semantic area P, is consciously directed toward an object of attention with a lower degree of attraction, and the process proceeds to S12.

[0047] As shown in Figure 6, the relationship between the attractiveness of the entire display image and the frequency of gaze of general users is an upwardly protruding characteristic D1, and the relationship between the attractiveness of the semantic area P and the frequency of gaze of general users is an upwardly protruding characteristic D2 in the high-attraction area. FIG. 7 shows a graph obtained by converting FIG. 6 into a relationship between the degree of eye-attraction and the probability of gaze of general users. As shown in Fig. 7, the characteristic D3 of the meaningful area P occupies approximately 70% or more of the area with high attractiveness, for example, the area of ​​the average value + standard deviation or more. This is presumably because, in addition to elements such as brightness, elements such as movement become large within the meaningful area P. Therefore, when the visual target L moves outside the meaningful area P, if the attractiveness of the destination is low, the user consciously directs his or her gaze toward the target of interest.

[0048] Returning to the explanation of the flowchart in FIG. After S7, the sound source is localized by the sound source separation function of the directional speaker 34 to guide the user's gaze to the advertisement posting area A (S8), and a highly eye-catching second image advertisement is displayed in the advertisement posting area A (S9), and the process returns. Here, if the user continues to look outside the meaningful area P (YES in both S5 and S6), the eye-catching degree of the second image displayed in the advertisement posting area A is set to be higher than the eye-catching degree of the image surrounding the user (living space including furniture, curtains, etc.) in order to attract the user's gaze to the monitor 31.

[0049] In S11, it is determined whether the first attractiveness of the semantic area P in which the user's visual target L exists is greater than a first threshold value. If the result of the determination in S11 is that the first attractiveness is greater than the first threshold value, the user is in a focused state where he or she is interested in the program content, and the level of concentration is determined to be medium (S12). After S12, an advertisement of a second image with low attractiveness is displayed in the advertisement display area A (S13), and the process returns. The advertisement is displayed so as not to interfere with the viewing of the user in the focused state.

[0050] If the result of the determination in S11 is that the first degree of attraction is equal to or less than the first threshold, the user is in a state of being engrossed in concentrating on the program content, and so the degree of concentration is determined to be high (S14). This is to prepare an environment in which the user can watch the program content with complete concentration. After S14, the display of advertisements in the advertisement display area is prohibited (S15), and the process returns.

[0051] Next, the operation and effect of the above advertisement providing system S will be described. According to the present advertisement providing system S, since it has the first camera 32 that detects the line of sight of the user who is viewing the image displayed on the monitor 31, it is possible to determine the visual target L (user's gaze point) that attracts the user's gaze. Since it has the semantic area setting unit 41 that detects the semantic area P that displays the important part of the image within the image, it is possible to detect the semantic area P that the creator wants the user to be interested in and direct their gaze to, and visually display it. Since it has an advertisement synthesis unit 45 that controls the second image (advertising information) superimposed on the advertisement posting area A, it is possible to display the second image without requiring a separate advertisement display time to interrupt the image. When the visual target L does not exist in the semantic area P, the advertisement synthesis unit 45 superimposes the second image on the advertisement posting area A, so that when the user is in a distracted state with a low level of concentration on the image, the second image can be displayed without disturbing the user's concentration.

[0052] The apparatus has an attractiveness determination unit 42 capable of detecting the attractiveness of the semantic area P, and the advertisement synthesis unit 45 changes the display form of the second image superimposed according to the attractiveness of the semantic area P when the visual target L is present in the semantic area P, thereby enabling the display form of the second image to be changed according to the user's concentration level.

[0053] When the degree of attraction of the semantic area P is lower than a predetermined threshold, the advertisement synthesis unit 45 does not superimpose the second image and prohibits the display of the second image. As a result, when the user is in a state of high concentration where he or she is engrossed in the image, the display of the second image, which is advertisement information, is prohibited, so that the user's concentration is not hindered.

[0054] The apparatus has an attractiveness determination unit 42 capable of detecting the degree of attraction for each divided area associated with the displayed image, and an advertising specification setting unit 44 changes the display form of the second image superimposed when the visual target L moves from a semantic area P to outside the semantic area P according to the degree of attraction of the moved area, so that the attractiveness of the second image can be set according to the user's concentration level.

[0055] When the degree of attraction of the destination is high, the advertisement specification setting unit 44 increases the degree of attraction of the second image to be superimposed and displayed, compared to when the degree of attraction of the destination is low. As a result, when the degree of attraction of the destination is high, the gaze is reflexively drawn to the degree of attraction and moves in a absent-minded state (low concentration), so the degree of attraction of the second image can be increased to increase the user's recognition of the advertisement.When the degree of attraction of the destination is low, the gaze is intentionally moved to the target of attention in a focused state (medium concentration), so the degree of attraction of the second image can be decreased to avoid disruption of the focused state.

[0056] This advertisement providing method, which provides advertisements using a program distribution company 4 that distributes an image including an advertisement posting area A on which advertising information can be superimposed, and a monitor 31 that displays the display image distributed from the program distribution company 4, includes a gaze detection step for detecting the gaze of a user viewing the display image, a meaningful area detection step for detecting a meaningful area P that displays an important part of the image within the display image, and an advertisement display step for superimposing advertising information on the advertisement posting area A when a visual target L is not present in the meaningful area P. According to this advertisement providing method, it is possible to achieve the same effects as those achieved by the advertisement providing system S described above.

[0057] Next, a modified example in which the above embodiment is partially changed will be described. 1) In the above embodiment, the image management means is the video production company 1, but the video production company 1 and the program distribution company 4 may be integrated into one image management means. It is also possible for the video production company 1, the advertising production company 2, and the program distribution company 4 to constitute an image management means.

[0058] 2) In the above embodiment, an example has been described in which the program distribution company 4 combines the first and second images it receives in advance, and distributes the combined program content to the controller 35 in the user area 3. However, the first and second images may be sent directly to the user area 3 from the video production company 1 and the advertising production company 2. In this case, the controller 35 is provided with a function for receiving the first and second images, as well as functions such as a semantic area setting unit 41, a visual attraction determination unit 42, a concentration determination unit 43, an advertising specification setting unit 44, and an advertising synthesis unit 45.

[0059] 3) In the above embodiment, an example has been described in which program content is distributed from the program distribution company 4 to the user area 3 via the network 5, but it may be configured so that the program content is distributed from the program distribution company 4 to the user area 3 via terrestrial waves (radio waves). In this case, the user's line of sight image and the user's surroundings image are separately transmitted from the user area 3 to the program distribution company 4 via wired or wireless communication.

[0060] 4) In the above embodiment, an example was described in which an overlay advertisement in which a second image was superimposed as part of a first image was displayed in advertising display area A, but the second image may also be superimposed using a superimposition method.

[0061] 5) In addition, a person skilled in the art can implement the present invention in a form in which various modifications are added to the above-mentioned embodiment without departing from the spirit of the present invention, and the present invention also includes such modifications. [Explanation of symbols]

[0062] 1. Video production company 4. Program distribution companies 31 Monitor 32 Camera No. 1 41 Significant area setting section 42 Attraction level determination section 44 Advertising specification setting section 45 Advertisement synthesis section S Advertising system P Significant domain L Visual target

Claims

1. 1. An advertisement providing system that provides advertisements using an image management means that delivers an image including an advertisement posting area on which advertisement information can be superimposed, and an image display means that displays the image delivered from the image management means, A gaze detection means for detecting a gaze of a viewer viewing the image; a significant area detection means for detecting significant areas within the image, the significant areas representing important parts of the image; an advertisement display control means for controlling advertisement information superimposed on the advertisement display area, The advertisement providing system is characterized in that the advertisement display control means superimposes and displays advertisement information on the advertisement display area when the viewer's line of sight is not in the semantic area.

2. an attractiveness detection means for detecting the attractiveness of the semantic area; The advertising provision system according to claim 1, characterized in that the advertising display control means changes the display format of the superimposed advertising information according to the attractiveness of the semantic area when the viewer's line of sight is in the semantic area.

3. 3. The advertisement providing system according to claim 2, wherein the advertisement display control means prohibits the display of the superimposed advertisement information when the degree of attraction of the semantic area is lower than a predetermined threshold value.

4. an attractiveness detection means for detecting an attractiveness of each divided area in association with the displayed image; The advertising provision system described in claim 1, characterized in that the advertising display control means changes the display format of the superimposed advertising information depending on the degree of visibility of the destination when the viewer's gaze moves from the semantic area to outside the semantic area.

5. The advertisement providing system according to claim 4, characterized in that the advertisement display control means increases the eye-catching degree of the superimposed advertising information when the eye-catching degree of the destination is high compared to when the eye-catching degree of the destination is low.

6. 1. An advertisement providing method for providing an advertisement using an image management means for delivering an image including an advertisement posting area on which advertisement information can be superimposed, and an image display means for displaying the image delivered from the image management means, comprising: a gaze detection step of detecting a gaze of a viewer viewing the image; a significant region detection step for detecting significant regions in the image, the significant regions representing important parts of the image; an advertisement display step of superimposing and displaying advertisement information on the advertisement display area when the viewer's line of sight is not in the semantic area; 13. A method for providing an advertisement, comprising:

Citation Information

Patent Citations

  • Information processing apparatus and method, and program

    JP2010154395A

  • Changing device behavior based on eye gaze

    JP2014516181A

  • System and apparatus for appearance mapping for overlay graphics synthesis

    JP2016515327A

  • Information processing apparatus and image display method

    JP2019039988A

  • Content delivery server, content delivery method and content delivery program

    JP2019047366A