Information processing method, program, and information processing device

Abstract

For example, it becomes possible to identify and control the delivery of more appropriate advertising materials based on information about a given program, including live broadcasts. [Solution] The information processing method involves an information processing device outputting predetermined live broadcast content to an object detection device that performs object detection processing, obtaining object detection results from the object detection device for the predetermined content, identifying television advertising material based on predetermined objects included in the object detection results, and controlling the broadcasting of the identified television advertising material in the advertising slot related to the predetermined content.

Description

Technical Field

[0001] The disclosed technology relates to an information processing method, a program, and an information processing apparatus.

Background Art

[0002] Conventionally, in the case where the conditions such as the broadcast time zone are determined by a contract between an advertising broadcaster and an advertisement provider, television advertisement materials broadcast from a television station or the like may be subject to such conditions (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Among the contents broadcast from a broadcasting station, there are live broadcast contents. In the case where an object such as a performer is broadcast in a predetermined live broadcast content (hereinafter also referred to as a live broadcast program), an advertiser or an advertisement provider (hereinafter, the advertiser or the advertisement provider is also collectively referred to as an advertisement provider) may want to broadcast a television advertisement material that matches the object broadcast immediately before.

[0005] However, for example, television advertisement materials related to a predetermined program including a live broadcast program have conditions such as broadcast time zones determined by a contract between an advertising broadcaster (for example, a broadcasting station) and an advertisement provider before the broadcast of the predetermined program. Therefore, there has been no idea of replacing or allocating television advertisement materials immediately before the broadcast.

[0006] Therefore, an object of the disclosed technology is to provide a technology that enables control to identify and transmit more appropriate advertisements based on information related to a predetermined program including, for example, a live broadcast program. [Means for solving the problem]

[0007] An information processing method in one aspect of the disclosure involves an information processing device outputting predetermined live broadcast content to an object detection device that performs object detection processing; obtaining object detection results from the object detection device of object detection processing performed on the predetermined content; identifying television advertising material based on predetermined objects included in the object detection results; and controlling the broadcasting of the identified television advertising material in the advertising slot related to the predetermined content. [Effects of the Invention]

[0008] According to the disclosure technology, it is possible to control the identification and delivery of more appropriate advertising materials based on information about a given program, including, for example, a live broadcast program. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an overview of the processing of an information processing system according to one embodiment. [Figure 2] This figure shows an example of the configuration of an information processing system according to one embodiment. [Figure 3] This figure shows an example of object detection results according to one embodiment. [Figure 4] This figure shows an example of object detection results according to one embodiment. [Figure 5] This figure shows an example of environmental data according to one embodiment. [Figure 6] This figure shows another overview of the processing of the information processing system according to one embodiment. [Figure 7] This figure shows an example of the configuration of an information processing device according to one embodiment. [Figure 8] This is a diagram illustrating an advertising space and an advertising playlist according to one embodiment. [Figure 9] This figure shows an example of real-time processing according to one embodiment. [Figure 10]This flowchart shows an example of the process related to sending advertisements according to one embodiment. [Modes for carrying out the invention]

[0010] A preferred embodiment of the disclosed technology will be described with reference to the attached drawings. In each drawing, components denoted by the same reference numerals have the same or similar components.

[0011] The following describes Information Processing System 1 in the Disclosure Technology. In the Disclosure Technology, "broadcast" is a concept that includes television broadcasting and radio broadcasting. "Program" includes television and radio programs broadcast during predetermined broadcast time slots. "Live broadcast" includes broadcasting a program in real time from a studio or live broadcast location at the broadcast time, without pre-recording or filming the program material. "Advertisement" refers to making information widely known to the general public, and includes advertisements broadcast via terrestrial broadcasting, etc., and in some cases, advertisements distributed via the internet. "Television advertising material" is a general term for television commercials and radio commercials, and television advertising material is a type of advertisement, and television advertising material is also called a commercial. "Transmission" includes broadcasting as television advertising material and distributing as internet advertising material.

[0012] <Overview of Disclosed Technology> The outline of the disclosed technology will be explained using Figures 1 and 2. Figure 1 is a diagram showing an overview of the processing of an information processing system 1 according to one embodiment of the disclosure. Figure 2 is a diagram showing an example of the configuration of an information processing system 1 according to one embodiment of the disclosure. The information processing system 1 of the disclosed technology may include, for example, an information processing device 10, an object detection device 20, and an external device 30. The object detection device 20 detects objects, etc., contained in a predetermined program (including recorded programs), including live broadcast programs that have video and audio. Hereinafter, the predetermined program will be explained using a live broadcast program as an example. For example, the object detection device 20 generates object detection results from the content broadcast in the live broadcast program, such as predetermined objects that were broadcast in the live broadcast program, and how many times (number of times or seconds) the predetermined object was reported. The predetermined object includes, for example, any object that can be detected from an image, such as a person such as a performer, a food item, or a landscape, and the detection results may include emotions, etc., that are evoked by the predetermined object.

[0013] The information processing device 10 identifies television advertising material based on predetermined objects included in the object detection results. The information processing device 10 identifies television advertising material that matches predetermined objects included in the object detection results for viewers watching a live broadcast program in real time. The information processing device 10 controls the transmission of the identified television advertising material in the advertising slots related to the live broadcast program. The information processing device 10 controls the transmission of the identified television advertising material based on, for example, the identification information ID of the advertising slot related to the live broadcast program (hereinafter also referred to as the advertising slot ID).

[0014] Figure 1 is a diagram illustrating the overview of the processing of the information processing system 1. In the example shown in Figure 1, the information processing device 10 outputs the live broadcast program A1 to the object detection device 20, and obtains object detection results related to the live broadcast program A1 from the object detection device 20.

[0015] In the example shown in FIG. 1, the information processing apparatus 10 identifies a TV commercial material based on the performer C1, which is a predetermined object included in the object detection result. For example, the information processing apparatus 10 identifies the TV commercial material B1 in which the performer C1 appears as the TV commercial material that matches the performer C1. The information processing apparatus 10 controls to send the identified TV commercial material B1 in the advertisement slot D1 related to the live broadcast program A1.

[0016] Through the above processing, the information processing apparatus 10 can identify a TV commercial material highly relevant to the live broadcast program based on the object detection result related to the live broadcast program for the viewers watching the live broadcast program or the advertisement providers conducting TV commercials. Thus, the information processing apparatus 10 can utilize the object detection result for contextual marketing of TV commercial materials. The information processing apparatus 10 can predict what the viewers watching the live broadcast program are likely to be interested in based on the object detection result. Therefore, the information processing apparatus 10 can identify and send a TV commercial material with a high advertising effect that matches the live broadcast program based on the object detection result of the live broadcast program immediately before the TV commercial material is broadcast.

[0017] Also, the information processing apparatus 10 may acquire environmental data related to the live broadcast program A from the external device 30. For example, the information processing apparatus 10 may acquire environmental data such as weather, temperature, stock price, event information, and viewing rate during the broadcast of the live broadcast program A1. The information processing apparatus 10 may identify a TV commercial material based on the object detection result and environmental data of the live broadcast program A1, and control to send the identified TV commercial material in the advertisement slot D1 related to the live broadcast program.

[0018] Through the above processing, the information processing apparatus 10 can identify a TV advertisement material with a higher relevance to the live broadcast program for the viewer watching the live broadcast program based on the object detection result and environmental data related to the live broadcast program. In addition to the object detection result, the information processing apparatus 10 can utilize the environmental data for contextual marketing of TV advertisement materials. The information processing apparatus 10 can identify and send out a TV advertisement material with a high advertising effect that matches the live broadcast program based on the object detection result and environmental data, thereby improving the value as an advertising medium.

[0019] <Information Processing System 1> In the example shown in FIG. 2, the information processing system 1 may include an information processing apparatus 10, an object detection apparatus 20, and an external apparatus 30. The object detection apparatus 20 and the external apparatus 30 may be configured to be provided as one service. Also, the object detection apparatus 20 and the external apparatus 30 may constitute one service. For example, this service may include a service provided on the cloud.

[0020] Each apparatus in the information processing system 1 transmits and receives data to and from each other via the network N. Each server may be constituted by a plurality of processing apparatuses (which may include a database). The network N is constituted by a wireless network or a wired network. As an example of the network, there are a mobile phone network, a PHS (Personal Handy-phone System) network, a wireless LAN (Local Area Network, including communication compliant with IEEE802.11 (so-called Wi-Fi (registered trademark))), 3G (3rd Generation), LTE (Long Term Evolution), 4G (4th Generation), 5G (5th Generation), WiMax (registered trademark), infrared communication, visible light communication, Bluetooth (registered trademark), a wired LAN, a telephone line, a power line communication network, a network compliant with IEEE1394, etc.

[0021] <Information Processing Apparatus 10> The information processing device 10 is composed of a server or a personal computer, and sends requests to each server via the network N and receives responses from each server. For example, the information processing device 10 is a device within a broadcasting station and, in response to user operations, transmits live broadcast programs to the object detection device 20 and receives object detection results of live broadcast programs from the object detection device 20.

[0022] The information processing device 10 controls the transmission of the identified television advertising material within the advertising slots of the live broadcast program. The information processing device 10 may also transmit the identification information of the identified television advertising material (hereinafter also referred to as the television advertising material ID) and the position of the advertising slot in the live broadcast program to the television advertising material transmission server. If the information processing device 10 has a television advertising material transmission function, it may transmit the identified television advertising material at the position of the advertising slot in the live broadcast program.

[0023] <Object detection device 20> The object detection device 20 detects information about broadcast objects in a live broadcast program output from the information processing device 10, based on predetermined perspectives, and generates object detection results. The predetermined perspectives may include, for example, "when," "what kind of program," "where," "what," "who," "what kind of object," "for how many seconds," and "how it was featured." Furthermore, the objects may include, for example, the program name, program content, keywords, performers, objects, scenery, animals, companies, services, products, stores, facilities, music used, spots, service brands, news, events, text information (captions or subtitles), etc.

[0024] Furthermore, the object detection device 20 may generate object detection results by associating the detected time spent by an object within a live broadcast program with the detected object. The object detection device 20 may also generate object detection results by associating the number of seconds or number of times an object was exposed with an object. In addition, the object detection device 20 may include the emotion evoked by a given object and the duration of that emotion in the object detection results, associating them with the detected object. Note that object detection results performed on a live broadcast program include television advertising materials broadcast during the live broadcast program. For example, the object detection results may include the performers appearing in the television advertising materials broadcast during the live broadcast program, the music used, the company name of the television advertising material, the product name, the content of the television advertising material, the duration of the broadcast, etc.

[0025] The object detection device 20 may, for example, use AI (Artificial Intelligence) to detect the presence of broadcast objects in a live broadcast program output from the information processing device 10 based on predetermined viewpoints, and generate object detection results. The object detection device 20 may, for example, use AI equipped with a high-precision recognition engine to detect performers appearing in the live broadcast program, their facial expressions, etc., and generate object detection results. Alternatively, a professional operator may detect the presence of broadcast objects in a live broadcast program output from the information processing device 10 based on predetermined viewpoints, and generate object detection results. The generated object detection results may be stored in a predetermined memory unit. Furthermore, the object detection device 20 can generate object detection results in real time (for example, within 1 second) for a live broadcast program output from the information processing device 10. Known techniques may be used for the object detection method.

[0026] The information processing device 10 can easily grasp the content of a live broadcast program based on the object detection results generated by the object detection device 20. Furthermore, unlike program listings found in newspapers, etc., which are displayed as previews, the object detection results are generated based on the actual broadcast history of the live program, resulting in highly accurate and granular data.

[0027] The information processing device 10 acquires the generated object detection results. The information processing device 10 may also acquire the object detection results from the object detection device 20 via an API (Application Programming Interface).

[0028] Figure 3 shows an example of object detection results according to one embodiment of the disclosure. In the example shown in Figure 3, the information processing device 10 stores the object detection results 500a obtained from the object detection device 20 in the storage device 130 in association with the identification information of the live broadcast program (hereinafter also referred to as the live broadcast program ID). In Figure 3, the object detection results 500a are associated with the live broadcast program "A100" and stored as object detection result 1 "Performer A", object detection result "Performer B", and object detection result "Performer C". In addition, the object detection results "Chopsticks", object detection result "Fried Chicken", and object detection result "Salad" are stored in association with the live broadcast program "A110".

[0029] Figure 4 shows an example of object detection results according to one embodiment of the disclosure. In the example shown in Figure 4, the information processing device 10 stores, for example, the object detection results 500b, which are organized by category and obtained from the object detection device 20, in the storage device 130 in association with the live broadcast program ID. The object detection results 500b shown in Figure 4 are object detection results related to the live broadcast program "A100". Associated with the live broadcast program "A100", for example, the category "Program" stores object detection result 1 "Drama", object detection result 2 "Mystery", and object detection result 3 "Performer A". In addition, each category may include, for example, travel, home, gardening (house, landscaping), beauty, health, banking, finance, etc.

[0030] <External device 30> The external device 30 generates predetermined environmental data. This predetermined environmental data may include, for example, weather, temperature, stock prices, events, and viewership ratings. The information processing device 10 may acquire predetermined environmental data related to live broadcast programs from the external device 30. The external device 30 may consist of multiple devices, each of which may generate data related to weather, temperature, stock prices, events currently taking place, or viewership ratings. The information processing device 10 may identify matching television advertising materials to be broadcast based on environmental data related to the weather while the live broadcast program is being aired, as environmental data related to the live broadcast program. The information processing device 10 may acquire environmental data via an API. The information processing device 10 may also acquire SNS and search data as environmental data. By acquiring SNS and search data, it is possible to acquire environmental data related to consumer interests and levels of engagement, thereby identifying more appropriate advertisements.

[0031] Figure 5 shows an example of environmental data according to one embodiment of the disclosure. In the example shown in Figure 5, the information processing device 10 stores environmental data 600a acquired from an external device 30, for example, in the storage device 130 in association with the live broadcast program ID. The environmental data 600a shown in Figure 5 is environmental data acquired during the live broadcast program "A100". Associated with the live broadcast program "A100", for example, the environmental data "weather" stores environmental data 1 "Tokyo", environmental data 2 "Sunny", and environmental data 3 "Probability of precipitation x%".

[0032] <Auction device> This section describes how the information processing device 10 determines television advertising materials in real time by coordinating with an auction device that holds auctions for programmatic advertising. For example, an advertiser who wants to broadcast an advertisement for beer may want to broadcast the television advertising material on a sunny, warm day, as this is more effective than on a rainy day. On the other hand, an advertiser who wants to broadcast an advertisement for umbrellas may want to broadcast the television advertising material on a rainy day, as this is more effective. Furthermore, during a live broadcast program, there may be moments that capture the attention of viewers (hereinafter also referred to as "moments"), such as when viewership temporarily increases. In such cases, the advertiser may want to broadcast specific television advertising materials to capitalize on the high viewership.

[0033] The information processing device 10 may utilize real-time bidding (hereinafter also referred to as RTB), operated by an auction device, if there are available advertising slots that can be selected before the television advertising material is broadcast. RTB is an auction transaction conducted by bidding before the television advertising material is broadcast.

[0034] The information processing device 10 may, for example, associate advertising space information regarding vacant advertising slots with object detection results corresponding to the advertising slots and output this information to the auction device. The information processing device 10 may also obtain identification information of the programmatic advertisement that won the vacant position from the auction device and control the device to send television advertising material for the programmatic advertisement corresponding to the obtained identification information to the vacant position.

[0035] The information processing device 10 may send an auction request to a distribution platform (hereinafter also referred to as a DSP (Demand-Side Platform)) operated by the auction device for advertisers who submit television advertising materials. If each receiving DSP determines that the television advertising material slot matches a television advertising material that is a candidate for bidding, it may set a high value for the television advertising material slot, make a bid, and win the auction. The information processing device 10 may also incorporate the auction results, such as identification information of the successful programmatic advertisement, into the advertising slot and control the transmission of the television advertising material.

[0036] Furthermore, the information processing device 10 may be connected to an SSP (Supply Side Platform), and the SSP may implement RTB (Real-Time Bidding) with a DSP (Demand Side Platform). The information processing device 10 may also utilize other auction services.

[0037] The auction device may disclose advertising information regarding available advertising slots, including advertising slot identification information (hereinafter also referred to as advertising slot ID), broadcast date, broadcast time, number of available positions, available time, program name, program genre, broadcast area, object detection results, exclusion conditions, etc. On the other hand, bidders may register bidding information such as bid advertising identification information, advertising period, advertising genre, TV advertising material ID, material length, unit price per view, and budget limit. The auction device may, for example, award the highest bid that matches the exclusion conditions and available positions (length / time) applied to the advertising slot.

[0038] Figure 6 is a diagram showing another overview of the processing of the information processing system 1 according to one embodiment of the disclosure. In the example shown in Figure 6, the information processing device 10 outputs the live broadcast program A2 to the object detection device 20, and obtains object detection results related to the live broadcast program A2 from the object detection device 20.

[0039] The information processing device 10 outputs to the auction device, along with the auction request, advertising slot information for the vacant advertising slot D2 and object C2, which is a predetermined object included in the object detection results corresponding to advertising slot D2. The information processing device 10 may also output to the auction device, for example, object detection results acquired up to a predetermined time before the television advertising material is broadcast, in association with the advertising slot information for the vacant advertising slot D2. The predetermined time is, for example, n seconds before (a very short time, for example, 3 seconds).

[0040] The auction system receives, for example, bids for available positions and information on television advertising material B2 from advertiser E2, a beverage manufacturer, which it considers to be a good match for the object detection results. Here, we assume that the available position was successfully bid on by advertiser E2, the beverage manufacturer.

[0041] The information processing device 10 obtains identification information for television advertising material B2, which is a programmatic advertisement that won a bid for an available position, from the auction device.

[0042] The information processing device 10 controls the delivery of television advertising material for programmatic advertising corresponding to the acquired identification information to an available position.

[0043] Through the above processing, the information processing device 10 becomes capable of controlling the transmission of television advertising materials selected by the advertiser in advertising slots that the advertiser recognizes as having high advertising effectiveness, thereby improving the value of television as an advertising medium for broadcasting television advertising materials.

[0044] Furthermore, the information processing device 10 may acquire environmental data related to the live broadcast program A2 from the external device 30. For example, the information processing device 10 may acquire environmental data such as weather, temperature, stock prices, event information, and viewership ratings from the external device 30 while the live broadcast program A2 is being broadcast. The information processing device 10 may output the auction request by associating advertising slot information related to the vacant advertising slot D2, object C2 which is a predetermined object included in the object detection result corresponding to advertising slot D2, and / or environmental data. This allows the auction device to further provide environmental data to advertisers and accept bids for vacant positions from advertisers.

[0045] <Configuration of the information processing device 10> The following describes the configuration of each device of the information processing system 1, specifically the information processing device 10, which performs the process of identifying television advertising materials that are highly relevant to a live broadcast program, according to the objects and other items actually broadcast in the live broadcast program. Figure 7 is a diagram showing an example of the configuration of the information processing device 10 according to one embodiment of the disclosure. The information processing device 10 includes one or more processors (CPUs) 110, one or more communication interfaces 120, a storage device 130, a user interface 150, and one or more communication buses 170 for interconnecting these components.

[0046] The user interface 150 is connected to a display and an input device (such as a keyboard and / or mouse, or any other pointing device), and has display and input functions.

[0047] The storage device 130 may be, for example, a high-speed random access memory such as DRAM, SRAM, or other random access solid-state memory, or it may be a non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state memory devices, or it may be a non-temporary recording medium that can be read by a computer.

[0048] The storage device 130 stores data used by the information processing system 1. For example, the storage device 130 stores live broadcast programs, object detection results of acquired live broadcast programs, information about television advertising materials, information about advertising slots, metadata of television advertising materials, and so on.

[0049] For example, when a broadcasting station stores television advertising material received from an advertiser in the storage device 130, it may store the television advertising material ID, summary information of the television advertising material, the length of the television advertising material, and the video data of the television advertising material in association with each other in the storage device 130. The television advertising material may be television advertising material that has completed review for content that violates public order and morals, etc. The summary information of the advertisement may be the content of the television advertising material, etc. The length may be the duration of the television advertising material.

[0050] Another example of the storage device 130 may be one or more storage devices located remotely from the processor 110. In one embodiment, the memory 130 stores programs, modules, and data structures, or subsets thereof, executed by the processor 110.

[0051] The processor 110 executes a program stored in the storage device 130, thereby configuring a control unit 111 that has the functions of the information processing device 10 described above. The control unit 111 comprises an output unit 112, a first acquisition unit 113, a second acquisition unit 114, a third acquisition unit 115, an association unit 116, a specification unit 117, and a transmission unit 118.

[0052] The output unit 112 outputs a live broadcast program, which is a predetermined content to be broadcast live, to the object detection device 20, which performs object detection processing. For example, the output unit 112 may output the live broadcast program to the object detection device 20 in real time via an API provided by the object detection device 20.

[0053] The first acquisition unit 113 acquires object detection results, which are the results of object detection processing performed on the live broadcast program, from the object detection device 20 that outputs the live broadcast program. The first acquisition unit 113 may also acquire object detection results generated for the live broadcast program up until immediately before the television advertising material is broadcast. The output unit 112 may output the live broadcast program to the object detection device 20 while the first acquisition unit 113 acquires object detection results for the live broadcast program.

[0054] The first acquisition unit 113 may associate the acquired object detection results with live broadcast programs. The first acquisition unit 113 may associate the acquired object detection results with advertising slot IDs related to live broadcast programs. The first acquisition unit 113 may classify the object detection results into categories based on classification information such as definitions and conditions for associating the object detection results with each category, keywords to exclude, and output format, and associate the object detection results with live broadcast programs along with the classified categories. It is preferable to prepare the category classification information in advance.

[0055] The metadata generation unit 116 may generate metadata for each live broadcast program and / or each television advertising material. For example, the metadata generation unit 116 generates metadata based on the category of each live broadcast program, the scheduled performers, and / or object detection results stored in the storage device 130, and associates the generated metadata with the content ID. The metadata generation unit 116 may also analyze each television advertising material using AI, etc., generate metadata related to the television advertising material, and associate it with the television advertising material ID. Alternatively, the metadata generation unit 116 may output each television advertising material to an external device, where the external device may analyze it using AI, etc., generate metadata related to the television advertising material, and the metadata generation unit 116 may acquire it. The information processing device 10 may, for example, generate metadata for each live broadcast program and each television advertising material in advance as a preprocessing step, and store it associated with the content ID and television advertising material ID.

[0056] The identification unit 117 identifies television advertising materials based on predetermined objects included in the object detection results. For matching determination, the identification unit 117 may, for example, identify at least one matching television advertising material based on the similarity between predetermined objects or keywords (examples of metadata) included in the object detection results acquired by the first acquisition unit 113 and the metadata associated with each television advertising material ID. The identification unit 117 may also prepare a matching table in advance for matching determination between the metadata associated with each television advertising material ID and the object detection results, and identify at least one matching television advertising material based on the matching table.

[0057] The identification unit 117 may identify the television advertising material with the highest average similarity between the metadata associated with each television advertising material ID and the acquired object detection results.

[0058] The identification unit 117 may, for example, identify television advertising material that matches the object detection results associated with the live broadcast program as television advertising material to be allocated to the vacant advertising slot associated with the live broadcast program if there is an empty position in the advertising slot associated with the live broadcast program. The identification unit 117 may also identify the advertisement to be allocated to the vacant position based on the object detection results obtained immediately before the broadcast time of the vacant position in the advertising slot of the live broadcast program.

[0059] Furthermore, the identification unit 117 may match the object detection results with television advertising materials based on predetermined conditions such as genre and constraints. In addition, the identification unit 117 may identify television advertising materials to be allocated to vacant positions by receiving requests from advertisers.

[0060] The transmission unit 118 controls the transmission of specified television advertising material within the advertising slots related to live broadcast programs. For example, the transmission unit 118 may output the position of the advertising slot and the television advertising material ID specified by the identification unit 117 to an external advertising transmission device. Advertising slots related to live broadcast programs include advertising slots inserted in the middle of a live broadcast program and advertising slots after the end of a live broadcast program.

[0061] Furthermore, if the transmission unit 118 has an advertising transmission function, it may control the transmission of identified television advertising material based on the position of the advertising slot. The transmission unit 118 may also control the transmission unit 118 to change the television advertising material to the television advertising material identified by the identification unit 117 immediately before the television advertising material is broadcast, and then transmit the changed television advertising material. The transmission unit 118 may also provide a buffer of a predetermined time before the transmission of the advertisement so that processing to associate the object detection results by the object detection device 20 with the live broadcast program, matching processing, advertisement identification processing, etc., can be completed in time before the advertisement is transmitted.

[0062] Through the above processing, it is possible to identify television advertising materials that match the object detection results generated for live broadcast programs. Furthermore, it is possible to broadcast television advertising materials with high advertising effectiveness that match the moments of viewers watching live broadcast programs, thereby improving the value of the advertising medium.

[0063] The identification unit 117 includes identifying television advertising material using performers, objects, scenery, and at least one of the emotions evoked by a given object, based on the object detection results. The identification unit 117 may, for example, identify at least one matching television advertising material based on the similarity between each item (performer, etc.) included in the object detection results acquired by the first acquisition unit 113 and the metadata associated with each television advertising material.

[0064] Through the above process, it is possible to identify television advertising materials that can be matched with, for example, performers who appeared on a live broadcast program.

[0065] The second data acquisition unit 114 may acquire environmental data from the external device 30, including at least one of weather, temperature, and stock price related to the live broadcast program. The second data acquisition unit 114 may acquire environmental data by cooperating with the external device 30, which provides a service for distributing environmental data. For example, the second data acquisition unit 114 may acquire environmental data from the external device 30 via an API. The second data acquisition unit 114 may acquire environmental data based on the current date and time. The second data acquisition unit 114 may also acquire environmental data from the external device 30 at predetermined intervals.

[0066] The second acquisition unit 114 may acquire environmental data generated for live broadcast programs up to the moment the television advertising material is broadcast. The output unit 112 may output the live broadcast program to the object detection device 20 while the second acquisition unit 114 acquires environmental data that can be acquired at that time.

[0067] The second acquisition unit 114 may associate the acquired environmental data with live broadcast programs. The second acquisition unit 114 may associate the acquired environmental data with advertising slot IDs related to live broadcast programs. The second acquisition unit 114 may classify the environmental data into categories based on classification information such as definitions and conditions for associating environmental data by category, keywords to be excluded, and output format, and associate the classified environmental data with live broadcast programs. It is preferable that the classification information for categories be prepared in advance.

[0068] The identification unit 117 may identify television advertising material based on the object detection results acquired by the first acquisition unit 113 and the environmental data acquired by the second acquisition unit 114.

[0069] Through the above processing, it becomes possible to identify television advertising materials that are more relevant to live broadcasts based on object detection results and environmental data. This allows for the delivery of television advertising materials that are more effectively matched to the moments of the live broadcast to viewers.

[0070] Furthermore, the identification unit 117 may identify television advertising materials using the dwell time of objects within a live broadcast program based on the object detection results. The identification unit 117 may, for example, use the dwell time of performers, text information (such as captions), objects, etc., included in the object detection results acquired by the first acquisition unit 113 to identify at least one matching television advertising material based on its similarity to the metadata associated with each television advertising material. For example, the identification unit 117 may identify television advertising materials using object detection results where the dwell time is equal to or greater than a threshold.

[0071] Through the above processing, it becomes possible to identify television advertising material to be broadcast that matches, for example, objects that are on screen for a long time, based on the duration of time objects are present in the live broadcast program.

[0072] Furthermore, the identification unit 117 may identify television advertising materials using the duration of emotions associated with a predetermined object based on the object detection results. The identification unit 117 may, for example, use the duration of emotions associated with performers included in the object detection results acquired by the first acquisition unit 113 to identify matching television advertising materials based on their similarity to metadata associated with each television advertising material. Regarding emotions, the identification unit 117 may estimate emotions based on the performers' facial expressions, voices, captions (text information), etc.

[0073] Through the above processing, it becomes possible to identify television advertising material to be broadcast that matches the emotions evoked by a given object, depending on its duration.

[0074] The output unit 112 may divide the live broadcast program into predetermined units and output each of the divided predetermined units to the object detection device 20. The output unit 112 may divide the content into predetermined units if the content capacity or playback time is above a threshold. The output unit 112 may output each of the divided predetermined units to the object detection device 20.

[0075] The identification unit 117 may assign television advertising material, which is matched based on predetermined objects detected in each predetermined unit of a live broadcast program, to each position in the advertising slot based on the broadcast order of the predetermined unit. For example, the identification unit 117 may, based on the object detection results obtained for predetermined units A, B, and C of a live broadcast program, assign television advertising material A that matches the object detection result A corresponding to predetermined unit A to the first position A in the advertising slot, and then perform the same process sequentially.

[0076] Through the above process, it becomes possible to identify television advertising materials to be broadcast that match the predetermined objects detected in each predetermined unit. Furthermore, by assigning them to each position in the advertising slot based on the broadcast order of the predetermined unit, it becomes possible to broadcast television advertising materials with high advertising effectiveness that match the predetermined unit in the broadcast order.

[0077] The output unit 112 may output to an auction device that holds an auction for programmatic advertising, associating advertising slot information for available advertising slots with object detection results corresponding to those advertising slots. If there are available advertising slots in the advertising slots associated with a live broadcast program, the output unit 112 may output advertising slot information for available advertising slots with object detection results corresponding to those advertising slots. Furthermore, the output unit 112 may output to the auction device, associating advertising slot information for available advertising slots with object detection results corresponding to those advertising slots, n seconds before the television advertising material is broadcast (a very short time, for example, 3 seconds).

[0078] The third acquisition unit 115 acquires identification information of the programmatic advertisement that won the vacant position from the auction device. The third acquisition unit 115 may acquire the identification information of the programmatic advertisement that won the vacant position up to n seconds before the television advertisement material is broadcast (a very short time, for example, 3 seconds).

[0079] The transmission unit 118 controls the transmission to send television advertising material for a programmatic advertisement corresponding to the acquired identification information to an available position. For example, the transmission unit 118 may output the position of the advertising slot and the television advertising material ID of the television advertising material identified by the identification unit 117 to an external advertising transmission device. If the third acquisition unit 115 is unable to acquire the identification information of the programmatic advertisement that has won the bid for the available position, the transmission unit 118 may control the transmission to send predetermined advertising material (advertising material of high public interest).

[0080] Through the above processing, the information processing device 10 outputs to the auction device advertising space information relating to the available advertising space and the object detection results corresponding to the advertising space. This allows advertisers to bid on the available space after judging efficiency and effectiveness based on the object detection results that reflect the content up to immediately before the live broadcast program.

[0081] <Advertisement space> Figure 8 is a diagram illustrating an advertising slot and an advertising playlist according to one embodiment of the disclosure. The advertising slot shown in Figure 8 is a predetermined television advertising material broadcast time slot for broadcasting television advertising material for goods or services, etc. Each advertising slot has at least a broadcast date and time and a broadcasting station determined. In the example shown in Figure 8, the advertising slot is an advertising slot for television advertising material. However, it is not limited to this, and the advertisement broadcast using the advertising slot may also be radio advertising material.

[0082] Each advertising slot is associated with advertising slot information. This advertising slot information is general information about each advertising slot and is provided by the advertising slot provider, such as a broadcasting station. The advertising slot information may also include the date and time the advertisement will be broadcast in the advertising slot (advertising broadcast time slot) and the broadcasting station. In addition, it may include the type of advertising slot, the length of the advertising slot in seconds, the name of the program associated with the advertising slot (specifically, the name of the program in which the advertising slot is scheduled within or immediately before the broadcast time), and the price of the advertising slot (purchase price). Here, the type (genre) of the advertising slot is information that shows the relationship between the advertising slot and the broadcast time of the program, and specifically includes "PT (Participating Commercial: time set within the program broadcast time)" and "SB (Station Break: time between one program and the next)."

[0083] Furthermore, the advertising slot information may store information for each individual advertising slot. The advertising slot information may also include other information not mentioned above, such as acquired object detection results, acquired environmental data, acquired programmatic ad identification information, number of positions, broadcast area (area information), broadcast period, broadcast time (time of day when the ad is broadcast: Daypart), information about the program genre associated with the advertising slot and the program's cast, information about events related to the program, program viewership ratings, number of viewers, main audience demographics, exclusion criteria for other ads, etc. However, these are merely examples and the information is not limited to these. The advertising slot information may also be updated in real time.

[0084] A single advertising slot can broadcast multiple advertisements. The order in which advertisements are broadcast within a single advertising slot is called the ad position. The ad position is also called "PIB (Position In Break)," and for example, if four 15-second advertisements can be broadcast within a single advertising slot (number of positions = 4), it indicates which position the advertisement will be broadcast in.

[0085] Furthermore, each ad slot is assigned an ad slot identifier (ad slot ID) that uniquely identifies that ad slot. This ad slot ID allows a single ad slot to be identified from among many others.

[0086] <Advertisement Playlist> Furthermore, Figure 8 will be used to explain the advertising playlist. The advertising playlist is a list associated with an advertising slot ID, which includes a TV advertising material ID that identifies the advertising material to be broadcast in that advertising slot, the position of that TV advertisement, and the duration information (length information) of that TV advertising material.

[0087] Figure 8 shows an example of a television advertising playlist with 4 positions, where the duration information and television advertising material ID are listed in order of position. For example, guaranteed advertisements may be assigned to positions 1 and 2 by the broadcasting system. A programmatic advertisement that was won in the previous auction may be assigned to position 3. Television advertising material identified according to the object detection results of the live broadcast program described above may be assigned to position 4.

[0088] <Real-time processing example> Next, an example of real-time processing will be described. Figure 9 is a diagram showing an example of real-time processing according to one embodiment of the disclosure. As shown in Figure 9, the live broadcast program A is divided into predetermined units such as 1A, 2A, etc. The information processing device 10 may, for example, input the divided predetermined units of the live broadcast program A to the object detection device 20 as a preprocessing step.

[0089] The information processing device 10 may identify television advertising material to be assigned to the advertising slot at position 1 related to live broadcast program A, based on the object detection results of live broadcast program 1A that was broadcast a predetermined time prior. The information processing device 10 may also identify television advertising material to be assigned to the advertising slot at position 2 related to live broadcast program A, based on the object detection results of live broadcast program 2A that was broadcast a predetermined time prior. For example, the information processing device 10 may identify television advertising material that matches the advertising slot based on the object detection results for each divided live broadcast program. This enables the delivery of highly relevant television advertising material at an appropriate timing based on the divided live broadcast program A. The predetermined time may be set to a time that takes into account the processing time for object detection by the object detection device and the advertising identification process. This enables the delivery of highly relevant advertisements based on object detection results related to live broadcast programs at an appropriate timing, neither too recent nor too far in the past.

[0090] <Operation Description> Next, we will describe each operation of the information processing system 1. Figure 10 is a flowchart showing an example of the process related to advertisement delivery according to one embodiment of the disclosure. In the example shown in Figure 10, each process is executed for one live broadcast program.

[0091] In step S101, the output unit 112 of the information processing device 10 outputs the live broadcast program to the object detection device 20, which performs object detection processing. For example, the output unit 112 outputs the live broadcast program via an API provided by the object detection device 20.

[0092] In step S102, the first acquisition unit 113 of the information processing device 10 acquires the object detection results of the object detection process performed on the live broadcast program from the object detection device 20.

[0093] In step S103, the identification unit 117 of the information processing device 10 identifies television advertising material based on predetermined objects included in the object detection results. For example, if there is an empty position in the advertising slot associated with a live broadcast program, the identification unit 117 may identify television advertising material that matches a predetermined object included in the object detection results of the live broadcast program as an advertisement to be assigned to this empty position.

[0094] In step S104, the transmission unit 118 of the information processing device 10 controls the transmission of the specified television advertising material in the advertising slots related to the live broadcast program. The transmission unit 118 may also output the position of the advertising slot and the specified television advertising material ID to an external advertising transmission server. If the transmission unit 118 has an advertising transmission function, it may also control the transmission of the specified television advertising material according to the position of the advertising slot.

[0095] Through the above processing, the information processing device 10 can identify television advertising materials based on object detection results that reflect the content and context of the live broadcast program, and can identify television advertising materials that match the moment in which the television advertising material is broadcast. As a result, advertisers can be provided with flexible allocation of television advertising materials, and viewers can watch more appropriate television advertising materials, thereby increasing viewer interest.

[0096] <Evidence Display> The information processing device 10 may provide the object detection results and / or environmental data as evidence to the successful bidder who has won the vacant position through an auction request that includes the object detection results and / or environmental data.

[0097] For example, when the control unit 111 receives a request to display evidence of object detection results or environmental data via the information processing terminal used by the successful bidder, it starts the evidence presentation process. Based on the information associated with the evidence display request, the control unit 111 presents the data from which the object detection results or environmental data were obtained as evidence to the successful bidder. For example, if it is an object detection result, the control unit 111 extracts a portion of a predetermined program corresponding to a predetermined time (the time corresponding to the successful bidder's vacant position) included in the evidence display request, and sends the extracted portion of the program to the processing terminal used by the successful bidder. If the predetermined program is a live broadcast, the programs from each broadcasting station are simultaneously recorded on a server in the cloud, and when the time of the live broadcast program used for object detection is specified, a link to view the live broadcast program for a predetermined amount of time including that time is sent to the processing terminal used by the successful bidder. When the successful bidder operates the link using the processing terminal, the portion of the live broadcast program may be streamed on the processing terminal's screen.

[0098] Furthermore, if an available position has been successfully bid on using environmental data, the control unit 111 retrieves the environmental data corresponding to a predetermined time included in the evidence display request from the source of that environmental data, and transmits the retrieved environmental data to the processing terminal used by the successful bidder.

[0099] Furthermore, the control unit 111 may, as a method of presenting evidence, display the aforementioned evidence data on the auction management screen when it receives a request from the auction management screen for the display of evidence for an available position won by the successful bidder.

[0100] The embodiments are illustrative examples for explaining the disclosed technology and are not intended to limit the disclosed technology to these embodiments only. The disclosed technology can be modified in various ways without departing from its essence. For example, the processes of each device may be integrated as appropriate, or the processes may be transferred to the other device.

[0101] For example, the information processing device 10 may further include a calculation unit. The calculation unit may calculate a score based on factors that contribute to viewing of a live broadcast program acquired by the second acquisition unit 114, including stock prices, news, weather, time of day, and season. For example, the calculation unit may use the acquired stock prices, news, weather, time of day, and season to calculate a score using a predetermined calculation method.

[0102] The calculation unit may change the data used to calculate the score depending on the category of the live broadcast program. For example, if the category of the live broadcast program is classified as news, the calculation unit may score the level of attention given to the news based on the object detection results related to the live broadcast program. Alternatively, the calculation unit may score the degree of matching between the object detection results of the live broadcast program and the acquired weather, time of day, or season, or, if the object detection results of the live broadcast program include economic terms, it may calculate a score based on the acquired up / down of stock prices.

[0103] The calculation unit may calculate a score based on viewing history data, including viewership ratings or unspecified viewing data, and / or viewing trend data, including live streaming performance or data related to live broadcast programs on social media, for acquired live broadcast programs. For example, the calculation unit may calculate a score using a calculation method that assigns a higher score to those with higher viewing history, based on viewership ratings obtained from companies that survey television viewership ratings, or viewing history data including unspecified viewing data that can be identified by IP addresses, etc.

[0104] For example, the second acquisition unit 114 may acquire distribution records when a live broadcast program is being distributed or has been distributed, or acquire viewing trend data, including the title of the live broadcast program and comments (tweets, etc.) on the live broadcast program, from web services such as SNS. The calculation unit 119 may calculate a score based on the acquired viewing trend data using a calculation method that results in a higher score when there is a large distribution record or when the viewing trend data indicates excitement (e.g., a large number of comments).

[0105] The calculation unit may calculate a score relating to the impression of a live broadcast program, which includes at least the dwell time of objects in the live broadcast program, the size of the on-screen text, or the type of keyword source, based on the object detection results acquired by the first acquisition unit 113. For example, the calculation unit may use the object detection results for the live broadcast program, such as the dwell time of a specific object detected, to calculate a score that takes impression into account using a predetermined calculation method relating to impression.

[0106] For example, the calculation unit may calculate the score using a calculation method such that the longer the time spent at a particular object, the higher the score.

[0107] Through the above process, it becomes possible to calculate and assign a score to live broadcast programs. The calculated score may be used to identify television advertising materials. The calculation unit may use at least one of the above scores, or it may use a total score calculated by assigning weights to each score. Furthermore, which score to use may be set by the user or by the genre of the live broadcast program.

[0108] The information processing device 10 may, for example, classify a live broadcast program into at least one category based on the object detection results for the live broadcast program. The identification unit 117 may identify television advertising material for the live broadcast program according to the classified category.

[0109] The identification unit 117 may also include identifying matching advertisements using the category to which the content is classified and the score of the live broadcast program. For example, the identification unit 117 identifies any television advertising material from among television advertising materials that have been assigned the same category (genre) as the category to which the live broadcast program is classified. For example, if the live broadcast program is classified under the category of sports, the identification unit 117 identifies at least one television advertising material from among those whose genre is sports.

[0110] Through the above process, by using the scores calculated for live broadcast programs to identify television advertising materials, it becomes possible to more appropriately identify television advertising materials that are relevant to the content of the program.

[0111] For example, if there are available advertising slots in the live broadcast program, the information processing device 10 identifies television advertising material that is highly relevant to the live broadcast program and assigns it to the available slot. In this case, the information processing device 10 may identify television advertising material for the live broadcast program based on object detection results and scored scores.

[0112] Furthermore, in the embodiments or modifications described above, a live broadcast program was used as an example for the specified program, but recorded programs may also be included. This makes it possible to assign metadata to the specified program regardless of the program format (recorded or live broadcast) by performing the object detection process described above while the specified program is being broadcast, regardless of whether it is a recorded or live broadcast, when metadata (e.g., detection result data) is generated from the specified program. In this case, the programs to which the television advertising material is transmitted are not limited to live broadcast programs, but also include recorded programs. In other words, the transmission of television advertising material that is highly relevant to object detection results and / or environmental data can be either live or recorded broadcasts.

[0113] Furthermore, the above-described embodiments or modifications illustrate examples in which the broadcasting station performs object detection, etc., and determines television advertising materials based on the object detection results, etc. (either directly within the broadcasting station or indirectly by adding information to auction requests). On the other hand, the advertiser may perform object detection, etc., on a predetermined program during broadcast using an information processing device under its own management, and use the object detection results, etc., to make auction bidding decisions provided by the auction device. [Explanation of Symbols]

[0114] 1...Information processing system, 10...Information processing device, 20...Object detection device, 30...External device, 110...Processor, 130...Memory, 111...Control unit, 112...Output unit, 113...First acquisition unit, 114...Second acquisition unit, 115...Third acquisition unit, 116...Metadata generation unit, 117...Specification unit, 118...Sending unit, 119...Calculation unit, 130...Storage device

Claims

1. Information processing device, The predetermined content to be broadcast live is output to an object detection device that performs object detection processing, To obtain the object detection results of the object detection process performed on the predetermined content from the object detection device, To identify television advertising material based on a predetermined object included in the object detection result, Controlling the broadcast of the specified television advertising material within the advertising slot for the predetermined content. An information processing method that performs the following.

2. The aforementioned determination means that This includes identifying the television commercial material using at least one of the performers, objects, scenery, and emotions evoked by the predetermined object, based on the object detection results. The information processing method according to claim 1.

3. To obtain the above means, This includes obtaining at least one of the following environmental data related to the predetermined content from a predetermined external device: weather, temperature, and stock price. The aforementioned determination means that This includes identifying the television advertising material based on the object detection results and the environmental data. The information processing method according to claim 1.

4. The aforementioned determination means that This includes identifying the television advertising material using the dwell time of an object within the predetermined content based on the object detection results, The information processing method according to claim 1.

5. The aforementioned determination means that This includes identifying the television advertising material using the duration of the emotion associated with the predetermined object based on the object detection results, The information processing method according to claim 1.

6. The output mentioned above is, This includes dividing the predetermined content into predetermined units and outputting each of the divided predetermined units to the object detection device, The aforementioned determination means that This includes allocating television advertising materials based on the predetermined objects detected in each of the predetermined units to each position in the advertising slots based on the broadcast order of the predetermined units. The information processing method according to claim 1.

7. Information processing device, To output to an auction device that holds an auction for managed advertising, advertising slot information relating to available advertising slots and the object detection results and / or environmental data corresponding to the advertising slots, The auction device is used to obtain identification information of the managed advertisement that won the vacant position. The aforementioned control means This includes controlling the transmission of television advertising material for a programmatic advertisement corresponding to the acquired identification information to the available position. The information processing method according to claim 3.

8. In an information processing device, The predetermined content to be broadcast live is output to an object detection device that performs object detection processing, To obtain the object detection results of the object detection process performed on the predetermined content from the object detection device, To identify television advertising material based on a predetermined object included in the object detection result, Controlling the broadcast of the specified television advertising material within the advertising slot for the predetermined content. A program that executes the command.

9. An output unit that outputs predetermined content to be broadcast live to an object detection device that performs object detection processing, An acquisition unit that acquires the object detection results of the object detection process performed on the predetermined content from the object detection device, A unit for identifying television advertising material based on a predetermined object included in the object detection result, A transmission unit controls the transmission of the specified television advertising material within the advertising slot for the predetermined content, An information processing device equipped with the following features.

Images