Information processing device, information processing method and information processing program

Abstract

To determine whether or not a logo image is appropriate.SOLUTION: An information processing device according to the present application includes a first determination unit, a detection unit, an identification unit, and a second determination unit. The first determination unit determines whether or not a background image of an image area estimated to contain a logo image satisfies a predetermined condition. The detection unit detects an edge contained in the image area. If the first determination unit determines that the background image is contained, the identification unit identifies a display position of the logo image based on the edge detected by the detection unit. The second determination unit determines whether or not the logo image at the display position identified by the identification unit satisfies a pre-specified rule.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] In recent years, advertisement distribution via networks such as the Internet has been actively carried out. In performing such advertisement distribution, for example, various techniques have been proposed for receiving submissions of content, which is an advertisement posted on a web page, from an advertiser and examining whether the received advertisement is appropriate for distribution (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the prior art, consideration has not been given to determining whether a logo image is appropriate.

[0005] The present invention has been made in view of the above, and an object thereof is to provide an information processing apparatus, an information processing method, and an information processing program capable of determining whether a logo image is appropriate.

Means for Solving the Problems

[0006] In order to solve the above-described problems and achieve the object, an information processing apparatus according to the present invention includes: a first determination unit that determines whether a background image of an image area estimated to include a logo image satisfies a predetermined condition; a detection unit that detects an edge included in the image area; a specification unit that specifies a display position of the logo image based on the edge detected by the detection unit when the first determination unit determines that the background image is included; and a second determination unit that determines whether the logo image at the display position specified by the specification unit satisfies a predetermined rule.

Effect of the Invention

[0007] According to the present invention, it is possible to determine whether a logo image is appropriate.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Mode for Carrying Out the Invention

[0009] Hereinafter, a mode (hereinafter referred to as "embodiment") for implementing an information processing apparatus, an information providing method, and an information providing program according to the present application will be described in detail with reference to the drawings. Note that the information processing apparatus, the information providing method, and the information providing program according to the present application are not limited by this embodiment. Also, each embodiment can be appropriately combined within a range that does not conflict with the processing contents. In the following embodiments, the same parts are denoted by the same reference numerals, and redundant explanations are omitted.

[0010] 〔1. Outline of Information Processing〕 FIG. 1 is a diagram showing an example of information providing processing according to an embodiment, and in this embodiment, an information processing method is executed by an information processing apparatus 1.

[0011] As shown in FIG. 1, the information processing apparatus 1 is communicably connected to a terminal device 2 used by a user U, and transmits and receives information to and from the terminal device 2. Note that the information processing apparatus 1 can also transmit and receive information to and from each of a plurality of terminal devices 2 used by a plurality of users U.

[0012] The terminal device 2 is a terminal device used by the user U, for example, a notebook PC (Personal Computer) or a desktop PC, but it may also be a smartphone or a tablet, etc. For example, the user U is an administrator responsible for tasks related to advertisement images such as the review of advertisement images created by each advertisement creator. For example, when an advertisement creator uses the logo image of the company that is the order source during the production process of an advertisement image, due to operational errors or the like, there are cases where the logo image is used that does not meet the criteria, such as when the aspect ratio of the logo image changes, or the color of the logo changes. Also, as another example, for instance, even though the criteria set by the company that is the order source for the logo image requires the background of the logo image to be plain, there are cases where, for example, the background of the logo image contains patterns.

[0013] The information processing device 1 receives an advertisement image from the terminal device 2 of the user U (step S1). For example, the information processing device 1 receives an advertisement image from the terminal device 2 through an application provided to the terminal device 2 or a web browser.

[0014] Here, on the terminal device 2, a reception screen G1 related to the advertisement image as shown in FIG. 1 is displayed. As shown in FIG. 1, the reception screen G1 has an upload area for uploading the image file of the advertisement image and an input area for inputting the type and quantity of the logo images included in the advertisement image to be uploaded.

[0015] When the user U uploads the image file of the advertisement image to the upload area, the information processing device 1 can receive the image file of the advertisement image uploaded from the terminal device 2.

[0016] At this time, for example, the information processing device 1 can receive a plurality of advertisement images as one image file according to the type of display medium or terminal (PC or smartphone) on which the advertisement image is displayed. That is, the information processing device 1 can receive a plurality of advertisement images associated with one campaign as one image file.

[0017] In addition, the user U inputs, for each type, the type and quantity of logo images included in each advertisement image into the input area. Thereby, the information processing apparatus 1 can easily recognize the type and quantity of the logo images. Here, the logo image is an image used by consumers to distinguish brands, and includes characters, figures, symbols, illustrations, and the like.

[0018] When the information processing apparatus 1 receives information regarding the advertisement image and the type and quantity of logo images included in the advertisement image, it specifies the display position where the logo image is displayed from each advertisement image (step S2).

[0019] Details will be described later, but the information processing apparatus 1 can specify the display position of the logo image by two-stage template matching using the template image of the type of logo image input by the user U.

[0020] Subsequently, the information processing apparatus 1 performs a rule determination regarding the logo image at the specified display position (step S3). The rule determination here determines whether the logo image included in the advertisement image satisfies the criteria for displaying as an advertisement image.

[0021] For example, if the logo image included in the advertisement image does not satisfy the criteria, there is a possibility that consumers cannot identify the brand from the advertisement image or that the brand image may be damaged. Therefore, the information processing apparatus 1 determines, for example, whether the logo image included in the advertisement image satisfies a preset rule. Such a rule can be preset by the user U, who is the administrator, etc., but it is also possible to download and set a predetermined convention, etc. from the web.

[0022] As will be described later, the information processing apparatus 1 determines whether the logo image included in the advertisement image satisfies a preset rule based on a plurality of determination items, and outputs the determination result of each determination item to the terminal device 2 (step S4).

[0023] Then, the terminal device 2 will display the determination result received from the information processing device 1. For example, a notification screen G2 indicating the determination result is displayed on the terminal device 2. On the notification screen G2, for each logo image included in the advertisement image, the determination result regarding each check item is displayed.

[0024] That is, the user U can easily grasp which logo image complies with or violates which check item based on the notification screen G2.

[0025] As described above, the information processing device 1 according to the embodiment determines whether the logo image included in the advertisement image satisfies a preset rule and outputs the determination result. Thus, according to the information processing device 1 according to the embodiment, it is possible to determine whether the logo image used in the advertisement image is appropriate.

[0026] [2. Configuration Example of Information Processing Device] Next, with reference to FIG. 2, a configuration example of the information processing device 1 according to the embodiment will be described. FIG. 2 is a block diagram showing a configuration example of the information processing device 1 according to the embodiment. As shown in FIG. 2, the information processing device 1 includes a communication unit 10, a storage unit 11, and a processing unit 12.

[0027] (Communication Unit 10) The communication unit 10 is realized by, for example, a NIC (Network Interface Card) or the like. Also, the communication unit 10 is connected to the network by wire or wirelessly.

[0028] (Storage Unit 11) The storage unit 11 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 2, the storage unit 11 includes a template information storage unit 20 and a rule information storage unit 21.

[0029] (Template Information Storage Unit 20) The template information storage unit 20 stores template information. The template information is information regarding a template image for extracting a logo image included in an advertisement image. FIG. 3 is a diagram showing an example of the template information storage unit 20 according to the embodiment.

[0030] As shown in FIG. 2, the template information storage unit 20 according to the embodiment has items such as "image ID", "image information", and "scale setting". The "image ID" is an identifier for identifying a template image corresponding to each logo image.

[0031] The "image information" is the image information of the template image. For example, the template information storage unit 20 stores different image information for each image ID. The "scale setting" is a setting regarding the scale (magnification) when using the image information identified by the image ID as a template image.

[0032] As shown in FIG. 3, the scale setting is different for each of the first granularity and the second granularity. In the example shown in FIG. 3, in the first granularity, the scale is in 25% increments, and in the second granularity, the scale is in 0.1% increments. That is, compared to the first granularity, the second granularity has the scale adjusted with a finer granularity.

[0033] (Rule information storage unit 21) The rule information storage unit 2** stores rule information. The rule information is information regarding rules used when determining whether a logo image included in an advertisement image satisfies the criteria for display as an advertisement image.

[0034] FIG. 4 is a diagram showing an example of the rule information storage unit 21 according to the embodiment. As shown in FIG. 4, the rule information storage unit 21 has items such as "rule ID", "outline", and "determination method".

[0035] The "Rule ID" is an identifier for identifying each rule. The "Summary" is the summary of the rule identified by the corresponding Rule ID. The "Judgment Method" is a method (e.g., an algorithm) for judging the rule identified by the corresponding Rule ID.

[0036] As shown in Figure 4, the summary of the rule identified by the rule ID "R1" is "Is there enough white space?", indicating that it is judged by the judgment method "#1". Here, the white space refers to the margin (clear space) between the logo image and the background of the advertisement image.

[0037] Also, the summary of the rule identified by the rule ID "R2" is "Is there no problem with the aspect ratio?", indicating that it is judged by the judgment method "#2". That is, in the rule ID "R2", it is judged whether the aspect ratio of the logo image included in the advertisement image is within an appropriate range. For example, when the aspect ratio of the template image is "A" and the aspect ratio of the logo image included in the advertisement image is "B", it can be judged using the score that is the absolute value of "(A - B) / A".

[0038] Also, the summary of the rule identified by the rule ID "R3" is "Is the logo size not too small?", indicating that it is judged by the judgment method "#3". That is, in the rule ID "R3", it is judged whether the logo image included in the advertisement image is not smaller than a reference value. For example, it is judged whether the width and height of the logo image satisfy a predetermined reference value such as 12 pixels or less.

[0039] Also, the summary of the rule identified by the rule ID "R4" is "Has the color used in the logo not changed color?", indicating that it is judged by the judgment method "#4". That is, it is judged whether the logo image included in the advertisement image has not changed color. Regarding a specific example of the judgment method "#4", it will be described later with reference to Figure 7.

[0040] Also, the outline of the rule identified by rule ID "R5" is "Is the color of the logo appropriate?", indicating that it is determined by the determination method "#5". For example, it is determined whether the color (RGB value) of text or the like included in the logo image is a predetermined value.

[0041] Also, the outline of the rule identified by rule ID "R5" is "Is the luminance difference from the background color sufficient?", indicating that it is determined by the determination method "#5". Also, the outline of the rule identified by rule ID "R6" is "Is the color difference from the background color sufficient?", indicating that it is determined by the determination method "#6".

[0042] That is, in rule ID "R5" and rule ID "R6", it is determined whether the contrast between the logo image included in the advertisement image and its background is appropriate. For example, in the determination method "#5", for the text and background of the logo image respectively, "((R×299)+(G×587)+(B×144)) / 10000" is calculated from the 255-level RGB values, and if the difference is equal to or greater than a predetermined value (for example, 125), it is determined as OK, and if it is less than the predetermined value, it is determined as NG.

[0043] For example, in the determination method "#6", based on the RGB values of the text and background of the logo image, when the maximum values (maxR, maxG, maxB) and minimum values (minR, minG, minB) are set respectively, "(maxR - minR)+(maxG - minG)+(maxB - minB)" is calculated, and if the difference is equal to or greater than a predetermined value (for example, 500), it is determined as OK, and if it is less than the predetermined value, it is determined as NG. Note that each rule shown in FIG. 4 is an example and is not limited thereto.

[0044] (Processing unit 12) The processing unit 12 is a controller, which is realized, for example, by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), etc., when various programs (corresponding to an example of an information providing program) stored in the storage device inside the information processing apparatus 1 are executed with the RAM as a work area. Further, the processing unit 12 is, for example, a controller, and is realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0045] As shown in FIG. 2, the processing unit 12 includes a reception unit 30, an extraction unit 31, a specification unit 32, a determination unit 33, and an output unit 34, and realizes or executes the functions and operations of information processing described below. Note that the internal configuration of the processing unit 12 is not limited to the configuration shown in FIG. 2, and may be any other configuration as long as it can perform the information processing described later. Also, the connection relationship between the respective processing units included in the processing unit 12 is not limited to the connection relationship shown in FIG. 2, and may be any other connection relationship.

[0046] (Reception Unit 30) The reception unit 30 receives an advertisement image including a logo image. For example, the reception unit 30 receives an advertisement image including a logo image from the terminal device 2 of the user U via the communication unit 10. At this time, the reception unit 30 receives information regarding the type and quantity of the logo images.

[0047] Note that the reception unit 30 may, for example, acquire various Web pages on which the advertisement image is actually displayed from the Web. That is, the reception unit 30 may receive not only the advertisement image before submission but also the advertisement image after submission.

[0048] In addition, the reception unit 30 receives, for example, a template image of a logo image and various rule information related to the logo image from the terminal device 2 of the user U. For example, the template image is stored in the template information storage unit 20 as template information, and the rule information is stored in the rule information storage unit 21.

[0049] In addition, as will be described later, the reception unit 30 can also receive an input operation for specifying the logo area or the display position of the logo image from the user U. This point will be described later with reference to FIG. 6.

[0050] (Extraction unit 31) The extraction unit 31 extracts a logo area including a logo image from the advertisement image received by the reception unit 30 by template matching using a template image of the first granularity.

[0051] For example, the extraction unit 31 sets the template image of the type of the logo image received by the reception unit 30 to the granularity set to the first granularity, and then, using the template images of each granularity, shifts the template image by one pixel at a time to calculate a score regarding the similarity for each pixel.

[0052] Then, the extraction unit 31 generates, for example, a heat map showing the score regarding the similarity for each pixel. After that, the extraction unit 31 extracts, for example, the top N logo areas based on the quantity input by the user U from the heat map. Note that the extraction of the logo area is not limited to template matching, and for example, it may be performed by inputting the advertisement image into a model generated by various machine learning methods.

[0053] (Specification unit 32) The specification unit 32 specifies the display position of the logo image from the logo area extracted by the extraction unit 31 by template matching using a template image of the second granularity finer than the first granularity.

[0054] Specifically, the specifying unit 32 sets the scale of the template image used by the extraction unit 31 for extracting the logo area to the second granularity, and performs template matching on the logo area. Then, the specifying unit 32 specifies, as the display position of the logo image, the area with the highest similarity score or an area exceeding a predetermined value.

[0055] Here, the template matching by the extraction unit 31 and the specifying unit 32 will be described with reference to FIG. 5. FIG. 5 is a conceptual diagram of the template matching according to the embodiment. As shown in FIG. 5, assume that the extraction unit 31 extracts the logo area 110 from the advertisement image 100 by template matching of the first granularity.

[0056] In this case, the specifying unit 32 excludes the area other than the logo area from the target area of the second granularity, using the logo area 110 as the target area for template matching using the template image of the second granularity.

[0057] That is, the specifying unit 32 performs template matching using the template image of the second granularity finer than the first granularity, limited to the logo area 110, so that compared with the case of performing template matching using the template image of the second granularity for all areas of the advertisement image 100, the processing load and processing time can be reduced.

[0058] In this way, the processing unit 12 performs two-stage template matching using the template images of the first granularity and the second granularity finer than the first granularity, thereby suppressing the processing load related to template matching and accurately specifying the display position of the logo image.

[0059] Thereafter, prior to the processing by the determination unit 33 described later, the processing unit 12 requests the user U to confirm the display position. FIG. 6 is a diagram showing an example of the confirmation screen displayed on the terminal device 2 according to the embodiment.

[0060] As shown in FIG. 6, a confirmation screen 200 is displayed on the terminal device 2 of the user U. For example, on the confirmation screen 200, an advertisement image that encloses a recognition area related to the display position recognized by the processing unit 12 with a rectangle is displayed.

[0061] If there is no deviation between the recognition area and the actual logo image, that is, if the processing unit 12 has appropriately recognized the display position of the logo image, the user U can request the start of the processing by the subsequent determination unit 33.

[0062] On the other hand, as shown in FIG. 6, when there is a deviation between the recognition area and the actual logo image, the user U operates the terminal device 2 and requests a screen transition to the input screen 210. On the input screen 210, the user U will enclose the display position of the logo image with a rectangle in response to an input operation using various input devices.

[0063] This information is transmitted from the terminal device 2 of the user U to the reception unit 30 of the processing unit 12. Again, the processing unit 12 performs template matching again for the area specified by the user U.

[0064] After these processes and after the visual check by the user U is completed, the processing unit 12 will start the processing by the subsequent determination unit 33. That is, by passing through the visual check by the user U, it is possible to prevent omission of the logo image.

[0065] (Determination Unit 33) The determination unit 33 determines whether the logo image included in the advertisement image received by the reception unit 30 satisfies a preset rule. For example, the determination unit 33 uses each determination method (algorithm) registered in the rule information storage unit 21 for the display position of the logo image specified by the specifying unit 32 to determine whether the logo image included in the advertisement image satisfies the rule.

[0066] The determination unit 33 uses each determination method set for rule ID "R1" to rule ID "R7" to determine whether the logo image included in the advertisement image satisfies the rule for each determination item.

[0067] Here, the determination method of color change determination corresponding to rule ID "R4" will be described with reference to FIG. 7. FIG. 7 is a diagram showing an example of color change determination according to the embodiment. As shown in FIG. 7, for the template image and the logo image, after removing their respective background colors, a histogram regarding the appearance frequency of RGB is generated (step S21).

[0068] Here, regarding the removal of the background color, first, the determination unit 33 registers the background color of the logo image for the template image using a predetermined tool (such as a spot tool) to unify the background colors of the template image and the logo image.

[0069] Subsequently, the determination unit 33 replaces the RGB values regarding the background color with values not used in the image, such as (-1, -1, -1). Thereby, the background colors of the template image and the logo image can be removed respectively.

[0070] Then, the determination unit 33 generates a histogram regarding the frequency of the RGB values of each pixel of the template image and the logo image with the background color removed. And the determination unit 33 performs color change determination by comparing the RGB values with the highest appearance frequencies in the respective histograms (step S22).

[0071] Specifically, in each histogram, when the RGB values with the highest application frequencies match, the determination unit 33 determines that the logo image has not changed color, and when the RGB values with the highest application frequencies do not match, the determination unit 33 determines that the logo image has changed color. That is, if the RGB values (r1, g1, b1) with the highest application frequency of the template image match the RGB values (r2, g2, b2) with the highest application frequency of the logo image, it is determined that there is no color change, and if they do not match, it is determined that there is color change.

[0072] Then, the determination unit 33 performs determination processing for each determination item including discoloration determination, and passes the result to the output unit 34. Note that the determination unit 33 may perform various determinations regarding the logo image, for example, by determination processing using a learned model. Furthermore, an explainable AI may be used as the learned model.

[0073] (Output unit 34) The output unit 34 outputs the determination result by the determination unit 33. For example, the output unit 34 generates various information regarding a notification screen G2 (see FIG. 1) including the determination results of each determination item based on the determination result by the determination unit 33.

[0074] Then, the output unit 34 outputs the various information regarding the generated notification screen G2 to the terminal device 2 of the user U. For example, as shown in FIG. 1, on the notification screen G2, it is possible to notify, using a check tool, which rules each logo image included in the advertisement image complies with and which rules it violates.

[0075] That is, the user U can easily grasp which rules are violated, so it becomes possible to easily improve the logo image. Note that the output unit 34 may output, for example, a proposed image in which a logo image violating any rule is replaced with a logo image that complies with the rule.

[0076] [3. Processing flow] Next, with reference to FIG. 8, the processing procedure executed by the information processing apparatus 1 according to the embodiment will be described. FIG. 8 is a flowchart showing an example of information processing according to the embodiment. Note that the following processing procedure is executed by the processing unit 12 of the information processing apparatus 1.

[0077] As shown in FIG. 8, the information processing apparatus 1 receives, for example, an advertisement image including a logo image from the terminal device 2 of the user U (step S101). Subsequently, the information processing apparatus 1 extracts a logo area from the advertisement image using a template image of the first granularity (step S102).

[0078] Subsequently, the information processing apparatus 1 specifies the display position of the logo image for the extracted logo area using the template image of the second granularity (step S103). Subsequently, the information processing apparatus 1 displays a confirmation screen regarding the display position on the terminal device 2 of the user U (step S104).

[0079] Subsequently, the information processing apparatus 1 determines whether or not it has received an input operation to correct the display position from the user U based on the display result of the confirmation screen (step S105). When the information processing apparatus 1 has received an input operation (step S105; Yes), it shifts to the process of step S102 for the area specified by the input operation.

[0080] Also, when the information processing apparatus 1 has not received an input operation, that is, when the visual check by the user U regarding the display position has passed (step S105; No), it determines the logo image for each determination item (step S106).

[0081] Then, the information processing apparatus 1 outputs the determination result to the terminal device 2 of the user U and ends the process (step S107).

[0082] [4. Modification Example] In the above-described embodiment, the case of determining the logo image for the advertisement image has been described, but it is not limited thereto. For example, it may be possible to determine whether the branded product is a genuine product or a pirated product from a product image in which the branded product is posted.

[0083] Also, in the above-described embodiment, the case where the user U inputs the type and quantity of the logo image has been described, but it is not limited thereto. For example, the information processing apparatus 1 may estimate the type and quantity of the logo image from the advertisement image using a model generated by various machine learning methods. In this case, the information processing apparatus 1 may use the model for a series of processes up to the extraction of the logo area, and may perform template matching for the process of specifying the display position of the logo image.

[0084] Further, without being limited to the above example, the information processing apparatus 1 may perform a series of processes including estimation processing of the type and quantity of logo images from an advertisement image, extraction processing of logo regions, and specification processing of the display positions of logo images using a model generated by various machine learning methods. At this time, for each process of extracting and shifting the logo region, the information processing apparatus 1 may set the quantity to be larger than the quantity estimated using the model. Thereby, for example, even when the logo region is estimated to be less than the actual quantity by the model, an appropriate quantity of logo regions can be extracted.

[0085] [5. Second Embodiment] In the above-described embodiment (first embodiment), for example, when something other than the logo is reflected in the logo image, it may not be applicable or the accuracy may be low because it is impossible to recognize where the logo body ends and where the non-logo part ends. For example, when a background image consisting of complex colors such as gradation or a faint pattern is included, it may not be recognized without an advanced deep learning method using a model trained with a large amount of data to determine where the gradation ends and where the pattern ends. The second embodiment has been made in view of the above, and an object thereof is to provide an information processing apparatus, an information processing method, and an information processing program that can determine whether a logo image is appropriate even for a complex logo image such as a case where something other than the logo is reflected in the logo image.

[0086] In the second embodiment, when the background color of the creative-side logo region is depicted with a gradation or a faint pattern so that it can be recognized even with a gradation background or the like, the logo image is determined to be appropriate or not by converting the image of the logo region into an edge image.

[0087] Hereinafter, the information provision process according to the second embodiment will be described with reference to FIG. 9. FIG. 9 is a diagram showing an example of the information provision process according to the second embodiment, and in this second embodiment, an information processing method is executed by the information processing apparatus 1A. Hereinafter, the case where the gradation background is reflected in the logo image will be described as an example, but the background image according to the second embodiment is not limited to the gradation background. For example, it may be a case where a light pattern such as a strawberry pattern is reflected in the logo image. Thus, any background that can be regarded as a background composed of complex colors may be the case where it is reflected in the logo image. Also, the same explanations as in the first embodiment will be omitted as appropriate below. For example, the explanations of the same reference numerals as in the first embodiment will be omitted as appropriate.

[0088] As shown in FIG. 9, the information processing apparatus 1A is communicably connected to the terminal device 2 used by the user U, and transmits and receives information to and from the terminal device 2. Note that the information processing apparatus 1A can also transmit and receive information to and from each of the plurality of terminal devices 2 used by the plurality of users U.

[0089] The information processing apparatus 1A receives an advertisement image from the terminal device 2 of the user U (step S31). For example, the information processing apparatus 1A receives an advertisement image from the terminal device 2 through an application or a web browser provided to the terminal device 2. At this time, the information processing apparatus 1A may presume that the received advertisement image includes a logo image.

[0090] By the user U uploading the image file of the advertisement image to the upload area, the information processing apparatus 1A can receive the image file of the advertisement image uploaded from the terminal device 2.

[0091] The information processing apparatus 1A converts an advertisement image (corresponding to the "image area presumed to include the logo image") into an edge image (step S32). In other words, the information processing apparatus 1A converts the advertisement image into an edge image by detecting the edges included in the advertisement image (performing edge detection on the advertisement image). By performing such edge detection, it becomes possible to determine the gradient between adjacent pixels even when there is a gradation background or a faint pattern. For example, when the difference in RGB values (or contrast) between adjacent pixels (the difference in RGB values between adjacent pixels) calculated by, for example, the Euclidean distance is large, it is highly likely that it is not a gradation background, and when the difference in RGB values between adjacent pixels is small, it becomes possible to make a determination that there is a high possibility that a gradation background or a faint pattern is drawn. FIG. 10 is an explanatory diagram for explaining the edge image according to the second embodiment. The hatched area shown in FIG. 10 is an area where the pixel values are regarded as the same, and this area is, for example, filled with solid black and excluded from the processing target in the subsequent processing (template matching processing). In FIG. 10, an edge image corresponding to the image display area of the logo image at the display position specified in the subsequent processing is shown as an example. Hereinafter, in the second embodiment, the hatched area will be used to separately explain the logo body (edge part) and the non-logo part (hatched part).

[0092] The information processing device 1A determines whether a gradation background or the like is drawn on the advertisement image (step S33). Specifically, the information processing device 1A determines whether the background color of the advertisement image is a single-color background. More specifically, the information processing device 1A determines whether the advertisement image includes pixels whose RGB value difference between adjacent pixels is equal to or less than a predetermined threshold value, and if it is determined that such pixels are included, it is determined that a gradation background or the like is drawn. And when the information processing device 1A determines that the background color of the advertisement image is not a single-color background, it performs the template matching process described below. Note that when the information processing device 1A determines that the background color of the advertisement image is a single-color background, it may determine the appropriateness of the logo image by the information processing according to the first embodiment. Also, in step S33, the information processing device 1A may determine whether a condition regarding complexity is satisfied as a predetermined condition. To give a specific example, the information processing device 1A may make a determination according to whether the number of colors is greater than a predetermined threshold value, or whether the number and area of regions where the pigment difference exceeds a predetermined threshold value are greater than a predetermined threshold value.

[0093] As an example of a method for distinguishing whether the background color of the advertisement image is a single-color background or a gradation background, there is a method of distinguishing based on whether the colors at the four corners (such as the four points at the upper left, lower left, upper right, and lower right of the rectangle) of the logo area cut out by a rectangle are exactly the same. The information processing device 1A may, for example, determine whether the colors at the four corners of the logo area are exactly the same, and if even one of them is different, it may be determined as a gradation background, a pattern background, or the like. However, if there is a background such as a strawberry pattern that does not cover any of the four corners, it may be overlooked. For this reason, in order to avoid such an oversight, the information processing device 1A may, for example, refer to the background area filled with transparent PNG in the template image, extract the background on the creative side, and determine whether all the pixel values on the background side are the same. And the information processing device 1A may, for example, perform a process of regarding it as a single-color background if all the pixel values on the background side are the same, and regarding it as a gradation background or the like if there is even one different color.

[0094] In addition, user U inputs the type and quantity of logo images included in each advertisement image for each type with respect to the input area. As a result, information processing apparatus 1A can easily recognize the type and quantity of the logo images.

[0095] When the information processing device 1A receives information regarding the types and quantities of advertisement images and logo images included in the advertisement images, it specifies the display positions where the logo images are displayed from each advertisement image by template matching using a template image (step S34). At this time, for example, the information processing device 1A may perform template matching by dividing it into a plurality of stages with different granularities. To give a specific example, the information processing device 1A may perform template matching in two stages: a template image with a first granularity and a template image with a second granularity. To give a more specific example, the information processing device 1A may specify the display position of the logo image by performing template matching using a template image with a second granularity that is finer than the first granularity on the logo area extracted by template matching using the template image with the first granularity. In this way, by roughly searching in the first stage and then searching again within that range in the second stage, it becomes possible to specify the accurate display position of the logo image. Also, during the template matching in the first stage, the information processing device 1A, for example, determines whether the aspect ratio of the extracted logo area matches the template image, and only when it is determined that they match, it may be the target of the template matching in the second stage. In the first stage, extraction is performed using a template image resized at a coarser scale interval. For this reason, although originally thousands of different sizes of template images should be prepared, in the first stage, it can be limited to, for example, about 100 types of template images, and in the second stage, also about 200 types of template images, so it is expected to speed up the processing with a small amount of calculation. Also, in determining whether the aspect ratio of the logo area matches the template image, the information processing device 1A, for example, calculates the aspect ratio of the logo registered as the template image, calculates the aspect ratio of the logo in the extracted logo area, and compares the aspect ratios of both to score the degree of distortion of the aspect ratio. When the score indicating the degree of distortion of the aspect ratio is large, it is regarded that the logo is distorted, and a notification (warning) is given that the logo is inappropriate.

[0096] Specifically, for example, when five logo images are drawn on the advertisement image, the information processing apparatus 1A may extract the top five regions with a high similarity to the template image and regard them as logo regions. However, here, the top eight regions with a high similarity are extracted, and the second-stage template matching is performed on the eight candidate regions. The similarity between the template image and the eight candidate regions is calculated again, and the top five of the eight with a high similarity may be regarded as logo regions. In this way, by increasing the number of regions extracted as candidate regions, an improvement in the extraction accuracy of the logo regions is expected.

[0097] For the logo image at the specified display position, the information processing apparatus 1A performs a rule determination regarding the logo image in the same manner as in the first embodiment (step S35). Similar to the first embodiment, the information processing apparatus 1A determines whether the logo image included in the advertisement image satisfies a pre-specified rule based on a plurality of determination items, and outputs the determination result of each determination item to the terminal device 2 (step S36).

[0098] Note that in step S35, the information processing apparatus 1A may perform further determination processing for improving the accuracy. Hereinafter, as a variation of the information processing in the second embodiment, a case where the information processing apparatus 1A performs further determination processing for improving the accuracy will be described. Note that in the following determination, the image before edge image conversion or the image after edge image conversion may be used.

[0099] (Clear Space Determination) The information processing apparatus 1A determines whether it is possible to surround the logo image at the specified display position with a rectangle and secure a specified clear space at the four corners of the rectangle (such as the four points at the upper left, lower left, upper right, and lower right of the rectangle). FIG. 11 is an explanatory diagram for explaining the clear space determination according to the second embodiment. As shown in FIG. 11, when a frame line (L1) is drawn on the logo image, edges are detected not only in the logo body but also in the frame line portion, and when a rectangle including the frame line is drawn, it becomes a large rectangle. Thus, when the logo body and the frame line cannot be clearly distinguished, all pixel values other than the hatched area are regarded as the logo body, and when a rectangle including the frame line is drawn, it becomes a large rectangle. Therefore, the information processing apparatus 1A determines whether the clear space generated by surrounding the logo image with a rectangle satisfies a predetermined condition, and determines that the rule is satisfied when sufficient clear space can be secured at the four corners of the rectangle, and determines that the rule is not satisfied when sufficient clear space cannot be secured at the four corners of the rectangle (when there is insufficient clear space). Also, the clear space determination may be performed during template matching, and by performing the clear space determination during template matching, accurate template matching becomes possible.

[0100] (Image size determination) The information processing device 1A determines whether the image size (such as pixel value) of the logo image at the specified display position is larger than the reference value. For example, the information processing device 1A determines whether both the vertical and horizontal dimensions (or width × height) of the logo image are larger than the reference value, and determines that the rule is not satisfied if either the vertical or horizontal dimension is below the reference value. Then, when the image size is below the reference value, the information processing device 1A may issue a notification (warning) indicating that the logo image is too small. At this time, the pixel value or the like serving as the reference value may be a value specified and registered by the user with reference to the brand logo guidelines or the like when registering the template image. Further, the information processing device 1A may determine whether the length of the side of the rectangle surrounding the logo body of the logo image at the specified display position is larger than the reference value. In this way, when a gradient background or the like is drawn, a rectangle may be drawn for the edge-imaged logo area, and it may be determined whether the rule is satisfied based on the length of the side of the rectangle.

[0101] (Brightness difference and color difference determination) In the brightness difference and color difference determination, the brightness difference and color difference between the background color of the logo image and the color of the logo body are determined. The information processing device 1A cuts out the image after ensuring a specified clear space for the logo image at the specified display position. Then, the information processing device 1A extracts the colors at the four corners of the cut-out logo image. At this time, the information processing device 1A may extract the colors inside by about several pixels at each of the four corners. By extracting the colors inside by about several pixels in this way, it becomes possible to extract the colors without including the frame line or the like drawn at the very edge of the logo image even if the cut-out is slightly off. Note that if the color is extracted using the frame line, the color of the frame line will become the background color and a bug will occur.

[0102] The information processing device 1A removes the background of the logo image and extracts the logo body (foreground part). Note that the background of the logo image is determined from the template image. For example, in the template image, a transparent area using an alpha channel, a completely white area, etc. can be regarded as the background. Also, to remove the background part defined in the template image from the logo image, it may be necessary to resize the logo image (including alignment, etc.). The information processing device 1A may, for example, remove the background of the logo image and extract the logo body through resizing of the logo image. Also, the information processing device 1A may, for example, further remove the background color that tends to remain on the outline of the logo through dilation and erosion processing and extract the logo body.

[0103] The information processing device 1A extracts a plurality of representative colors of the logo body part by applying the k-means method (k-average method) to the logo body (foreground part). In the k-means method, the greater the color difference, the greater the calculated distance. Also, the center point of the cluster becomes the representative color. Note that the information processing device 1A extracts the representative color instead of the most frequent color as described above. For example, in the contour part of the logo, the color may bleed, and even if red is drawn, in many cases, various colors from pink to red are actually included. Even when red and gray should be extracted, there is a possibility that only colors close to red, such as "red and pink", may be extracted. Therefore, it is necessary to extract the representative color. Also, by using the k-means method, it becomes possible to extract representative colors close to human perception.

[0104] Note that when extracting representative colors, the background may be erroneously extracted as the foreground. In such a case, it is necessary to exclude the color close to the background color from the representative colors. For example, when the background color is white and the foreground contains white, white is likely to be extracted as a representative color, so it is necessary to exclude white from the representative colors. For example, when six colors are extracted as representative colors, the information processing apparatus 1A may delete four colors close to the colors at the four corners of the logo and use the remaining two colors as the representative colors of the foreground part of the logo. Then, the information processing apparatus 1A determines the luminance difference and color difference based on the extracted representative colors and the background color. For example, when six colors are extracted as representative colors, the information processing apparatus 1A determines the luminance difference and color difference based on two representative colors (denoted as representative colors A1 and A2) and four background colors (background colors B1 to B4). Thus, when the background color is not a simple case such as a single color but a complex case such as a gradient background, the information processing apparatus 1A determines the luminance difference and color difference based on the representative colors and the background color. Note that in the above example, the representative colors are two colors, A1 and A2. Here, the representative colors are set to two colors instead of four colors or the like so that no bugs occur when only a very small number of representative colors are used, but it may not be particularly limited.

[0105] The information processing apparatus 1A may use Equation (1) for luminance difference determination. Equation (1) is an example of a function for determining whether the luminance difference is sufficient. Also, the information processing apparatus 1A may use Equation (2) for color difference determination. Equation (2) is an example of a function for determining whether the color difference is sufficient. Also, the information processing apparatus 1A may, for example, weight the luminance difference and color difference according to the use of the representative colors of the logo.

Equation

Equation

[0106] Thus, the information processing apparatus 1A according to the second embodiment converts the advertisement image into an edge image, determines whether a background image consisting of complex colors such as a gradient background is drawn, and when a background image consisting of complex colors is drawn, identifies the display position of the logo image through template matching using a template image such as a first granularity / a second granularity, determines whether the logo image at the identified display position satisfies a preset rule, and outputs the determination result. Thereby, according to the information processing apparatus 1A according to the second embodiment, it is possible to determine whether the logo image is appropriate.

[0107] [6. Configuration example of the information processing apparatus according to the second embodiment] Next, with reference to FIG. 12, a configuration example of the information processing apparatus 1A according to the second embodiment will be described. FIG. 12 is a block diagram showing a configuration example of the information processing apparatus 1A according to the second embodiment. As shown in FIG. 12, the information processing apparatus 1A includes a communication unit 10, a storage unit 11, and a processing unit 12A. Hereinafter, the same description as in the first embodiment will be appropriately omitted.

[0108] (Processing unit 12A) The processing unit 12A is a controller and is realized, for example, by various programs stored in the storage device inside the information processing apparatus 1A being executed with the RAM as a work area by a CPU, an MPU, or the like. Further, the processing unit 12A is realized by an integrated circuit such as an ASIC or an FPGA, for example.

[0109] As shown in FIG. 12, the processing unit 12A includes a reception unit 30, an extraction unit 31A, a specification unit 32A, a first determination unit 33A, a second determination unit 33B, an output unit 34, and a detection unit 35, and realizes or executes the information processing functions and operations described below. Note that the internal configuration of the processing unit 12A is not limited to the configuration shown in FIG. 12, and may be any other configuration as long as it can perform the information processing described later. Also, the connection relationship between the respective processing units included in the processing unit 12A is not limited to the connection relationship shown in FIG. 12, and may be any other connection relationship.

[0110] (Detection unit 35) The detection unit 35 detects edges included in a predetermined image region. Also, the detection unit 35 converts an image corresponding to the predetermined image region into an edge image through edge detection.

[0111] (First determination unit 33A) The first determination unit 33A determines whether the background image of the image region where a logo image is presumed to be included satisfies a predetermined condition. For example, the first determination unit 33A uses the edge image converted by the detection unit 35 to determine whether a background image composed of complex colors is included. For example, the first determination unit 33A determines whether a background image composed of complex colors is included by determining whether pixels whose RGB value difference between adjacent pixels is equal to or less than a predetermined threshold value are included in the image region.

[0112] (Specification unit 32A) The specification unit 32A specifies the display position of the logo image by performing template matching using a template image according to the determination result by the first determination unit 33A. For example, the specification unit 32A specifies the display position of the logo image from the edge image converted by the detection unit 35. Also, the specification unit 32A may perform a determination of the aspect ratio on the logo region extracted by the extraction unit 31A, and perform further template matching only when the aspect ratio satisfies a rule to specify the display position of the logo image.

[0113] (Second determination unit 33B) The second determination unit 33B determines whether the logo image at the display position specified by the specifying unit 32A satisfies a preset rule. For example, the second determination unit 33B determines whether the logo image satisfies a preset rule based on a plurality of determination items described in the first embodiment. Also, for example, the second determination unit 33B determines whether the logo image satisfies a preset rule in clear space determination, image size determination, and luminance difference / color difference determination.

[0114] (Extraction unit 31A) The extraction unit 31A extracts a logo area from the edge image converted by the detection unit 35. For example, the extraction unit 31A extracts the logo area by template matching using a template image of the first granularity.

[0115] [7. Processing flow according to the second embodiment] Next, with reference to FIG. 13, the processing procedure executed by the information processing apparatus 1A according to the second embodiment will be described. FIG. 13 is a flowchart showing an example of information processing according to the second embodiment. Note that the following processing procedure is executed by the processing unit 12A of the information processing apparatus 1A.

[0116] As shown in FIG. 13, the information processing apparatus 1A receives, for example, an advertisement image including a logo image from the terminal device 2 of the user U (step S201).

[0117] Subsequently, the information processing apparatus 1A converts the advertisement image into an edge image (step S202).

[0118] Subsequently, the information processing apparatus 1A extracts a logo area from the edge image using a template image of the first granularity (step S203).

[0119] Subsequently, the information processing apparatus 1A specifies the display position of the logo image for the extracted logo area using a template image of the second granularity (step S204).

[0120] Subsequently, the information processing apparatus 1A determines whether the logo image at the specified display position satisfies a preset rule (step S205).

[0121] Then, the information processing apparatus 1A outputs the determination result to the terminal device 2 of the user U and ends the process (step S206).

[0122] 〔8. Hardware Configuration〕 The information processing apparatus 1 (and the information processing apparatus 1A according to the second embodiment) or the terminal device 2 according to the first embodiment described above is realized by a computer 80 having a configuration as shown in FIG. 14, for example. Hereinafter, the information processing apparatus 1A will be described as an example. FIG. 14 is a hardware configuration diagram showing an example of a computer 80 that realizes the functions of the information processing apparatus 1A according to the second embodiment. The computer 80 includes a CPU 81, a RAM 82, a ROM (Read Only Memory) 83, an HDD (Hard Disk Drive) 84, a communication interface (I / F) 85, an input / output interface (I / F) 86, and a media interface (I / F) 87.

[0123] The CPU 81 operates based on programs stored in the ROM 83 or the HDD 84 and controls each part. The ROM 83 stores a boot program executed by the CPU 81 when the computer 80 is started up, programs dependent on the hardware of the computer 80, and the like.

[0124] The HDD 84 stores programs executed by the CPU 81, data used by such programs, and the like. The communication interface 85 receives data from other devices via a communication network and sends it to the CPU 81, and sends data generated by the CPU 81 to other devices via the communication network.

[0125] The CPU 81 controls output devices such as displays and printers, and input devices such as keyboards or mice via the input / output interface 86. The CPU 81 acquires data from the input device via the input / output interface 86. Also, the CPU 81 outputs the data generated via the input / output interface 86 to the output device.

[0126] The media interface 87 reads a program or data stored in the recording medium 88 and provides it to the CPU 81 via the RAM 82. The CPU 81 loads such a program from the recording medium 88 onto the RAM 82 via the media interface 87 and executes the loaded program. The recording medium 88 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc), PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0127] For example, when the computer 80 functions as the information processing apparatus 1A according to the second embodiment, the CPU 81 of the computer 80 realizes the functions of the processing unit 12A by executing the program loaded onto the RAM 82. Also, the data in the storage unit 11 is stored in the HDD 84. The CPU 81 of the computer 80 reads and executes these programs from the recording medium 88, but as another example, these programs may be acquired from other devices via a communication network.

[0128] 〔9. Others〕 Also, among the processes described in the above embodiments, all or part of the processes described as being automatically performed can be manually performed, or all or part of the processes described as being manually performed can be automatically performed by a known method. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the illustrated information.

[0129] Moreover, each component of each illustrated device is conceptually functional and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of the distribution and integration of each device is not limited to that shown in the figure, and all or part of it can be functionally or physically distributed and integrated in any unit according to various loads and usage situations.

[0130] Also, the information processing device 1 (and the information processing device 1A) may be composed of two or more server devices. In this case, the information processing device 1 (and the information processing device 1A) has a configuration including, for example, a processing server and a storage server. In addition, the above-described embodiments and modification examples can be appropriately combined within a range that does not conflict with the processing content.

[0131] 〔10. Effects〕 The information processing device 1A according to the second embodiment described above includes a first determination unit 33A that determines whether a background image of an image region estimated to include a logo image satisfies a predetermined condition, a detection unit 35 that detects an edge included in the image region, and a specifying unit 32A that specifies the display position of the logo image based on the edge detected by the detection unit 35 when the first determination unit 33A determines that the background image is included, and a second determination unit 33B that determines whether the logo image at the display position specified by the specifying unit 32A satisfies a predetermined rule.

[0132] In addition, when the image area includes pixels where the RGB value difference between adjacent pixels is equal to or less than a predetermined threshold value, the first determination unit 33A determines that the background image is included.

[0133] In addition, the specifying unit 32A specifies the display position of the logo image by template matching using the template image.

[0134] In addition, the specifying unit 32A specifies the display position of the logo image by template matching using a template image with a second granularity finer than the first granularity for the logo area extracted by template matching using the template image with the first granularity.

[0135] In addition, the second determination unit 33B determines whether or not a blank area generated by surrounding the logo image at the display position with a rectangle satisfies a predetermined condition.

[0136] In addition, the second determination unit 33B determines whether or not the size of the rectangle surrounding the logo image at the display position satisfies a predetermined condition.

[0137] In addition, the second determination unit 33B determines whether or not the color information of the logo image at the display position and the color information of the background image satisfy a predetermined condition.

[0138] By any one or combination of the above-described processes, the information processing apparatus according to the present application can determine whether or not the logo image is appropriate.

[0139] As described above, the embodiments of the present application have been described in detail with reference to the drawings. However, this is an example, and the present invention can be implemented in other forms in which various modifications and improvements are made based on the knowledge of those skilled in the art, including the aspects described in the column of the disclosure of the invention.

[0140] In addition, the "section, module, unit" described above can be read as "means", "circuit", or the like. For example, the acquisition unit can be read as an acquisition means or an acquisition circuit.

Description of Reference Numerals

[0141] 1. 1A Information Processing Device 2. Terminal Device 10. Communication Unit 11. Memory Unit 12, 12A Processing Unit 20. Template Information Memory Unit 21. Rule Information Memory Unit 30. Reception Unit 31, 31A Extraction Unit 32, 32A Identification Unit 33. Judgment Unit 33A. First Judgment Unit 33B. Second Judgment Unit 34. Output Unit 35. Detection Unit U User

Claims

1. A first determination unit that determines whether or not a background image of an image area estimated to include a logo image satisfies a predetermined condition; A detection unit that detects edges included in the image area; When the first determination unit determines that the background image is included, a specifying unit that specifies a display position of the logo image based on the edges detected by the detection unit; A second determination unit that determines whether or not the logo image at the display position specified by the specifying unit satisfies a predetermined rule; An information processing apparatus, comprising:

2. The first determination unit: When pixels having an RGB value difference between adjacent pixels equal to or less than a predetermined threshold value are included in the image area, determines that the background image is included. The information processing apparatus according to claim 1, characterized in that:

3. The specifying unit: Specifies the display position of the logo image by template matching using a template image. The information processing apparatus according to claim 1, characterized in that:

4. The specifying unit: For a logo area extracted by template matching using a template image of a first granularity, specifies the display position of the logo image by template matching using a template image of a second granularity finer than the first granularity. The information processing apparatus according to claim 1, characterized in that:

5. The second determination unit: Determines whether or not a blank area generated by surrounding the logo image at the display position with a rectangle satisfies a predetermined condition. The information processing apparatus according to claim 1, characterized in that:

6. The second determination unit: Determines whether or not the size of a rectangle surrounding the logo image at the display position satisfies a predetermined condition. The information processing apparatus according to claim 1, characterized in that:

7. The second determination unit: Determines whether or not color information of the logo image at the display position and color information of the background image satisfy a predetermined condition. The information processing apparatus according to claim 1, characterized in that:

8. An information processing method executed by a computer, comprising: A first determination step of determining whether or not a background image of an image area estimated to include a logo image satisfies a predetermined condition; A detection step of detecting edges included in the image area; When it is determined in the first determination step that the background image is included, a specifying step of specifying a display position of the logo image based on the edges detected in the detection step; A second determination step of determining whether or not the logo image at the display position specified by the specific step satisfies a pre-specified rule An information processing method characterized by including the above

9. A first determination procedure for determining whether or not the background image of the image area presumed to include the logo image satisfies a predetermined condition A detection procedure for detecting an edge included in the image area When it is determined by the first determination procedure that the background image is included, a specifying procedure for specifying the display position of the logo image based on the edge detected by the detection procedure A second determination procedure for determining whether or not the logo image at the display position specified by the specifying procedure satisfies a pre-specified rule An information processing program characterized by causing a computer to execute the above

Images