system

Abstract

A system is provided.SOLUTION: The system includes a means for collecting advertisement creatives on the Internet, a means for storing the collected advertisement creatives in a database, a means for analyzing texts, images, and moving images of the advertisement creatives and extracting risk elements, a means for calculating an NG rate by comparing a past determination result with a current analysis result, and a means for generating a determination reason and a risk level in a text format and storing them in the database.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] As advertising creatives on the Internet increase, there has been a sharp increase in cases where automated ads generated by generative AI contain illegal or fraudulent activities. Such illegal ads can degrade the quality of media and have a negative impact on users' lives. However, many Internet media outlets find it difficult to conduct thorough ad screening due to profitability and cost constraints. Therefore, it is necessary to improve media quality and protect users by quickly and efficiently detecting and responding to illegal and fraudulent ads. [Means for solving the problem]

[0005] The present invention provides a system that has a means for collecting advertising creatives on the Internet, a means for storing the collected advertising creatives in a database, a means for analyzing the text, images, and videos of the advertising creatives and extracting risky elements, a means for calculating an NG rate by comparing past judgment results with current analysis results, and a means for generating judgment reasons and risk levels in text format and storing them in a database.

[0006] Furthermore, by providing a system including a means for providing an interface that allows human judges to check the judgment results and make corrections or add feedback, a means for saving the obtained feedback information as re-learning data, a means for calculating the response priority of advertising creatives based on risk and profitability, and a means for providing an interface that lists advertising creatives in order of priority, it is possible to efficiently detect and respond to illegal advertisements.

[0007] The system also provides a means for generating a final review report for the ad creatives that have been reviewed, and a means for storing the generated final report in a database and notifying the relevant personnel. This system will streamline the ad review process, provide highly accurate review results, improve media quality, and ensure user safety.

[0008] The "Internet" is a general term for a computer network that interconnects users from all over the world, and is a digital space where information can be viewed and exchanged.

[0009] "Advertising creative" refers to the entire content of an advertisement delivered over the Internet, including text, banners, and videos.

[0010] "Harvesting" refers to the process of obtaining specific advertising creative from the internet and incorporating it into the system.

[0011] "Database" means a system that systematically stores and manages information such as collected advertising creatives and evaluation results.

[0012] "Analysis" is the process of examining the content of advertising creative in detail and extracting specific elements and features.

[0013] "Risk elements" refer to specific keywords, images, or video parts contained in ad creative that are associated with illegal or fraudulent activity.

[0014] "Past review results" refers to information that records the results of previous reviews of advertising creatives.

[0015] The "NG rate" is a number that indicates the probability that an advertising creative will be determined to be illegal or fraudulent.

[0016] The "reason for judgment" is a statement generated based on the analysis results of the advertising creative, explaining why the advertisement was judged to be illegal or fraudulent.

[0017] The "risk level" is an evaluation index that indicates the severity of the risk of the advertising creative.

[0018] A "judge" is a human representative whose role is to review the results of the ad creative and provide corrections and feedback.

[0019] "Retraining data" refers to data that the AI ​​model uses to retrain itself to improve its accuracy based on feedback provided by judges.

[0020] "Profitability" refers to the degree of economic benefit that an advertising creative brings.

[0021] "Priority" is an evaluation criterion that indicates the urgency and importance of the advertising creative response.

[0022] "Interface" refers to the operation screen and input means that allow the user to interact with the system. [Brief explanation of the drawings]

[0023] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0024] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0025] First, the terms used in the following description will be explained.

[0026] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0027] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0028] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0029] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0030] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0031] [First embodiment]

[0032] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0033] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0034] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0035] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0036] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0037] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0038] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0039] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0040] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0041] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0042] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0043] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0044] This invention is a system that detects illegality and fraudulent activities by collecting advertising creatives from the Internet and analyzing their content. This system automatically analyzes the text, banners, videos, etc. of advertising creatives and calculates the NG rate by comparing it with past judgment results. This improves the quality of advertising and prevents the negative impact of illegal advertising on people's lives.

[0045] System configuration

[0046] The system consists of the following main components:

[0047] 1. Ad creative acquisition module:

[0048] The server collects ad creatives from various internet sources, including ad serving platform APIs, RSS feeds, web scraping, etc. The collected data is stored in a database and assigned a unique ID.

[0049] 2. Text Analysis Module:

[0050] The server analyzes the text of the saved ad creative using natural language processing technology, which detects risky keywords and expressions such as "free" and "limited."

[0051] 3. Image Analysis Module:

[0052] The server uses image recognition technology to analyze the content of the advertising banners, extracting icons, text, and color schemes from the images to determine whether they may be illegal.

[0053] 4. Video Analysis Module:

[0054] The server uses video analysis technology to analyze the content of the advertising video frame by frame, and uses voice recognition technology to convert the audio content into text and detect inappropriate messages.

[0055] 5. Risk Assessment Module:

[0056] The server compares past results with the current analysis result and calculates a NG rate, which includes weighting certain factors.

[0057] 6. Reasons for decision generation module:

[0058] The server generates the reason for the judgment and the risk level based on the analysis results and stores them in a database in text format. For example, it outputs a reason such as "The risk is high because it contains specific keywords."

[0059] 7. Feedback Collection Module:

[0060] The terminal provides an interface that allows users (human judges) to check the results of the assessment and input feedback, which is sent to the server and saved as retraining data.

[0061] 8. Priority Judgment Module:

[0062] The server calculates the priority of the ad creative based on the risk and profitability of the analysis results, sorts the ads in order of priority, and displays them on the device.

[0063] 9. Final Report Generation Module:

[0064] The server generates a final review report for the reviewed advertising creatives, which is stored in a database and notified to relevant personnel.

[0065] Specific examples

[0066] For example, if a banner ad contains text like "Buy now" or "Limited edition":

[0067] 1. The server collects these banner ads and stores them in a database.

[0068] 2. The text analysis module analyzes the banner text and detects risky keywords.

[0069] 3. The image analysis module analyzes banner images to check for inappropriate content and design.

[0070] 4. The risk assessment module references past data on similar advertisements and calculates the NG rate.

[0071] 5. The judgment reason generation module generates a reason such as "It is highly risky because it contains specific keywords."

[0072] 6. The feedback collection module accepts the judges' review and feedback.

[0073] 7. The priority determination module displays this advertisement on the list as a high priority.

[0074] 8. The final report generation module generates the final audit report and notifies the relevant personnel.

[0075] In this way, the system improves the quality of online advertising by quickly assessing the risks of advertising creatives and encouraging appropriate responses.

[0076] The processing flow will be explained below.

[0077] Step 1: Collect ad creatives

[0078] Server: Collects advertising creatives (text, banners, videos) from multiple advertising platforms and websites on the Internet. Ad creatives are periodically obtained using API integration and web scraping technology.

[0079] Example: The server retrieves "new ads" through the API of an ad distribution platform and stores them in a database.

[0080] Step 2: Saving to the database

[0081] Server: Collected ad creatives are assigned a unique ID and stored in a database along with metadata (acquisition date, advertiser, ad type, etc.).

[0082] Example: The server saves the image file of the advertisement banner it has acquired in storage, and records the file path and related metadata in a database.

[0083] Step 3: Content analysis

[0084] Server: Performs text analysis, image analysis, and video analysis on ad creatives. Natural language processing (NLP) technology is used to analyze text, OCR technology is used to extract text from images, image recognition is performed, video analysis technology is used to analyze the content of each frame, and speech recognition is used to convert audio content into text.

[0085] Example: The server analyzes images in advertising banners to detect risky keywords such as "special price" and "buy now."

[0086] Step 4: Risk Assessment

[0087] Server: Based on the analysis results, the NG rate of the ad creative is calculated by comparing it with past judgment results. Specific risk factors are weighted to determine the overall risk level.

[0088] Example: Calculate the similarity between ads that have been judged to be "illegal" in the past and current ad creatives, and calculate the NG rate.

[0089] Step 5: Generate and save the reason for the decision

[0090] Server: Based on the analysis results and risk level of the advertising creative, the reason for the judgment and the risk level are generated in text format and saved in the database.

[0091] Example: The server generates and saves the reason for the judgment in text format, such as "This ad contains the keyword 'free' and therefore has a high risk of fraud."

[0092] Step 6: Judges gather feedback

[0093] Terminal: Provides an interface where human judges can review the results and make corrections or provide feedback as needed. Judges review the analysis results of the ad creative and the reasons for their decisions, and evaluate their accuracy.

[0094] Example: A judge checks the evaluation results of an advertising banner on a terminal and enters feedback such as "This advertisement is appropriate."

[0095] Step 7: Save the feedback and use it for retraining

[0096] Server: The acquired feedback information is saved as re-learning data. By re-learning the AI ​​model, the accuracy of judgments will be improved from the next time onwards.

[0097] Example: The server adds feedback data from judges to the database and incorporates it into the retraining dataset.

[0098] Step 8: Determine and list priorities

[0099] Server: Calculates the response priority of each ad creative based on risk and profitability, and displays a list of ad creatives in order of priority through the interface.

[0100] Example: The server sets high profitability and high risk advertisements as priority 1 and displays them in a prominent position on the device.

[0101] Step 9: Generate the final report

[0102] Server: Generates a final review report for the ad creatives that have been reviewed, stores it in a database, and notifies the relevant personnel.

[0103] Example: The server generates a final report for the advertisement "New Product Launch" and notifies the person in charge, including results such as "No problems" and "Appropriate wording."

[0104] These are the processing steps of the system, which allows for efficient advertising screening and improves the reliability of media.

[0105] Example 1

[0106] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0107] Some online advertisements contain illegal or fraudulent content, which can have a negative impact on users. In particular, it is difficult to properly evaluate and monitor advertisements in a consistent manner across a wide range of formats (text, images, and videos). Furthermore, there is a lack of a system for effectively feeding back the results of human judges' judgments and using them as retraining data. To address these issues, it is necessary to automate and streamline the entire process, from analyzing advertising creatives to assessing risk, prioritizing, and generating a final report.

[0108] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0109] In this invention, the server includes: means for collecting advertising creatives from the Internet; means for storing the collected advertising creatives in a database; means for analyzing the text, images, and video of the advertising creatives to extract risky elements; means for calculating a rejection rate by comparing past judgment results with current analysis results; means for analyzing the text portion of the advertising creatives using natural language processing technology to detect risky keywords; means for analyzing the content of advertising banners using image recognition technology to determine potential illegality; means for analyzing the content of advertising videos frame by frame using video analysis technology to detect inappropriate messages; and means for generating and storing the judgment reasons and risk levels in a database. This improves the quality of advertising on the Internet and minimizes adverse effects on users. The server also includes means for providing an interface for human reviewers to review the judgment results and make corrections or provide feedback; means for saving the acquired feedback information as re-learning data; means for calculating the response priority of advertising creatives based on risk and profitability; and means for providing an interface for displaying advertising creatives in order of priority, thereby automating and streamlining this entire process.

[0110] "Internet advertising creative" refers to the overall content of advertisements distributed over the Internet, including text, images, and videos.

[0111] "Means of collection" refers to programs and algorithms used to obtain data from the internet, including methods such as APIs, RSS feeds, and web scraping.

[0112] "Means of storing data in a database" refers to systems and algorithms for efficiently storing and managing collected data, including platforms for storing data in a particular format.

[0113] "Means of analysis" refers to technologies and programs that use machine learning and AI technology to evaluate and analyze the text, images, and videos of advertising creatives.

[0114] "Risk elements" refer to specific text, image, or video elements that pose concerns about illegality or fraud, such as specific keywords, icons, or colors.

[0115] The "NG rate" is an indicator that shows how risky an ad creative is compared to past judgment results, and is often expressed as a number or percentage.

[0116] "Natural language processing technology" refers to artificial intelligence technology for analyzing meaning and structure from text data, and specifically includes the use of machine learning algorithms and APIs.

[0117] "Image recognition technology" refers to the technology of analyzing image data to extract and recognize specific features, and computer vision technology and deep learning algorithms are commonly used.

[0118] "Video analysis technology" refers to the technology of analyzing video data frame by frame to extract and evaluate specific content, and includes video processing software and voice recognition technology.

[0119] "Reasons for judgment and risk level" refers to the analytical reasons generated as a result of evaluating the advertising creative and indicators that quantitatively and qualitatively indicate the level of risk.

[0120] "Feedback interface" refers to a user interface that allows human reviewers to review the analysis results and enter corrections and feedback.

[0121] "Retraining data" refers to data used to improve the performance of an AI model based on feedback information, and is used through a continuous learning process.

[0122] The "means for calculating priority" refers to a program for using an algorithm to calculate the priority of a response based on the risk and profitability of an advertising creative.

[0123] "Listing interface" refers to a user interface that displays advertising creatives in order of priority, allowing users to efficiently respond.

[0124] "Final Report" refers to a comprehensive report containing analysis results and feedback on the advertising creatives that have been reviewed, and is a document stored in a database.

[0125] "Means of notification" refers to the system or method for notifying relevant personnel of the generated final report, including email and notification systems.

[0126] MODE FOR CARRYING OUT THE INVENTION

[0127] The present invention is a system for detecting illegal or fraudulent activities by collecting and analyzing advertising creatives on the Internet. Specific embodiments of this system will be described below.

[0128] System configuration

[0129] The system mainly consists of three elements: a server, a terminal, and a user. The server collects, analyzes, and evaluates data, and the terminal provides a user interface. The user checks the information provided by the system and provides feedback as needed.

[0130] Hardware and software used

[0131] 1. The server uses the following technologies to collect, analyze, and evaluate advertising creatives:

[0132] Use APIs and RSS feeds to gather advertising creatives from across the internet. Examples include APIs of ad serving platforms, RSS feeds, and web scraping.

[0133] The text is analyzed using natural language processing technology (e.g., Google (registered trademark) Cloud Natural Language API) to detect risky keywords such as "free" and "limited."

[0134] Image recognition technology (e.g., OpenCV, Google Cloud Vision API) is used to analyze the content of advertising banners and determine whether they may be illegal.

[0135] Video analysis technology (e.g., FFmpeg, Google Cloud Video Intelligence API) is used to analyze the video content frame by frame, and speech recognition technology is used to convert the audio content into text and detect inappropriate messages.

[0136] The database will be used to store collected advertising creatives, analysis results, reasons for judgments, etc.

[0137] 2. The device provides an interface for the user to review the analysis results and provide feedback. Specific elements include:

[0138] The user interface visually displays the analysis results, allowing the user to easily check the judgment results.

[0139] The feedback function provides an interface where the user can enter corrections and feedback on the results.

[0140] 3. The user acts as a judge, reviewing the information provided by the system and providing feedback if necessary.

[0141] Specific examples

[0142] For example, if a banner ad contains text such as "Buy now" or "Limited edition," the system's processing flow is as follows:

[0143] 1. The server collects these banner ads and stores them in a database.

[0144] 2. The server uses a text analysis module to analyze the banner text and detect risky keywords (e.g., "Buy now" or "Limited Edition").

[0145] 3. The server uses an image analysis module to analyze the banner image and check for inappropriate content and design.

[0146] 4. The server uses the risk assessment module to refer to past similar ad data and calculate the NG rate.

[0147] 5. The server uses the judgment reason generation module to generate a reason such as "It is highly risky because it contains specific keywords."

[0148] 6. The device accepts the judges' confirmation and feedback through the feedback collection module.

[0149] 7. The server uses the priority determination module to display this advertisement on the list as a high priority.

[0150] 8. The server uses the final report generation module to generate the final review report and notify the relevant personnel.

[0151] Prompt Sentence Examples

[0152] Here is an example of a prompt to input to the generative AI model:

[0153] Please explain the system that collects and analyzes advertising creatives containing keywords such as "buy now" and "limited edition" to assess their potential for illegality.

[0154] In this way, the system improves the quality of online advertising by quickly assessing the risks of advertising creatives and encouraging appropriate responses.

[0155] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0156] Step 1: Collect ad creatives

[0157] Input: Ad serving platform API, RSS feed, website URL

[0158] How it works: The server collects ad creatives from specified internet sources. The server uses the ad serving platform's API to get the latest ad lists, extracts newly published ads from RSS feeds, and uses web scraping to collect ad banners from specific web pages.

[0159] Output: Collected ad creative data

[0160] Step 2: Saving to the database

[0161] Input: Collected ad creative data

[0162] Specific operation: The server stores the collected ad creatives in a database, assigns a unique ID, and stores metadata for each ad creative (e.g., collection source, date, format, etc.).

[0163] Output: Ad creatives and their metadata stored in a database

[0164] Step 3: Text analysis

[0165] Input: Ad creative text data loaded from the database

[0166] Specific operation: The server analyzes the text using natural language processing technology (e.g., Google Cloud Natural Language API). It detects risky keywords and expressions such as "free" and "limited" and marks them.

[0167] Output: Parsed text data with risk keywords marked

[0168] Step 4: Image analysis

[0169] Input: Ad creative image data loaded from database

[0170] How it works: The server analyzes the content of the ad banner using image recognition technology (e.g., OpenCV, Google Cloud Vision API), extracting icons, text, and colors from the image and determining whether they match indicators of illegality.

[0171] Output: Analyzed image data and a list of potentially illegal elements

[0172] Step 5: Video Analysis

[0173] Input: Video data of the ad creative loaded from the database

[0174] Specific operation: The server analyzes the content of the advertisement video frame by frame using video analysis technology (e.g., FFmpeg, Google Cloud Video Intelligence API), converts the audio content into text using speech recognition technology, and detects inappropriate messages.

[0175] Output: Analyzed frame data and text data based on speech recognition

[0176] Step 6: Risk Assessment

[0177] Input: Analyzed text data, image data, video data

[0178] Specific operation: The server compares past judgment results with the current analysis results to calculate the NG rate. It weights each specific element and evaluates the overall risk level.

[0179] Output: Risk assessment results including NG rate

[0180] Step 7: Generate reasons for the decision

[0181] Input: Risk assessment results

[0182] Specific operation: The server generates the reason for the judgment and the risk level based on the analysis results and saves them in a database in text format. The reason for the judgment is automatically generated and the specific reason is output in text format.

[0183] Output: Reason for judgment and risk level stored in the database

[0184] Step 8: Gather feedback

[0185] Input: Reason for judgment and risk level stored in the database

[0186] Specific operation: The terminal provides an interface that allows the user (judge) to check the judging results and add corrections or feedback. The user enters feedback, and the terminal sends it to the server.

[0187] Output: Feedback data sent to the server

[0188] Step 9: Prioritization

[0189] Input: Risk assessment results, feedback data

[0190] Specific operation: The server calculates the response priority of the ad creative based on the risk and profitability obtained from the analysis results, sorts the ads in order of priority, and displays them on the device.

[0191] Output: A list of ad creatives sorted by priority

[0192] Step 10: Generate the final report

[0193] Input: Final evaluation and feedback results

[0194] Specific operations: The server generates a final review report for the advertising creatives that have been reviewed, stores it in the database, and notifies the relevant personnel.

[0195] Output: Final report stored in database and notification sent to relevant parties through notification system

[0196] In this way, the processes at each step are organically linked to provide a system for assessing and managing the risks of advertising creatives.

[0197] (Application example 1)

[0198] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0199] The Internet is flooded with a wide variety of advertisements, many of which are illegal or involve fraudulent activities. This puts users at high risk of coming into contact with inappropriate advertisements, which could potentially impair their quality of life. Manually checking advertisement content requires a huge amount of time and effort, making it inefficient. Furthermore, there is an inadequate system for detecting advertising risks in real time and notifying users, making it difficult to prevent harm caused by advertisements.

[0200] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0201] In this invention, the server includes means for collecting advertising creatives from the Internet, means for saving the collected advertising creatives in a database, means for analyzing the text, images, and videos of the advertising creatives and extracting risky elements, means for comparing past judgment results with current analysis results to calculate an NG rate, means for generating judgment reasons and risk levels in text format and saving them in a database, means for scanning and analyzing the collected advertisements in real time while browsing, and means for displaying a warning to the user based on the analysis results. This makes it possible to automatically detect illegal or fraudulent advertising and notify the user in real time.

[0202] "Internet advertising creative" refers to advertising materials such as text, images, and video that are distributed via the Internet, such as websites and applications.

[0203] "Means of collection" refers to functions and modules for detecting advertising creatives on the Internet and capturing and storing them as data.

[0204] "Database" means an information management system that enables efficient storage, management, and access of collected advertising creatives and analysis results.

[0205] "Means of analysis" refers to algorithms or programs that examine the content (text, images, video) of advertising creatives and detect risky elements.

[0206] "Risk elements" refer to keywords, images, audio, etc. contained in the content of advertising creatives that may raise suspicions of illegality or fraud.

[0207] The "judgment result" is the evaluation or judgment derived from the analysis of the advertising creative.

[0208] The "NG rate" is a number that indicates the probability that illegal or fraudulent activity will be detected in the analyzed advertising creative.

[0209] "Text format" refers to a format in which analysis results and reasons for judgments are expressed as text information.

[0210] "Real-time scanning while browsing" refers to the ability to instantly detect and analyze advertisements displayed while a user is browsing a website.

[0211] "Means to display a warning to users" refers to a function that notifies users of risky advertising creatives when they are detected, and provides a warning or caution.

[0212] The present invention relates to a system that analyzes advertising creatives on the Internet in real time to detect illegal or fraudulent activities. This system has the function of analyzing the text, images, and videos of advertisements, evaluating the risk level, and displaying a warning to the user. The following describes in detail the embodiments of the present invention.

[0213] System configuration

[0214] Hardware and software used

[0215] 1. Hardware

[0216] Smartphones: Used for real-time analysis when users view ads, using built-in sensors such as the camera, display, and microphone.

[0217] Server: Responsible for major back-end processing such as collecting ad creatives, analyzing, and storing data.

[0218] 2. Software

[0219] Python: A programming language used to implement the analysis program.

[0220] BeautifulSoup: A web scraping library used to scrape ad creatives from the web.

[0221] OpenCV: An image processing library used to analyze advertising images and videos.

[0222] Pytesseract: An OCR library used for analyzing text in images.

[0223] TENSORFLOW(R): A machine learning framework used for natural language processing such as text analysis and speech recognition.

[0224] System Operation

[0225] 1. Collecting advertising creatives

[0226] The server collects ad creatives from designated internet sources (websites, APIs, etc.), stores the collected data in a database, and then analyzes it. For example, it retrieves ads from news sites and social media platforms.

[0227] 2. Analysis of advertising creatives

[0228] The server analyzes the text, images, and videos of the stored ad creatives. Text analysis uses natural language processing technology to detect risky keywords and expressions. Image analysis examines the text, icons, and color usage within images to determine the possibility of fraud. Video analysis analyzes each frame of video, converting audio into text to detect inappropriate content.

[0229] 3. Risk assessment and warning signs

[0230] The server compares past judgment results with the analysis results to calculate the NG rate. The analysis results are used to generate the reason for the judgment and the risk level, which are then saved in a text database. When a user views an advertisement in real time while browsing, the advertisement is scanned, the risk level is assessed based on the analysis results, and a warning is displayed if necessary.

[0231] 4. User Interface and Feedback Collection

[0232] The system provides an interface for users to respond to warning messages and collects user feedback. This feedback is sent to the server and saved as retraining data. Once the review is complete, a detailed review report is generated and notified to the relevant personnel.

[0233] Specific examples

[0234] As a concrete example, consider the case where analysis is performed using the following prompt sentence:

[0235] Analyze the ad text below to see if it contains any risky keywords.

[0236] Text: "Buy now, limited time offer, 100% free"

[0237] In this way, the system can efficiently detect illegal or fraudulent advertising creatives and provide users with real-time warnings, thereby improving the quality of advertising on the Internet and ensuring user safety.

[0238] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0239] Step 1:

[0240] The server collects ad creatives from specified internet sources (websites, APIs, etc.). For example, it detects ad containers from news sites or social media platforms and retrieves their contents in HTML format. The input is the website URL or API endpoint, and the output is the ad creative data.

[0241] Step 2:

[0242] The server stores the collected ad creatives in a database. The collected ad creatives are recorded in the database with a unique ID for subsequent analysis. The input is the ad creative data collected in step 1, and the output is a new record added to the database.

[0243] Step 3:

[0244] The server uses natural language processing technology to analyze the text portion of the ad creative stored in the database. Specifically, it extracts risky keywords (e.g., "free," "now," "limited," etc.) from the text data. The input is the text data of the ad creative, and the output is a list of risky keywords and expressions.

[0245] Step 4:

[0246] The server uses image recognition technology to analyze the content of the ad banner. It uses OCR technology to detect text, icons, color usage, etc. in the image and extracts risky elements. The input is the image data of the ad banner, and the output is the analyzed text and risky elements.

[0247] Step 5:

[0248] The server uses video analysis technology to analyze the content of the ad video. It analyzes the text and audio for each frame of the video to detect inappropriate content and expressions. The input is the ad video data, and the output is the extracted text and audio content, as well as risky elements.

[0249] Step 6:

[0250] The server compares past judgment results with the current analysis results to calculate the NG rate. Based on the analysis results for each element of the ad creative (text, images, video), the NG rate is calculated while weighting specific risk elements. The input at this time is the analysis result data, and the NG rate is obtained as the output.

[0251] Step 7:

[0252] The server generates the reason for the judgment and the risk level in text format and saves them in a database. Based on the analysis results, it generates a reason for the judgment, such as "It is highly risky because it contains specific keywords." The input at this time is the analysis result and NG rate, and the reason for the judgment is obtained in text format as an output.

[0253] Step 8:

[0254] The device scans and analyzes the ads displayed while the user is browsing in real time. The collected ads are analyzed on the server and a warning is displayed to the user if necessary. The input is the ad data displayed in real time, and the output is a warning about risky ads.

[0255] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0256] The advertising review system of the present invention analyzes the content of advertising creatives (text, banners, videos) collected from the Internet to detect illegality and fraudulent activities. Furthermore, by incorporating an emotion engine that recognizes user emotions, the present invention performs more advanced risk assessment of advertising and realizes optimization of display methods.

[0257] System configuration

[0258] The system consists of the following main components:

[0259] 1. Ad creative acquisition module:

[0260] The server collects advertising creatives (text, banners, videos) from various internet sources using the APIs of ad serving platforms and web scraping technology.

[0261] Example: A server uses the API of an ad distribution platform to periodically collect the latest ad banners and store them in a database.

[0262] 2. Text Analysis Module:

[0263] The server uses natural language processing technology to analyze the text content of the collected advertising creatives and identify inappropriate language and risky keywords.

[0264] Example: The server uses OCR technology to extract text from banner images and detect keywords such as "Buy now" and "Limited Edition."

[0265] 3. Image Analysis Module:

[0266] The server uses image recognition technology to analyze the visual content of advertising banners, identifying those promoting counterfeit products or false health information.

[0267] Example: The server analyzes the icons and colors in an advertising banner and determines that it may contain false information.

[0268] 4. Video Analysis Module:

[0269] The server uses video analysis technology to analyze each frame of the advertising video and uses voice recognition technology to convert the audio content into text.

[0270] Example: The server extracts and analyzes messages such as "Great deals" from the audio in the video.

[0271] 5. Emotion Engine:

[0272] The server is equipped with an emotion engine that recognizes the user's emotions in real time, detecting their emotional state (happiness, surprise, anger, sadness, etc.) through facial recognition and voice analysis.

[0273] Example: The server analyzes the user's camera footage and microphone input using an emotion engine to determine the emotion the user is expressing while watching an advertisement.

[0274] 6. Risk Assessment Module:

[0275] The server calculates the NG rate and risk level based on the analysis of the ad creative and the user's emotion recognition. For example, if the user expresses negative emotions toward the ad, it will determine that the risk is high.

[0276] Example: The server reflects the fact that "the user showed anger while watching an advertisement" in the risk assessment.

[0277] 7. Reasons for decision generation module:

[0278] The server generates a reason for the judgment based on the risk assessment result and stores it in a database. The reason for the judgment is generated in text format.

[0279] Example: The server generates and stores a reason for the judgment, such as "There is a high risk of fraud because it contains certain keywords."

[0280] 8. Feedback Collection Module:

[0281] The terminal provides an interface that allows users (human judges) to review the results and provide corrections or feedback as needed.

[0282] Example: A judge adds feedback to the evaluation results, such as "This advertisement is not illegal."

[0283] 9. Priority Judgment Module:

[0284] The server determines the priority of the advertising creatives taking into consideration both the risk and profitability, and displays them on the terminal in order of priority.

[0285] Example: The server determines that the advertisements that are both highly profitable and highly risky are given priority 1 and are displayed on the device.

[0286] 10. Final Report Generation Module:

[0287] The server prepares a final review report for the advertising creatives for which review has been completed, and notifies the relevant personnel.

[0288] Example: The server generates a final report for the advertisement "New Product Launch" and notifies the person in charge, including results such as "No problems" and "Appropriate wording."

[0289] This system effectively detects illegal and fraudulent advertising creatives and recognizes user sentiment in real time, thereby improving advertising quality and optimizing the user experience.

[0290] The processing flow will be explained below.

[0291] Step 1: Collect ad creatives

[0292] The server collects advertising creatives (text, banners, videos) from multiple ad distribution platforms and websites on the Internet, periodically retrieving advertisements using API integration and web scraping technology, and storing them in a database.

[0293] Step 2: Saving to the database

[0294] The server stores the collected ad creatives in a database with a unique ID, along with metadata (date and time of acquisition, advertiser, ad type, etc.).

[0295] Step 3: Content analysis

[0296] The server performs the following analysis on the ad creative:

[0297] Text analysis: Analyzes text content using natural language processing (NLP) techniques to detect risky keywords.

[0298] Image analysis: Using image recognition technology to analyze the visual content of advertising banners to identify inappropriate elements or fraudulent products.

[0299] Video analysis: Using video analysis technology, each frame of the advertising video is analyzed and the audio content is converted into text through speech recognition.

[0300] Step 4: Emotion Recognition

[0301] The server analyzes the user's camera footage and microphone input in real time using an emotion engine that recognizes the user's emotions, detecting the emotions (happiness, surprise, anger, sadness, etc.) that the user shows while watching the advertisement.

[0302] Step 5: Risk Assessment

[0303] The server integrates the results of the ad creative analysis and emotion recognition, and calculates the NG rate by comparing it with past judgment results. It also weights specific risk factors and determines the overall risk level.

[0304] Step 6: Generate and save the reason for the decision

[0305] The server generates a reason for the judgment in text format based on the risk assessment result and stores it in the database. The reason for the judgment includes the specific elements that were judged to be high risk.

[0306] Step 7: Collect and store feedback

[0307] The terminal provides an interface for human judges to review the results and provide corrections or feedback as necessary. The judges' feedback is sent to the server and saved as re-learning data.

[0308] Step 8: Determine and list priorities

[0309] The server calculates the response priority of the ad creatives based on the analysis results and profitability information, and displays them on the device in order of priority, allowing high-risk ads to be quickly addressed.

[0310] Step 9: Generate and communicate final report

[0311] The server generates a final review report for the reviewed advertising creatives and stores it in a database, and notifies the relevant personnel of the report.

[0312] Specific examples

[0313] Collection phase: The server retrieves new banner ads from the ad distribution platform via API and stores the collected data in a database.

[0314] Analysis phase: The server uses natural language processing technology to analyze the text of the advertising banner, detecting risky keywords such as "only now" and "limited edition." It also analyzes the image content to assess the possibility that it contains fraudulent products.

[0315] Emotion recognition phase: When a user watches an advertisement, the server uses an emotion engine to analyze the user's facial expressions and detect the emotions shown by the user in real time.

[0316] Evaluation phase: The server integrates the analysis results and emotion recognition results, compares them with past judgment results, calculates the NG rate, and generates the reason for the judgment.

[0317] Feedback phase: The judges check the reasons for the judgement on the terminal and input their feedback on whether the judgement was appropriate. This feedback is sent to the server and used for the next analysis.

[0318] Priority calculation phase: The server calculates the priority based on the profitability and risk of the ad creative, and displays it on the device in order of priority.

[0319] Final report phase: The server generates a final review report, stores it in the database, and notifies the person in charge.

[0320] In this way, the system efficiently detects illegal and fraudulent advertising and, by taking into account evaluations based on user emotions, improves advertising quality and optimizes the user experience.

[0321] Example 2

[0322] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0323] There is a wide variety of advertising creatives on the Internet, some of which may contain inappropriate or illegal content. This poses a risk of adversely affecting users, so there is a need for a system that can efficiently and effectively analyze advertising content and detect illegal or inappropriate content. However, conventional systems have problems such as low analysis accuracy and an inability to take user emotions into account.

[0324] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0325] In this invention, the server includes means for collecting advertising creatives from the Internet, means for storing the collected advertising creatives in a database, means for analyzing the text, images, and videos of the advertising creatives and extracting risky elements, means for recognizing user emotions in real time, means for calculating a risk level based on the results of the advertising creative analysis and emotion recognition, and means for generating the reasons for the determination and the risk level in text format and storing them in the database. This makes it possible to analyze the content of advertising creatives with high accuracy, efficiently detect illegal or inappropriate content, and perform risk assessment taking user emotions into consideration.

[0326] The "Internet" is a global digital communications system that connects computer networks around the world.

[0327] "Ad creative" refers to the elements of an advertisement displayed on the Internet, including content in any media format, such as text, banners, or video.

[0328] A "server" is a computer system that provides information and services over a network and processes data in response to requests from clients.

[0329] A "database" is an integrated data storage system for efficiently managing, searching, and updating large amounts of data.

[0330] "Text analysis" is the process of using natural language processing technology to understand the content of text data and extract inappropriate expressions and risky keywords.

[0331] "Image analysis" is the process of using image recognition technology to analyze the content of image data and detect inappropriate elements or false information.

[0332] "Video analysis" is the process of analyzing video data frame by frame and converting the audio content into text using speech recognition technology.

[0333] "Emotion recognition" is a technology that detects a user's emotional state (joy, surprise, anger, sadness, etc.) in real time through facial recognition technology and voice analysis.

[0334] The "risk level" is an evaluation index regarding the inappropriateness or illegality of an advertisement, calculated based on the content of the advertising creative and the results of user emotion recognition.

[0335] The "reason for assessment" is text that describes the specific reason for the risk assessment, generated based on the analysis results of the ad creative.

[0336] "Feedback" is evaluation information provided by judges who review the results of their assessments and add corrections or opinions as necessary.

[0337] "Profitability" is a metric that measures the degree of profit an advertisement is likely to generate.

[0338] "Priority" is an evaluation indicator that indicates the urgency and importance of a response based on the analysis results of advertising creative.

[0339] "Review Report" means a document that describes the final evaluation results for the advertising creatives that have been reviewed and is notified to the relevant personnel.

[0340] This invention is a system that can effectively detect illegal or inappropriate content in advertisements by analyzing the content of advertising creatives (text, banners, videos) collected on the Internet with high accuracy and recognizing user emotions in real time. Furthermore, it aims to perform risk assessment of advertisements and select appropriate display methods.

[0341] Hardware and software used

[0342] Server: The server, which is the core of this system, collects and analyzes advertising creatives and stores data. Its primary hardware is a high-performance computer with large-capacity storage. Software includes APIs from ad distribution platforms, web scraping tools, natural language processing engines (e.g., Python's NLTK or spaCy), OCR engines (e.g., Tesseract), image recognition technology (e.g., OpenCV), speech recognition engines (e.g., Google Speech-to-Text API), and emotion recognition engines (e.g., Microsoft® Azure®'s Emotion API).

[0343] Devices: The devices used by judges display the results of their judgement and provide an interface for providing feedback. These devices can be PCs, tablets, or smartphones.

[0344] Users: The users of this system include advertisers, judges, and general advertising users.

[0345] Specific examples of program processing

[0346] 1. Collecting advertising creatives

[0347] The server collects advertising creatives from various sources on the Internet using the API of the ad serving platform or a web scraping tool. For example, the server periodically sends a request to the ad serving platform to obtain the latest advertising banners and stores the collected data in a database.

[0348] 2. Analysis of advertising creatives

[0349] The server analyzes the collected text, images, and videos of the advertising creative using various analytical techniques, such as using an OCR engine to extract text from banner images and a natural language processing engine to detect specific keywords such as "buy now."

[0350] 3. Emotion recognition

[0351] The server uses an emotion recognition engine to recognize the user's emotions in real time. It analyzes the video and audio data acquired through the user's camera and microphone to recognize emotions such as joy, surprise, anger, and sadness. For example, the server detects the emotion "the user is surprised" from the user's microphone input.

[0352] 4. Risk assessment and risk calculation

[0353] The server calculates the risk level of the ad based on the analysis results of the ad creative and the user's emotion recognition. For example, if the user's anger towards the ad is 50% or more, the risk level is set to "high."

[0354] 5. Generating reasons for judgment

[0355] The server generates a reason for the judgment based on the risk assessment result and stores it in the database. The reason for the judgment is detailed in text format. For example, the server may generate a reason for the judgment that "the risk of fraud is high because a specific keyword is included."

[0356] 6. Gathering User Feedback

[0357] The terminal provides an interface for the judge to check the judging results and send feedback if necessary. For example, the judge can use the terminal to input feedback such as "This ad is OK" and send it to the server.

[0358] 7. Priority determination and display

[0359] The server determines the priority of the ad creative based on the risk and profitability, and displays them on the device in order of priority. For example, the server may set and display ads that are both highly profitable and highly risky as priority 1.

[0360] 8. Generate the final report

[0361] The server creates a final review report for the advertising creative that has been reviewed and notifies the relevant person in charge. For example, the server generates a final report for the advertisement "New Product Launch," concludes that there are no problems, and notifies the person in charge.

[0362] Example prompts to input to the generative AI model

[0363] As an example of a prompt statement, you can enter:

[0364] "Analyze the following ad creative to determine if its content is inappropriate and perform a risk assessment taking into account the results of the sentiment engine. Analyze the following text: 'Shop now! Limited offer!'"

[0365] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0366] Step 1: Collect ad creatives

[0367] Specific actions

[0368] The server collects advertising creatives from online ad distribution platforms and websites, using APIs and web scraping technology to obtain data and store it in a database.

[0369] Input: API endpoint of ad serving platform, target URL for web scraping

[0370] Processing: Sending API requests, parsing HTML, extracting data

[0371] Output: Collected ad creative dataset (text, banner, video)

[0372] Step 2: Text analysis

[0373] Specific actions

[0374] The server analyzes the text content of the collected ad creatives using natural language processing technology, extracts text from images using OCR technology, and detects risky keywords.

[0375] Input: Ad creative dataset (text, banner)

[0376] Processing: Text extraction using OCR, keyword detection using natural language processing technology

[0377] Output: Text data tagged with keywords

[0378] Step 3: Image analysis

[0379] Specific actions

[0380] The server uses image recognition technology to analyze the visual content of advertising banners to detect inappropriate products and false health information.

[0381] Input: Ad creative dataset (banner image)

[0382] Processing: Image recognition algorithms are used to analyze and detect inappropriate content.

[0383] Output: Image tag as analysis result (if inappropriate elements are detected)

[0384] Step 4: Video Analysis

[0385] Specific actions

[0386] The server uses video analysis technology to analyze each frame of the ad video, and speech recognition technology to convert the audio content into text and determine whether it contains inappropriate content.

[0387] Input: Ad creative dataset (video files)

[0388] Processing: Frame-by-frame analysis, speech-to-text recognition, and inappropriate content detection

[0389] Output: Texted audio data and analysis report

[0390] Step 5: Emotion Recognition

[0391] Specific actions

[0392] The server utilizes an emotion recognition engine to recognize the user's emotions in real time, using facial recognition technology and voice analysis to detect the emotional state the user is expressing.

[0393] Input: User's camera and microphone input

[0394] Processing: Analysis by emotion recognition engine, identification of emotional state

[0395] Output: Emotion recognition results (happiness, surprise, anger, sadness, etc.)

[0396] Step 6: Risk Assessment

[0397] Specific actions

[0398] The server calculates the risk level based on the analysis results of the ad creative and the user's emotion recognition. For example, if the user expresses negative emotions toward the ad, it will determine the risk level as high.

[0399] Input: text, image, video analysis results, emotion recognition results

[0400] Processing: Data integration and analysis, risk level calculation

[0401] Output: Risk assessment report

[0402] Step 7: Generate reasons for the decision

[0403] Specific actions

[0404] The server generates a reason for the judgment based on the risk assessment result and stores it in the database. The reason for the judgment is detailed in text format.

[0405] Input: Hazard Assessment Report

[0406] Processing: Generating and formatting the reason for the decision

[0407] Output: Reason for decision in text format

[0408] Step 8: Gather feedback

[0409] Specific actions

[0410] The terminal provides an interface for judges to review the results and provide feedback if necessary.

[0411] Input: Judge feedback information

[0412] Process: Enter and send feedback information

[0413] Output: Corrected evaluation data

[0414] Step 9: Determine and display priorities

[0415] Specific actions

[0416] The server determines the priority of the advertising creatives based on the risk and profitability, and displays them on the terminal in order of priority.

[0417] Inputs: Risk assessment report, profitability data

[0418] Processing: Evaluating and determining priority, generating display data

[0419] Output: A list of ad creatives ordered by priority

[0420] Step 10: Generate the final report

[0421] Specific actions

[0422] The server prepares a final review report for the advertising creatives for which review has been completed, and notifies the relevant personnel.

[0423] Input: All analysis results and feedback information

[0424] Processing: Final report generation, formatting, and notification

[0425] Output: Final review report

[0426] (Application example 2)

[0427] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0428] There is a wide variety of advertising creatives on the Internet, many of which are illegal or fraudulent. Therefore, there is a need for a system that can automatically and effectively analyze advertising content and detect illegal or fraudulent activities. It is also necessary to evaluate the emotional impact of advertising on users and improve advertising quality. Furthermore, it is important to perform these evaluations in real time and improve advertising based on appropriate feedback.

[0429] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0430] In this invention, the server includes means for collecting advertising creatives from the Internet, means for saving the collected advertising creatives in a database, means for analyzing the text, images, and videos of the advertising creatives and extracting risky elements, means for comparing past judgment results with current analysis results to calculate an NG rate, means for evaluating the risk of displaying an advertisement using an emotion engine that recognizes the user's emotional state in real time, means for optimizing the advertising creatives based on the user's emotion recognition results, and means for generating judgment reasons and risk levels in text format and saving them in a database. This makes it possible to automatically detect illegality and fraudulent behavior in advertising creatives in real time and optimize advertising display taking user emotions into consideration.

[0431] "Internet advertising creative" means the content, including text, images, or video, of an advertisement displayed over the Internet.

[0432] "Means of collection" refers to the technology or method used to automatically collect advertising creatives published on the Internet and store them in a database.

[0433] "Means for storing in a database" refers to the technology or method for storing the collected advertising creative content in a digital format and managing it so that it can be subsequently analyzed and searched.

[0434] "Means of analysis and extraction of risky elements" refers to technologies and methods that use natural language processing, image recognition, voice analysis, etc. to analyze the content of advertising creatives and detect elements that may be illegal or fraudulent.

[0435] "Means for calculating NG rate" refers to the technology or method for comparing past judgment results with current analysis results and calculating the percentage of inappropriate advertisements.

[0436] The "emotion engine that recognizes the user's emotional state in real time" is a system that uses facial recognition and voice analysis technology to detect the emotions (joy, surprise, anger, sadness, etc.) of users viewing advertisements in real time.

[0437] "Means for assessing the risk of displaying advertisements" refers to technologies and methods for assessing the risk level of advertising creatives based on the results of user emotion recognition and taking appropriate measures.

[0438] "Means for optimizing advertising creatives based on user emotion recognition results" refers to technologies and methods that adjust the content and display method of advertisements based on user emotion data, thereby improving the user experience.

[0439] "Means for generating the reasons for judgment and risk levels in text format and storing them in a database" refers to the technology and methods for generating the analysis results of advertising creatives as text and storing them in a database.

[0440] "An interface where human judges can review the results of their assessments and make corrections or provide feedback" refers to a user interface that allows human judges to review the analysis results of advertisements on a screen and make corrections or comments as necessary.

[0441] "Means for saving as re-learning data" refers to the technology and methods for collecting and saving feedback information as data for machine learning and using it to improve the system.

[0442] The "means for calculating the response priority of advertising creatives based on risk level and profitability" refers to a technique or method for determining the response priority in consideration of the risk level and profitability of advertising creatives.

[0443] The "interface for displaying a list of advertising creatives in order of priority" refers to a user interface for displaying a list of advertising creatives on a screen according to their corresponding priorities.

[0444] "Means for auditing the legality of advertising creatives in real time using smartphones" refers to technologies and methods that use smartphone functions to analyze and audit the illegality of displayed advertising creatives in real time.

[0445] "Means for generating an examination report" refers to a technology or method for creating a final analysis result for an advertising creative in the form of a report.

[0446] "Means for storing the audit report in a database and notifying relevant personnel" refers to the technology or method for storing the generated audit report in a database and notifying the personnel in charge of the contents of the report.

[0447] "Means for creating an evaluation report based on the results of user emotion recognition" refers to a technology or method for creating an evaluation report that includes an evaluation of the advertising creative and points for improvement, taking into account user emotion data.

[0448] A specific method for implementing the advertisement quality audit application according to the present invention will be described. In this embodiment, an application using a smartphone as a platform is exemplified, but the application can be implemented in a similar manner on other devices.

[0449] System configuration

[0450] 1. How to obtain advertising creatives

[0451] The server automatically collects advertising creatives published on the Internet using the API of the advertising distribution platform. For example, it periodically retrieves the latest advertising banners and videos and stores them in a database.

[0452] 2. Database storage method

[0453] The server stores the collected advertising creatives in a digital database, which is structured to facilitate subsequent analysis and retrieval.

[0454] 3. Text Analysis Methods

[0455] The server analyzes the text of the ad creative using natural language processing techniques, for example, OCR technology to extract text from ad images and detect marketing keywords such as "buy now."

[0456] 4. Image and video analysis methods

[0457] The server uses image recognition and video analysis technology to analyze the content of the ad creative in real time, checking the content and audio data of each frame to identify counterfeit products and false information.

[0458] 5. Use of Emotion Engine

[0459] The server uses the smartphone's camera and microphone to recognize the user's emotions in real time, using facial recognition and voice analysis technology to analyze the emotional state (happiness, surprise, anger, sadness, etc.) displayed by the user while watching the advertisement.

[0460] 6. Advertisement display risk assessment method

[0461] The server evaluates the NG rate and risk level based on the results of ad analysis and user emotion recognition. For example, if a user expresses anger or discomfort while watching an ad, it will be deemed high risk.

[0462] 7. Ad creative optimization methods

[0463] The server adjusts the content and display method of the advertisement based on the user's emotional data to improve the user experience.

[0464] 8. Judgment reason generation / storage means

[0465] The server generates a reason for the judgment and a risk level in text format based on the analysis results and stores this in a database. For example, it may generate a judgment reason such as "There is a high risk of fraud because a specific keyword is included."

[0466] 9. Feedback Collection Methods

[0467] Users can review the results through a provided interface and provide corrections and feedback if necessary, allowing the system to continuously learn and improve its accuracy.

[0468] Examples of examples and prompts to use

[0469] For example, if an ad creative contains the text "Buy Now" and the user expresses anger, the ad will be judged to be highly likely to be fraudulent. Feedback provided by users regarding the ad will also be saved as learning data for the system and reflected in future analyses.

[0470] Prompt Sentence Examples

[0471] We need an application that can detect illegal and fraudulent advertising creatives and analyze user sentiment in real time. This includes the following features:

[0472] 1. Collect advertising creatives from the internet

[0473] 2. Illegal detection through text analysis

[0474] 3. Image and video analysis

[0475] 4. Facial Recognition and Emotion Analysis

[0476] 5. Generate a report of the results

[0477] Based on this requirement, generate the best model.

[0478] The present invention effectively detects advertisements that contain illegal or fraudulent activities and optimizes advertisement display taking into account user emotions, thereby enabling improvements in advertisement quality and user experience.

[0479] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0480] Step 1:

[0481] The server automatically collects advertising creatives (images, videos, text) from the Internet through the advertising distribution platform's API.

[0482] Input: Ad serving platform API

[0483] Data processing and calculation: Obtaining advertising data from the internet and converting it into digital format

[0484] Output: Collected ad creatives

[0485] Step 2:

[0486] The server stores the collected advertising creatives in a database.

[0487] Input: Collected ad creatives

[0488] Data processing and calculation: Store advertising data in the appropriate tables in the database

[0489] Output: Ad creatives stored in a database

[0490] Step 3:

[0491] The server uses OCR technology to extract text from the advertising image and analyzes the text using natural language processing (NLP) technology.

[0492] Input: Ad creative image data stored in a database

[0493] Data processing and calculation: Extract text using OCR and analyze keywords using NLP

[0494] Output: Parsed text and its risk factors

[0495] Step 4:

[0496] The server uses image recognition and video analysis technology to analyze the ad creative and identify elements that may contain counterfeit products or false information.

[0497] Input: Image and video data of advertising creatives stored in the database

[0498] Data processing and calculation: Risk element extraction through image recognition and video frame analysis

[0499] Output: Analyzed image and video data and their risk factors

[0500] Step 5:

[0501] The server uses the smartphone's camera and microphone to collect data to recognize the user's emotions in real time.

[0502] Input: Camera image and microphone audio from the user's smartphone

[0503] Data processing and calculation: Emotion recognition using facial recognition and voice analysis technology

[0504] Output: User's emotional state data

[0505] Step 6:

[0506] The server evaluates the rejection rate and risk level based on the analysis results of the advertising creative and the user's emotion recognition results.

[0507] Input: Parsed advertising data and user emotional state data

[0508] Data processing and calculation: Risk assessment based on risk factors and emotional data

[0509] Output: Evaluation results of NG rate and risk level

[0510] Step 7:

[0511] The server optimizes the content and display method of the advertising creative based on the user's emotional data.

[0512] Input: Emotion recognition results and ad creative

[0513] Data processing and calculation: Determining and adjusting the optimal ad display method

[0514] Output: Optimized ad creative

[0515] Step 8:

[0516] Based on the analysis results, the server generates the reason for the judgment and the risk level in text format and stores this in a database.

[0517] Input: Risk factors, emotional data, risk level

[0518] Data processing and calculation: Generate textual reasons for judgment and risk level

[0519] Output: Text data of the generated judgment reason and risk level

[0520] Step 9:

[0521] The user can check the results of the assessment through the provided interface and submit corrections or feedback as necessary.

[0522] Input: Generated judgment reason and danger level text

[0523] Data processing and calculation: Adding and correcting feedback

[0524] Output: Corrected feedback data

[0525] Step 10:

[0526] The server stores user feedback as re-learning data and uses it to improve the accuracy of the system.

[0527] Input: User feedback data

[0528] Data processing and calculation: Collecting feedback data and adding it to the retraining dataset

[0529] Output: Updated training dataset

[0530] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0531] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0532] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0533] [Second embodiment]

[0534] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0535] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0536] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0537] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0538] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0539] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0540] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0541] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0542] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0543] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0544] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0545] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0546] This invention is a system that detects illegality and fraudulent activities by collecting advertising creatives from the Internet and analyzing their content. This system automatically analyzes the text, banners, videos, etc. of advertising creatives and calculates the NG rate by comparing it with past judgment results. This improves the quality of advertising and prevents the negative impact of illegal advertising on people's lives.

[0547] System configuration

[0548] The system consists of the following main components:

[0549] 1. Ad creative acquisition module:

[0550] The server collects ad creatives from various internet sources, including ad serving platform APIs, RSS feeds, web scraping, etc. The collected data is stored in a database and assigned a unique ID.

[0551] 2. Text Analysis Module:

[0552] The server analyzes the text of the saved ad creative using natural language processing technology, which detects risky keywords and expressions such as "free" and "limited."

[0553] 3. Image Analysis Module:

[0554] The server uses image recognition technology to analyze the content of the advertising banners, extracting icons, text, and color schemes from the images to determine whether they may be illegal.

[0555] 4. Video Analysis Module:

[0556] The server uses video analysis technology to analyze the content of the advertising video frame by frame, and uses voice recognition technology to convert the audio content into text and detect inappropriate messages.

[0557] 5. Risk Assessment Module:

[0558] The server compares past results with the current analysis result and calculates a NG rate, which includes weighting certain factors.

[0559] 6. Reasons for decision generation module:

[0560] The server generates the reason for the judgment and the risk level based on the analysis results and stores them in a database in text format. For example, it outputs a reason such as "The risk is high because it contains specific keywords."

[0561] 7. Feedback Collection Module:

[0562] The terminal provides an interface that allows users (human judges) to check the results of the assessment and input feedback, which is sent to the server and saved as retraining data.

[0563] 8. Priority Judgment Module:

[0564] The server calculates the priority of the ad creative based on the risk and profitability of the analysis results, sorts the ads in order of priority, and displays them on the device.

[0565] 9. Final Report Generation Module:

[0566] The server generates a final review report for the reviewed advertising creatives, which is stored in a database and notified to relevant personnel.

[0567] Specific examples

[0568] For example, if a banner ad contains text like "Buy now" or "Limited edition":

[0569] 1. The server collects these banner ads and stores them in a database.

[0570] 2. The text analysis module analyzes the banner text and detects risky keywords.

[0571] 3. The image analysis module analyzes banner images to check for inappropriate content and design.

[0572] 4. The risk assessment module references past data on similar advertisements and calculates the NG rate.

[0573] 5. The judgment reason generation module generates a reason such as "It is highly risky because it contains specific keywords."

[0574] 6. The feedback collection module accepts the judges' review and feedback.

[0575] 7. The priority determination module displays this advertisement on the list as a high priority.

[0576] 8. The final report generation module generates the final audit report and notifies the relevant personnel.

[0577] In this way, the system improves the quality of online advertising by quickly assessing the risks of advertising creatives and encouraging appropriate responses.

[0578] The processing flow will be explained below.

[0579] Step 1: Collect ad creatives

[0580] Server: Collects advertising creatives (text, banners, videos) from multiple advertising platforms and websites on the Internet. Ad creatives are periodically obtained using API integration and web scraping technology.

[0581] Example: The server retrieves "new ads" through the API of an ad distribution platform and stores them in a database.

[0582] Step 2: Saving to the database

[0583] Server: Collected ad creatives are assigned a unique ID and stored in a database along with metadata (acquisition date, advertiser, ad type, etc.).

[0584] Example: The server saves the image file of the advertisement banner it has acquired in storage, and records the file path and related metadata in a database.

[0585] Step 3: Content analysis

[0586] Server: Performs text analysis, image analysis, and video analysis on ad creatives. Natural language processing (NLP) technology is used to analyze text, OCR technology is used to extract text from images, image recognition is performed, video analysis technology is used to analyze the content of each frame, and speech recognition is used to convert audio content into text.

[0587] Example: The server analyzes images in advertising banners to detect risky keywords such as "special price" and "buy now."

[0588] Step 4: Risk Assessment

[0589] Server: Based on the analysis results, the NG rate of the ad creative is calculated by comparing it with past judgment results. Specific risk factors are weighted to determine the overall risk level.

[0590] Example: Calculate the similarity between ads that have been judged to be "illegal" in the past and current ad creatives, and calculate the NG rate.

[0591] Step 5: Generate and save the reason for the decision

[0592] Server: Based on the analysis results and risk level of the advertising creative, the reason for the judgment and the risk level are generated in text format and saved in the database.

[0593] Example: The server generates and saves the reason for the judgment in text format, such as "This ad contains the keyword 'free' and therefore has a high risk of fraud."

[0594] Step 6: Judges gather feedback

[0595] Terminal: Provides an interface where human judges can review the results and make corrections or provide feedback as needed. Judges review the analysis results of the ad creative and the reasons for their decisions, and evaluate their accuracy.

[0596] Example: A judge checks the evaluation results of an advertising banner on a terminal and enters feedback such as "This advertisement is appropriate."

[0597] Step 7: Save the feedback and use it for retraining

[0598] Server: The acquired feedback information is saved as re-learning data. By re-learning the AI ​​model, the accuracy of judgments will be improved from the next time onwards.

[0599] Example: The server adds feedback data from judges to the database and incorporates it into the retraining dataset.

[0600] Step 8: Determine and list priorities

[0601] Server: Calculates the response priority of each ad creative based on risk and profitability, and displays a list of ad creatives in order of priority through the interface.

[0602] Example: The server sets high profitability and high risk advertisements as priority 1 and displays them in a prominent position on the device.

[0603] Step 9: Generate the final report

[0604] Server: Generates a final review report for the ad creatives that have been reviewed, stores it in a database, and notifies the relevant personnel.

[0605] Example: The server generates a final report for the advertisement "New Product Launch" and notifies the person in charge, including results such as "No problems" and "Appropriate wording."

[0606] These are the processing steps of the system, which allows for efficient advertising screening and improves the reliability of media.

[0607] Example 1

[0608] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0609] Some online advertisements contain illegal or fraudulent content, which can have a negative impact on users. In particular, it is difficult to properly evaluate and monitor advertisements in a consistent manner across a wide range of formats (text, images, and videos). Furthermore, there is a lack of a system for effectively feeding back the results of human judges' judgments and using them as retraining data. To address these issues, it is necessary to automate and streamline the entire process, from analyzing advertising creatives to assessing risk, prioritizing, and generating a final report.

[0610] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0611] In this invention, the server includes: means for collecting advertising creatives from the Internet; means for storing the collected advertising creatives in a database; means for analyzing the text, images, and video of the advertising creatives to extract risky elements; means for calculating a rejection rate by comparing past judgment results with current analysis results; means for analyzing the text portion of the advertising creatives using natural language processing technology to detect risky keywords; means for analyzing the content of advertising banners using image recognition technology to determine potential illegality; means for analyzing the content of advertising videos frame by frame using video analysis technology to detect inappropriate messages; and means for generating and storing the judgment reasons and risk levels in a database. This improves the quality of advertising on the Internet and minimizes adverse effects on users. The server also includes means for providing an interface for human reviewers to review the judgment results and make corrections or provide feedback; means for saving the acquired feedback information as re-learning data; means for calculating the response priority of advertising creatives based on risk and profitability; and means for providing an interface for displaying advertising creatives in order of priority, thereby automating and streamlining this entire process.

[0612] "Internet advertising creative" refers to the overall content of advertisements distributed over the Internet, including text, images, and videos.

[0613] "Means of collection" refers to programs and algorithms used to obtain data from the internet, including methods such as APIs, RSS feeds, and web scraping.

[0614] "Means of storing data in a database" refers to systems and algorithms for efficiently storing and managing collected data, including platforms for storing data in a particular format.

[0615] "Means of analysis" refers to technologies and programs that use machine learning and AI technology to evaluate and analyze the text, images, and videos of advertising creatives.

[0616] "Risk elements" refer to specific text, image, or video elements that pose concerns about illegality or fraud, such as specific keywords, icons, or colors.

[0617] The "NG rate" is an indicator that shows how risky an ad creative is compared to past judgment results, and is often expressed as a number or percentage.

[0618] "Natural language processing technology" refers to artificial intelligence technology for analyzing meaning and structure from text data, and specifically includes the use of machine learning algorithms and APIs.

[0619] "Image recognition technology" refers to the technology of analyzing image data to extract and recognize specific features, and computer vision technology and deep learning algorithms are commonly used.

[0620] "Video analysis technology" refers to the technology of analyzing video data frame by frame to extract and evaluate specific content, and includes video processing software and voice recognition technology.

[0621] "Reasons for judgment and risk level" refers to the analytical reasons generated as a result of evaluating the advertising creative and indicators that quantitatively and qualitatively indicate the level of risk.

[0622] "Feedback interface" refers to a user interface that allows human reviewers to review the analysis results and enter corrections and feedback.

[0623] "Retraining data" refers to data used to improve the performance of an AI model based on feedback information, and is used through a continuous learning process.

[0624] The "means for calculating priority" refers to a program for using an algorithm to calculate the priority of a response based on the risk and profitability of an advertising creative.

[0625] "Listing interface" refers to a user interface that displays advertising creatives in order of priority, allowing users to efficiently respond.

[0626] "Final Report" refers to a comprehensive report containing analysis results and feedback on the advertising creatives that have been reviewed, and is a document stored in a database.

[0627] "Means of notification" refers to the system or method for notifying relevant personnel of the generated final report, including email and notification systems.

[0628] MODE FOR CARRYING OUT THE INVENTION

[0629] The present invention is a system for detecting illegal or fraudulent activities by collecting and analyzing advertising creatives on the Internet. Specific embodiments of this system will be described below.

[0630] System configuration

[0631] The system mainly consists of three elements: a server, a terminal, and a user. The server collects, analyzes, and evaluates data, and the terminal provides a user interface. The user checks the information provided by the system and provides feedback as needed.

[0632] Hardware and software used

[0633] 1. The server uses the following technologies to collect, analyze, and evaluate advertising creatives:

[0634] Use APIs and RSS feeds to gather advertising creatives from across the internet. Examples include APIs of ad serving platforms, RSS feeds, and web scraping.

[0635] Natural language processing technology (e.g., Google Cloud Natural Language API) is used to analyze the text and detect risky keywords such as "free" and "limited."

[0636] Image recognition technology (e.g., OpenCV, Google Cloud Vision API) is used to analyze the content of advertising banners and determine whether they may be illegal.

[0637] Video analysis technology (e.g., FFmpeg, Google Cloud Video Intelligence API) is used to analyze the video content frame by frame, and speech recognition technology is used to convert the audio content into text and detect inappropriate messages.

[0638] The database will be used to store collected advertising creatives, analysis results, reasons for judgments, etc.

[0639] 2. The device provides an interface for the user to review the analysis results and provide feedback. Specific elements include:

[0640] The user interface visually displays the analysis results, allowing the user to easily check the judgment results.

[0641] The feedback function provides an interface where the user can enter corrections and feedback on the results.

[0642] 3. The user acts as a judge, reviewing the information provided by the system and providing feedback if necessary.

[0643] Specific examples

[0644] For example, if a banner ad contains text such as "Buy now" or "Limited edition," the system's processing flow is as follows:

[0645] 1. The server collects these banner ads and stores them in a database.

[0646] 2. The server uses a text analysis module to analyze the banner text and detect risky keywords (e.g., "Buy now" or "Limited Edition").

[0647] 3. The server uses an image analysis module to analyze the banner image and check for inappropriate content and design.

[0648] 4. The server uses the risk assessment module to refer to past similar ad data and calculate the NG rate.

[0649] 5. The server uses the judgment reason generation module to generate a reason such as "It is highly risky because it contains specific keywords."

[0650] 6. The device accepts the judges' confirmation and feedback through the feedback collection module.

[0651] 7. The server uses the priority determination module to display this advertisement on the list as a high priority.

[0652] 8. The server uses the final report generation module to generate the final review report and notify the relevant personnel.

[0653] Prompt Sentence Examples

[0654] Here is an example of a prompt to input to the generative AI model:

[0655] Please explain the system that collects and analyzes advertising creatives containing keywords such as "buy now" and "limited edition" to assess their potential for illegality.

[0656] In this way, the system improves the quality of online advertising by quickly assessing the risks of advertising creatives and encouraging appropriate responses.

[0657] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0658] Step 1: Collect ad creatives

[0659] Input: Ad serving platform API, RSS feed, website URL

[0660] How it works: The server collects ad creatives from specified internet sources. The server uses the ad serving platform's API to get the latest ad lists, extracts newly published ads from RSS feeds, and uses web scraping to collect ad banners from specific web pages.

[0661] Output: Collected ad creative data

[0662] Step 2: Saving to the database

[0663] Input: Collected ad creative data

[0664] Specific operation: The server stores the collected ad creatives in a database, assigns a unique ID, and stores metadata for each ad creative (e.g., collection source, date, format, etc.).

[0665] Output: Ad creatives and their metadata stored in a database

[0666] Step 3: Text analysis

[0667] Input: Ad creative text data loaded from the database

[0668] Specific operation: The server analyzes the text using natural language processing technology (e.g., Google Cloud Natural Language API). It detects risky keywords and expressions such as "free" and "limited" and marks them.

[0669] Output: Parsed text data with risk keywords marked

[0670] Step 4: Image analysis

[0671] Input: Ad creative image data loaded from database

[0672] How it works: The server analyzes the content of the ad banner using image recognition technology (e.g., OpenCV, Google Cloud Vision API), extracting icons, text, and colors from the image and determining whether they match indicators of illegality.

[0673] Output: Analyzed image data and a list of potentially illegal elements

[0674] Step 5: Video Analysis

[0675] Input: Video data of the ad creative loaded from the database

[0676] Specific operation: The server analyzes the content of the advertisement video frame by frame using video analysis technology (e.g., FFmpeg, Google Cloud Video Intelligence API), converts the audio content into text using speech recognition technology, and detects inappropriate messages.

[0677] Output: Analyzed frame data and text data based on speech recognition

[0678] Step 6: Risk Assessment

[0679] Input: Analyzed text data, image data, video data

[0680] Specific operation: The server compares past judgment results with the current analysis results to calculate the NG rate. It weights each specific element and evaluates the overall risk level.

[0681] Output: Risk assessment results including NG rate

[0682] Step 7: Generate reasons for the decision

[0683] Input: Risk assessment results

[0684] Specific operation: The server generates the reason for the judgment and the risk level based on the analysis results and saves them in a database in text format. The reason for the judgment is automatically generated and the specific reason is output in text format.

[0685] Output: Reason for judgment and risk level stored in the database

[0686] Step 8: Gather feedback

[0687] Input: Reason for judgment and risk level stored in the database

[0688] Specific operation: The terminal provides an interface that allows the user (judge) to check the judging results and add corrections or feedback. The user enters feedback, and the terminal sends it to the server.

[0689] Output: Feedback data sent to the server

[0690] Step 9: Prioritization

[0691] Input: Risk assessment results, feedback data

[0692] Specific operation: The server calculates the response priority of the ad creative based on the risk and profitability obtained from the analysis results, sorts the ads in order of priority, and displays them on the device.

[0693] Output: A list of ad creatives sorted by priority

[0694] Step 10: Generate the final report

[0695] Input: Final evaluation and feedback results

[0696] Specific operations: The server generates a final review report for the advertising creatives that have been reviewed, stores it in the database, and notifies the relevant personnel.

[0697] Output: Final report stored in database and notification sent to relevant parties through notification system

[0698] In this way, the processes at each step are organically linked to provide a system for assessing and managing the risks of advertising creatives.

[0699] (Application example 1)

[0700] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0701] The Internet is flooded with a wide variety of advertisements, many of which are illegal or involve fraudulent activities. This puts users at high risk of coming into contact with inappropriate advertisements, which could potentially impair their quality of life. Manually checking advertisement content requires a huge amount of time and effort, making it inefficient. Furthermore, there is an inadequate system for detecting advertising risks in real time and notifying users, making it difficult to prevent harm caused by advertisements.

[0702] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0703] In this invention, the server includes means for collecting advertising creatives from the Internet, means for saving the collected advertising creatives in a database, means for analyzing the text, images, and videos of the advertising creatives and extracting risky elements, means for comparing past judgment results with current analysis results to calculate an NG rate, means for generating judgment reasons and risk levels in text format and saving them in a database, means for scanning and analyzing the collected advertisements in real time while browsing, and means for displaying a warning to the user based on the analysis results. This makes it possible to automatically detect illegal or fraudulent advertising and notify the user in real time.

[0704] "Internet advertising creative" refers to advertising materials such as text, images, and video that are distributed via the Internet, such as websites and applications.

[0705] "Means of collection" refers to functions and modules for detecting advertising creatives on the Internet and capturing and storing them as data.

[0706] "Database" means an information management system that enables efficient storage, management, and access of collected advertising creatives and analysis results.

[0707] "Means of analysis" refers to algorithms or programs that examine the content (text, images, video) of advertising creatives and detect risky elements.

[0708] "Risk elements" refer to keywords, images, audio, etc. contained in the content of advertising creatives that may raise suspicions of illegality or fraud.

[0709] The "judgment result" is the evaluation or judgment derived from the analysis of the advertising creative.

[0710] The "NG rate" is a number that indicates the probability that illegal or fraudulent activity will be detected in the analyzed advertising creative.

[0711] "Text format" refers to a format in which analysis results and reasons for judgments are expressed as text information.

[0712] "Real-time scanning while browsing" refers to the ability to instantly detect and analyze advertisements displayed while a user is browsing a website.

[0713] "Means to display a warning to users" refers to a function that notifies users of risky advertising creatives when they are detected, and provides a warning or caution.

[0714] The present invention relates to a system that analyzes advertising creatives on the Internet in real time to detect illegal or fraudulent activities. This system has the function of analyzing the text, images, and videos of advertisements, evaluating the risk level, and displaying a warning to the user. The following describes in detail the embodiments of the present invention.

[0715] System configuration

[0716] Hardware and software used

[0717] 1. Hardware

[0718] Smartphones: Used for real-time analysis when users view ads, using built-in sensors such as the camera, display, and microphone.

[0719] Server: Responsible for major back-end processing such as collecting ad creatives, analyzing, and storing data.

[0720] 2. Software

[0721] Python: A programming language used to implement the analysis program.

[0722] BeautifulSoup: A web scraping library used to scrape ad creatives from the web.

[0723] OpenCV: An image processing library used to analyze advertising images and videos.

[0724] Pytesseract: An OCR library used for analyzing text in images.

[0725] TensorFlow: A machine learning framework used for natural language processing, including text analysis and speech recognition.

[0726] System Operation

[0727] 1. Collecting advertising creatives

[0728] The server collects ad creatives from designated internet sources (websites, APIs, etc.), stores the collected data in a database, and then analyzes it. For example, it retrieves ads from news sites and social media platforms.

[0729] 2. Analysis of advertising creatives

[0730] The server analyzes the text, images, and videos of the stored ad creatives. Text analysis uses natural language processing technology to detect risky keywords and expressions. Image analysis examines the text, icons, and color usage within images to determine the possibility of fraud. Video analysis analyzes each frame of video, converting audio into text to detect inappropriate content.

[0731] 3. Risk assessment and warning signs

[0732] The server compares past judgment results with the analysis results to calculate the NG rate. The analysis results are used to generate the reason for the judgment and the risk level, which are then saved in a text database. When a user views an advertisement in real time while browsing, the advertisement is scanned, the risk level is assessed based on the analysis results, and a warning is displayed if necessary.

[0733] 4. User Interface and Feedback Collection

[0734] The system provides an interface for users to respond to warning messages and collects user feedback. This feedback is sent to the server and saved as retraining data. Once the review is complete, a detailed review report is generated and notified to the relevant personnel.

[0735] Specific examples

[0736] As a concrete example, consider the case where analysis is performed using the following prompt sentence:

[0737] Analyze the ad text below to see if it contains any risky keywords.

[0738] Text: "Buy now, limited time offer, 100% free"

[0739] In this way, the system can efficiently detect illegal or fraudulent advertising creatives and provide users with real-time warnings, thereby improving the quality of advertising on the Internet and ensuring user safety.

[0740] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0741] Step 1:

[0742] The server collects ad creatives from specified internet sources (websites, APIs, etc.). For example, it detects ad containers from news sites or social media platforms and retrieves their contents in HTML format. The input is the website URL or API endpoint, and the output is the ad creative data.

[0743] Step 2:

[0744] The server stores the collected ad creatives in a database. The collected ad creatives are recorded in the database with a unique ID for subsequent analysis. The input is the ad creative data collected in step 1, and the output is a new record added to the database.

[0745] Step 3:

[0746] The server uses natural language processing technology to analyze the text portion of the ad creative stored in the database. Specifically, it extracts risky keywords (e.g., "free," "now," "limited," etc.) from the text data. The input is the text data of the ad creative, and the output is a list of risky keywords and expressions.

[0747] Step 4:

[0748] The server uses image recognition technology to analyze the content of the ad banner. It uses OCR technology to detect text, icons, color usage, etc. in the image and extracts risky elements. The input is the image data of the ad banner, and the output is the analyzed text and risky elements.

[0749] Step 5:

[0750] The server uses video analysis technology to analyze the content of the ad video. It analyzes the text and audio for each frame of the video to detect inappropriate content and expressions. The input is the ad video data, and the output is the extracted text and audio content, as well as risky elements.

[0751] Step 6:

[0752] The server compares past judgment results with the current analysis results to calculate the NG rate. Based on the analysis results for each element of the ad creative (text, images, video), the NG rate is calculated while weighting specific risk elements. The input at this time is the analysis result data, and the NG rate is obtained as the output.

[0753] Step 7:

[0754] The server generates the reason for the judgment and the risk level in text format and saves them in a database. Based on the analysis results, it generates a reason for the judgment, such as "It is highly risky because it contains specific keywords." The input at this time is the analysis result and NG rate, and the reason for the judgment is obtained in text format as an output.

[0755] Step 8:

[0756] The device scans and analyzes the ads displayed while the user is browsing in real time. The collected ads are analyzed on the server and a warning is displayed to the user if necessary. The input is the ad data displayed in real time, and the output is a warning about risky ads.

[0757] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0758] The advertising review system of the present invention analyzes the content of advertising creatives (text, banners, videos) collected from the Internet to detect illegality and fraudulent activities. Furthermore, by incorporating an emotion engine that recognizes user emotions, the present invention performs more advanced risk assessment of advertising and realizes optimization of display methods.

[0759] System configuration

[0760] The system consists of the following main components:

[0761] 1. Ad creative acquisition module:

[0762] The server collects advertising creatives (text, banners, videos) from various internet sources using the APIs of ad serving platforms and web scraping technology.

[0763] Example: A server uses the API of an ad distribution platform to periodically collect the latest ad banners and store them in a database.

[0764] 2. Text Analysis Module:

[0765] The server uses natural language processing technology to analyze the text content of the collected advertising creatives and identify inappropriate language and risky keywords.

[0766] Example: The server uses OCR technology to extract text from banner images and detect keywords such as "Buy now" and "Limited Edition."

[0767] 3. Image Analysis Module:

[0768] The server uses image recognition technology to analyze the visual content of advertising banners, identifying those promoting counterfeit products or false health information.

[0769] Example: The server analyzes the icons and colors in an advertising banner and determines that it may contain false information.

[0770] 4. Video Analysis Module:

[0771] The server uses video analysis technology to analyze each frame of the advertising video and uses voice recognition technology to convert the audio content into text.

[0772] Example: The server extracts and analyzes messages such as "Great deals" from the audio in the video.

[0773] 5. Emotion Engine:

[0774] The server is equipped with an emotion engine that recognizes the user's emotions in real time, detecting their emotional state (happiness, surprise, anger, sadness, etc.) through facial recognition and voice analysis.

[0775] Example: The server analyzes the user's camera footage and microphone input using an emotion engine to determine the emotion the user is expressing while watching an advertisement.

[0776] 6. Risk Assessment Module:

[0777] The server calculates the NG rate and risk level based on the analysis of the ad creative and the user's emotion recognition. For example, if the user expresses negative emotions toward the ad, it will determine that the risk is high.

[0778] Example: The server reflects the fact that "the user showed anger while watching an advertisement" in the risk assessment.

[0779] 7. Reasons for decision generation module:

[0780] The server generates a reason for the judgment based on the risk assessment result and stores it in a database. The reason for the judgment is generated in text format.

[0781] Example: The server generates and stores a reason for the judgment, such as "There is a high risk of fraud because it contains certain keywords."

[0782] 8. Feedback Collection Module:

[0783] The terminal provides an interface that allows users (human judges) to review the results and provide corrections or feedback as needed.

[0784] Example: A judge adds feedback to the evaluation results, such as "This advertisement is not illegal."

[0785] 9. Priority Judgment Module:

[0786] The server determines the priority of the advertising creatives taking into consideration both the risk and profitability, and displays them on the terminal in order of priority.

[0787] Example: The server determines that the advertisements that are both highly profitable and highly risky are given priority 1 and are displayed on the device.

[0788] 10. Final Report Generation Module:

[0789] The server prepares a final review report for the advertising creatives for which review has been completed, and notifies the relevant personnel.

[0790] Example: The server generates a final report for the advertisement "New Product Launch" and notifies the person in charge, including results such as "No problems" and "Appropriate wording."

[0791] This system effectively detects illegal and fraudulent advertising creatives and recognizes user sentiment in real time, thereby improving advertising quality and optimizing the user experience.

[0792] The processing flow will be explained below.

[0793] Step 1: Collect ad creatives

[0794] The server collects advertising creatives (text, banners, videos) from multiple ad distribution platforms and websites on the Internet, periodically retrieving advertisements using API integration and web scraping technology, and storing them in a database.

[0795] Step 2: Saving to the database

[0796] The server stores the collected ad creatives in a database with a unique ID, along with metadata (date and time of acquisition, advertiser, ad type, etc.).

[0797] Step 3: Content analysis

[0798] The server performs the following analysis on the ad creative:

[0799] Text analysis: Analyzes text content using natural language processing (NLP) techniques to detect risky keywords.

[0800] Image analysis: Using image recognition technology to analyze the visual content of advertising banners to identify inappropriate elements or fraudulent products.

[0801] Video analysis: Using video analysis technology, each frame of the advertising video is analyzed and the audio content is converted into text through speech recognition.

[0802] Step 4: Emotion Recognition

[0803] The server analyzes the user's camera footage and microphone input in real time using an emotion engine that recognizes the user's emotions, detecting the emotions (happiness, surprise, anger, sadness, etc.) that the user shows while watching the advertisement.

[0804] Step 5: Risk Assessment

[0805] The server integrates the results of the ad creative analysis and emotion recognition, and calculates the NG rate by comparing it with past judgment results. It also weights specific risk factors and determines the overall risk level.

[0806] Step 6: Generate and save the reason for the decision

[0807] The server generates a reason for the judgment in text format based on the risk assessment result and stores it in the database. The reason for the judgment includes the specific elements that were judged to be high risk.

[0808] Step 7: Collect and store feedback

[0809] The terminal provides an interface for human judges to review the results and provide corrections or feedback as necessary. The judges' feedback is sent to the server and saved as re-learning data.

[0810] Step 8: Determine and list priorities

[0811] The server calculates the response priority of the ad creatives based on the analysis results and profitability information, and displays them on the device in order of priority, allowing high-risk ads to be quickly addressed.

[0812] Step 9: Generate and communicate final report

[0813] The server generates a final review report for the reviewed advertising creatives and stores it in a database, and notifies the relevant personnel of the report.

[0814] Specific examples

[0815] Collection phase: The server retrieves new banner ads from the ad distribution platform via API and stores the collected data in a database.

[0816] Analysis phase: The server uses natural language processing technology to analyze the text of the advertising banner, detecting risky keywords such as "only now" and "limited edition." It also analyzes the image content to assess the possibility that it contains fraudulent products.

[0817] Emotion recognition phase: When a user watches an advertisement, the server uses an emotion engine to analyze the user's facial expressions and detect the emotions shown by the user in real time.

[0818] Evaluation phase: The server integrates the analysis results and emotion recognition results, compares them with past judgment results, calculates the NG rate, and generates the reason for the judgment.

[0819] Feedback phase: The judges check the reasons for the judgement on the terminal and input their feedback on whether the judgement was appropriate. This feedback is sent to the server and used for the next analysis.

[0820] Priority calculation phase: The server calculates the priority based on the profitability and risk of the ad creative, and displays it on the device in order of priority.

[0821] Final report phase: The server generates a final review report, stores it in the database, and notifies the person in charge.

[0822] In this way, the system efficiently detects illegal and fraudulent advertising and, by taking into account evaluations based on user emotions, improves advertising quality and optimizes the user experience.

[0823] Example 2

[0824] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0825] There is a wide variety of advertising creatives on the Internet, some of which may contain inappropriate or illegal content. This poses a risk of adversely affecting users, so there is a need for a system that can efficiently and effectively analyze advertising content and detect illegal or inappropriate content. However, conventional systems have problems such as low analysis accuracy and an inability to take user emotions into account.

[0826] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0827] In this invention, the server includes means for collecting advertising creatives from the Internet, means for storing the collected advertising creatives in a database, means for analyzing the text, images, and videos of the advertising creatives and extracting risky elements, means for recognizing user emotions in real time, means for calculating a risk level based on the results of the advertising creative analysis and emotion recognition, and means for generating the reasons for the determination and the risk level in text format and storing them in the database. This makes it possible to analyze the content of advertising creatives with high accuracy, efficiently detect illegal or inappropriate content, and perform risk assessment taking user emotions into consideration.

[0828] The "Internet" is a global digital communications system that connects computer networks around the world.

[0829] "Ad creative" refers to the elements of an advertisement displayed on the Internet, including content in any media format, such as text, banners, or video.

[0830] A "server" is a computer system that provides information and services over a network and processes data in response to requests from clients.

[0831] A "database" is an integrated data storage system for efficiently managing, searching, and updating large amounts of data.

[0832] "Text analysis" is the process of using natural language processing technology to understand the content of text data and extract inappropriate expressions and risky keywords.

[0833] "Image analysis" is the process of using image recognition technology to analyze the content of image data and detect inappropriate elements or false information.

[0834] "Video analysis" is the process of analyzing video data frame by frame and converting the audio content into text using speech recognition technology.

[0835] "Emotion recognition" is a technology that detects a user's emotional state (joy, surprise, anger, sadness, etc.) in real time through facial recognition technology and voice analysis.

[0836] The "risk level" is an evaluation index regarding the inappropriateness or illegality of an advertisement, calculated based on the content of the advertising creative and the results of user emotion recognition.

[0837] The "reason for assessment" is text that describes the specific reason for the risk assessment, generated based on the analysis results of the ad creative.

[0838] "Feedback" is evaluation information provided by judges who review the results of their assessments and add corrections or opinions as necessary.

[0839] "Profitability" is a metric that measures the degree of profit an advertisement is likely to generate.

[0840] "Priority" is an evaluation indicator that indicates the urgency and importance of a response based on the analysis results of advertising creative.

[0841] "Review Report" means a document that describes the final evaluation results for the advertising creatives that have been reviewed and is notified to the relevant personnel.

[0842] This invention is a system that can effectively detect illegal or inappropriate content in advertisements by analyzing the content of advertising creatives (text, banners, videos) collected on the Internet with high accuracy and recognizing user emotions in real time. Furthermore, it aims to perform risk assessment of advertisements and select appropriate display methods.

[0843] Hardware and software used

[0844] Server: The server, which is the core of this system, collects and analyzes advertising creatives and stores data. Its primary hardware is a high-performance computer with large storage capacity. Software includes APIs from ad distribution platforms, web scraping tools, natural language processing engines (e.g., Python's NLTK or spaCy), OCR engines (e.g., Tesseract), image recognition technology (e.g., OpenCV), speech recognition engines (e.g., Google Speech-to-Text API), and emotion recognition engines (e.g., Microsoft Azure's Emotion API).

[0845] Devices: The devices used by judges display the results of their judgement and provide an interface for providing feedback. These devices can be PCs, tablets, or smartphones.

[0846] Users: The users of this system include advertisers, judges, and general advertising users.

[0847] Specific examples of program processing

[0848] 1. Collecting advertising creatives

[0849] The server collects advertising creatives from various sources on the Internet using the API of the ad serving platform or a web scraping tool. For example, the server periodically sends a request to the ad serving platform to obtain the latest advertising banners and stores the collected data in a database.

[0850] 2. Analysis of advertising creatives

[0851] The server analyzes the collected text, images, and videos of the advertising creative using various analytical techniques, such as using an OCR engine to extract text from banner images and a natural language processing engine to detect specific keywords such as "buy now."

[0852] 3. Emotion recognition

[0853] The server uses an emotion recognition engine to recognize the user's emotions in real time. It analyzes the video and audio data acquired through the user's camera and microphone to recognize emotions such as joy, surprise, anger, and sadness. For example, the server detects the emotion "the user is surprised" from the user's microphone input.

[0854] 4. Risk assessment and risk calculation

[0855] The server calculates the risk level of the ad based on the analysis results of the ad creative and the user's emotion recognition. For example, if the user's anger towards the ad is 50% or more, the risk level is set to "high."

[0856] 5. Generating reasons for judgment

[0857] The server generates a reason for the judgment based on the risk assessment result and stores it in the database. The reason for the judgment is detailed in text format. For example, the server may generate a reason for the judgment that "the risk of fraud is high because a specific keyword is included."

[0858] 6. Gathering User Feedback

[0859] The terminal provides an interface for the judge to check the judging results and send feedback if necessary. For example, the judge can use the terminal to input feedback such as "This ad is OK" and send it to the server.

[0860] 7. Priority determination and display

[0861] The server determines the priority of the ad creative based on the risk and profitability, and displays them on the device in order of priority. For example, the server may set and display ads that are both highly profitable and highly risky as priority 1.

[0862] 8. Generate the final report

[0863] The server creates a final review report for the advertising creative that has been reviewed and notifies the relevant person in charge. For example, the server generates a final report for the advertisement "New Product Launch," concludes that there are no problems, and notifies the person in charge.

[0864] Example prompts to input to the generative AI model

[0865] As an example of a prompt statement, you can enter:

[0866] "Analyze the following ad creative to determine if its content is inappropriate and perform a risk assessment taking into account the results of the sentiment engine. Analyze the following text: 'Shop now! Limited offer!'"

[0867] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0868] Step 1: Collect ad creatives

[0869] Specific actions

[0870] The server collects advertising creatives from online ad distribution platforms and websites, using APIs and web scraping technology to obtain data and store it in a database.

[0871] Input: API endpoint of ad serving platform, target URL for web scraping

[0872] Processing: Sending API requests, parsing HTML, extracting data

[0873] Output: Collected ad creative dataset (text, banner, video)

[0874] Step 2: Text analysis

[0875] Specific actions

[0876] The server analyzes the text content of the collected ad creatives using natural language processing technology, extracts text from images using OCR technology, and detects risky keywords.

[0877] Input: Ad creative dataset (text, banner)

[0878] Processing: Text extraction using OCR, keyword detection using natural language processing technology

[0879] Output: Text data tagged with keywords

[0880] Step 3: Image analysis

[0881] Specific actions

[0882] The server uses image recognition technology to analyze the visual content of advertising banners to detect inappropriate products and false health information.

[0883] Input: Ad creative dataset (banner image)

[0884] Processing: Image recognition algorithms are used to analyze and detect inappropriate content.

[0885] Output: Image tag as analysis result (if inappropriate elements are detected)

[0886] Step 4: Video Analysis

[0887] Specific actions

[0888] The server uses video analysis technology to analyze each frame of the ad video, and speech recognition technology to convert the audio content into text and determine whether it contains inappropriate content.

[0889] Input: Ad creative dataset (video files)

[0890] Processing: Frame-by-frame analysis, speech-to-text recognition, and inappropriate content detection

[0891] Output: Texted audio data and analysis report

[0892] Step 5: Emotion Recognition

[0893] Specific actions

[0894] The server utilizes an emotion recognition engine to recognize the user's emotions in real time, using facial recognition technology and voice analysis to detect the emotional state the user is expressing.

[0895] Input: User's camera and microphone input

[0896] Processing: Analysis by emotion recognition engine, identification of emotional state

[0897] Output: Emotion recognition results (happiness, surprise, anger, sadness, etc.)

[0898] Step 6: Risk Assessment

[0899] Specific actions

[0900] The server calculates the risk level based on the analysis results of the ad creative and the user's emotion recognition. For example, if the user expresses negative emotions toward the ad, it will determine the risk level as high.

[0901] Input: text, image, video analysis results, emotion recognition results

[0902] Processing: Data integration and analysis, risk level calculation

[0903] Output: Risk assessment report

[0904] Step 7: Generate reasons for the decision

[0905] Specific actions

[0906] The server generates a reason for the judgment based on the risk assessment result and stores it in the database. The reason for the judgment is detailed in text format.

[0907] Input: Hazard Assessment Report

[0908] Processing: Generating and formatting the reason for the decision

[0909] Output: Reason for decision in text format

[0910] Step 8: Gather feedback

[0911] Specific actions

[0912] The terminal provides an interface for judges to review the results and provide feedback if necessary.

[0913] Input: Judge feedback information

[0914] Process: Enter and send feedback information

[0915] Output: Corrected evaluation data

[0916] Step 9: Determine and display priorities

[0917] Specific actions

[0918] The server determines the priority of the advertising creatives based on the risk and profitability, and displays them on the terminal in order of priority.

[0919] Inputs: Risk assessment report, profitability data

[0920] Processing: Evaluating and determining priority, generating display data

[0921] Output: A list of ad creatives ordered by priority

[0922] Step 10: Generate the final report

[0923] Specific actions

[0924] The server prepares a final review report for the advertising creatives for which review has been completed, and notifies the relevant personnel.

[0925] Input: All analysis results and feedback information

[0926] Processing: Final report generation, formatting, and notification

[0927] Output: Final review report

[0928] (Application example 2)

[0929] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0930] There is a wide variety of advertising creatives on the Internet, many of which are illegal or fraudulent. Therefore, there is a need for a system that can automatically and effectively analyze advertising content and detect illegal or fraudulent activities. It is also necessary to evaluate the emotional impact of advertising on users and improve advertising quality. Furthermore, it is important to perform these evaluations in real time and improve advertising based on appropriate feedback.

[0931] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0932] In this invention, the server includes means for collecting advertising creatives from the Internet, means for saving the collected advertising creatives in a database, means for analyzing the text, images, and videos of the advertising creatives and extracting risky elements, means for comparing past judgment results with current analysis results to calculate an NG rate, means for evaluating the risk of displaying an advertisement using an emotion engine that recognizes the user's emotional state in real time, means for optimizing the advertising creatives based on the user's emotion recognition results, and means for generating judgment reasons and risk levels in text format and saving them in a database. This makes it possible to automatically detect illegality and fraudulent behavior in advertising creatives in real time and optimize advertising display taking user emotions into consideration.

[0933] "Internet advertising creative" means the content, including text, images, or video, of an advertisement displayed over the Internet.

[0934] "Means of collection" refers to the technology or method used to automatically collect advertising creatives published on the Internet and store them in a database.

[0935] "Means for storing in a database" refers to the technology or method for storing the collected advertising creative content in a digital format and managing it so that it can be subsequently analyzed and searched.

[0936] "Means of analysis and extraction of risky elements" refers to technologies and methods that use natural language processing, image recognition, voice analysis, etc. to analyze the content of advertising creatives and detect elements that may be illegal or fraudulent.

[0937] "Means for calculating NG rate" refers to the technology or method for comparing past judgment results with current analysis results and calculating the percentage of inappropriate advertisements.

[0938] The "emotion engine that recognizes the user's emotional state in real time" is a system that uses facial recognition and voice analysis technology to detect the emotions (joy, surprise, anger, sadness, etc.) of users viewing advertisements in real time.

[0939] "Means for assessing the risk of displaying advertisements" refers to technologies and methods for assessing the risk level of advertising creatives based on the results of user emotion recognition and taking appropriate measures.

[0940] "Means for optimizing advertising creatives based on user emotion recognition results" refers to technologies and methods that adjust the content and display method of advertisements based on user emotion data, thereby improving the user experience.

[0941] "Means for generating the reasons for judgment and risk levels in text format and storing them in a database" refers to the technology and methods for generating the analysis results of advertising creatives as text and storing them in a database.

[0942] "An interface where human judges can review the results of their assessments and make corrections or provide feedback" refers to a user interface that allows human judges to review the analysis results of advertisements on a screen and make corrections or comments as necessary.

[0943] "Means for saving as re-learning data" refers to the technology and methods for collecting and saving feedback information as data for machine learning and using it to improve the system.

[0944] The "means for calculating the response priority of advertising creatives based on risk level and profitability" refers to a technique or method for determining the response priority in consideration of the risk level and profitability of advertising creatives.

[0945] The "interface for displaying a list of advertising creatives in order of priority" refers to a user interface for displaying a list of advertising creatives on a screen according to their corresponding priorities.

[0946] "Means for auditing the legality of advertising creatives in real time using smartphones" refers to technologies and methods that use smartphone functions to analyze and audit the illegality of displayed advertising creatives in real time.

[0947] "Means for generating an examination report" refers to a technology or method for creating a final analysis result for an advertising creative in the form of a report.

[0948] "Means for storing the audit report in a database and notifying relevant personnel" refers to the technology or method for storing the generated audit report in a database and notifying the personnel in charge of the contents of the report.

[0949] "Means for creating an evaluation report based on the results of user emotion recognition" refers to a technology or method for creating an evaluation report that includes an evaluation of the advertising creative and points for improvement, taking into account user emotion data.

[0950] A specific method for implementing the advertisement quality audit application according to the present invention will be described. In this embodiment, an application using a smartphone as a platform is exemplified, but the application can be implemented in a similar manner on other devices.

[0951] System configuration

[0952] 1. How to obtain advertising creatives

[0953] The server automatically collects advertising creatives published on the Internet using the API of the advertising distribution platform. For example, it periodically retrieves the latest advertising banners and videos and stores them in a database.

[0954] 2. Database storage method

[0955] The server stores the collected advertising creatives in a digital database, which is structured to facilitate subsequent analysis and retrieval.

[0956] 3. Text Analysis Methods

[0957] The server analyzes the text of the ad creative using natural language processing techniques, for example, OCR technology to extract text from ad images and detect marketing keywords such as "buy now."

[0958] 4. Image and video analysis methods

[0959] The server uses image recognition and video analysis technology to analyze the content of the ad creative in real time, checking the content and audio data of each frame to identify counterfeit products and false information.

[0960] 5. Use of Emotion Engine

[0961] The server uses the smartphone's camera and microphone to recognize the user's emotions in real time, using facial recognition and voice analysis technology to analyze the emotional state (happiness, surprise, anger, sadness, etc.) displayed by the user while watching the advertisement.

[0962] 6. Advertisement display risk assessment method

[0963] The server evaluates the NG rate and risk level based on the results of ad analysis and user emotion recognition. For example, if a user expresses anger or discomfort while watching an ad, it will be deemed high risk.

[0964] 7. Ad creative optimization methods

[0965] The server adjusts the content and display method of the advertisement based on the user's emotional data to improve the user experience.

[0966] 8. Judgment reason generation / storage means

[0967] The server generates a reason for the judgment and a risk level in text format based on the analysis results and stores this in a database. For example, it may generate a judgment reason such as "There is a high risk of fraud because a specific keyword is included."

[0968] 9. Feedback Collection Methods

[0969] Users can review the results through a provided interface and provide corrections and feedback if necessary, allowing the system to continuously learn and improve its accuracy.

[0970] Examples of examples and prompts to use

[0971] For example, if an ad creative contains the text "Buy Now" and the user expresses anger, the ad will be judged to be highly likely to be fraudulent. Feedback provided by users regarding the ad will also be saved as learning data for the system and reflected in future analyses.

[0972] Prompt Sentence Examples

[0973] We need an application that can detect illegal and fraudulent advertising creatives and analyze user sentiment in real time. This includes the following features:

[0974] 1. Collect advertising creatives from the internet

[0975] 2. Illegal detection through text analysis

[0976] 3. Image and video analysis

[0977] 4. Facial Recognition and Emotion Analysis

[0978] 5. Generate a report of the results

[0979] Based on this requirement, generate the best model.

[0980] The present invention effectively detects advertisements that contain illegal or fraudulent activities and optimizes advertisement display taking into account user emotions, thereby enabling improvements in advertisement quality and user experience.

[0981] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0982] Step 1:

[0983] The server automatically collects advertising creatives (images, videos, text) from the Internet through the advertising distribution platform's API.

[0984] Input: Ad serving platform API

[0985] Data processing and calculation: Obtaining advertising data from the internet and converting it into digital format

[0986] Output: Collected ad creatives

[0987] Step 2:

[0988] The server stores the collected advertising creatives in a database.

[0989] Input: Collected ad creatives

[0990] Data processing and calculation: Store advertising data in the appropriate tables in the database

[0991] Output: Ad creatives stored in a database

[0992] Step 3:

[0993] The server uses OCR technology to extract text from the advertising image and analyzes the text using natural language processing (NLP) technology.

[0994] Input: Ad creative image data stored in a database

[0995] Data processing and calculation: Extract text using OCR and analyze keywords using NLP

[0996] Output: Parsed text and its risk factors

[0997] Step 4:

[0998] The server uses image recognition and video analysis technology to analyze the ad creative and identify elements that may contain counterfeit products or false information.

[0999] Input: Image and video data of advertising creatives stored in the database

[1000] Data processing and calculation: Risk element extraction through image recognition and video frame analysis

[1001] Output: Analyzed image and video data and their risk factors

[1002] Step 5:

[1003] The server uses the smartphone's camera and microphone to collect data to recognize the user's emotions in real time.

[1004] Input: Camera image and microphone audio from the user's smartphone

[1005] Data processing and calculation: Emotion recognition using facial recognition and voice analysis technology

[1006] Output: User's emotional state data

[1007] Step 6:

[1008] The server evaluates the rejection rate and risk level based on the analysis results of the advertising creative and the user's emotion recognition results.

[1009] Input: Parsed advertising data and user emotional state data

[1010] Data processing and calculation: Risk assessment based on risk factors and emotional data

[1011] Output: Evaluation results of NG rate and risk level

[1012] Step 7:

[1013] The server optimizes the content and display method of the advertising creative based on the user's emotional data.

[1014] Input: Emotion recognition results and ad creative

[1015] Data processing and calculation: Determining and adjusting the optimal ad display method

[1016] Output: Optimized ad creative

[1017] Step 8:

[1018] Based on the analysis results, the server generates the reason for the judgment and the risk level in text format and stores this in a database.

[1019] Input: Risk factors, emotional data, risk level

[1020] Data processing and calculation: Generate textual reasons for judgment and risk level

[1021] Output: Text data of the generated judgment reason and risk level

[1022] Step 9:

[1023] The user can check the results of the assessment through the provided interface and submit corrections or feedback as necessary.

[1024] Input: Generated judgment reason and danger level text

[1025] Data processing and calculation: Adding and correcting feedback

[1026] Output: Corrected feedback data

[1027] Step 10:

[1028] The server stores user feedback as re-learning data and uses it to improve the accuracy of the system.

[1029] Input: User feedback data

[1030] Data processing and calculation: Collecting feedback data and adding it to the retraining dataset

[1031] Output: Updated training dataset

[1032] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1033] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1034] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1035] [Third embodiment]

[1036] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1037] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1038] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1039] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1040] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1041] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1042] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1043] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1044] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1045] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1046] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1047] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1048] This invention is a system that detects illegality and fraudulent activities by collecting advertising creatives from the Internet and analyzing their content. This system automatically analyzes the text, banners, videos, etc. of advertising creatives and calculates the NG rate by comparing it with past judgment results. This improves the quality of advertising and prevents the negative impact of illegal advertising on people's lives.

[1049] System configuration

[1050] The system consists of the following main components:

[1051] 1. Ad creative acquisition module:

[1052] The server collects ad creatives from various internet sources, including ad serving platform APIs, RSS feeds, web scraping, etc. The collected data is stored in a database and assigned a unique ID.

[1053] 2. Text Analysis Module:

[1054] The server analyzes the text of the saved ad creative using natural language processing technology, which detects risky keywords and expressions such as "free" and "limited."

[1055] 3. Image Analysis Module:

[1056] The server uses image recognition technology to analyze the content of the advertising banners, extracting icons, text, and color schemes from the images to determine whether they may be illegal.

[1057] 4. Video Analysis Module:

[1058] The server uses video analysis technology to analyze the content of the advertising video frame by frame, and uses voice recognition technology to convert the audio content into text and detect inappropriate messages.

[1059] 5. Risk Assessment Module:

[1060] The server compares past results with the current analysis result and calculates a NG rate, which includes weighting certain factors.

[1061] 6. Reasons for decision generation module:

[1062] The server generates the reason for the judgment and the risk level based on the analysis results and stores them in a database in text format. For example, it outputs a reason such as "The risk is high because it contains specific keywords."

[1063] 7. Feedback Collection Module:

[1064] The terminal provides an interface that allows users (human judges) to check the results of the assessment and input feedback, which is sent to the server and saved as retraining data.

[1065] 8. Priority Judgment Module:

[1066] The server calculates the priority of the ad creative based on the risk and profitability of the analysis results, sorts the ads in order of priority, and displays them on the device.

[1067] 9. Final Report Generation Module:

[1068] The server generates a final review report for the reviewed advertising creatives, which is stored in a database and notified to relevant personnel.

[1069] Specific examples

[1070] For example, if a banner ad contains text like "Buy now" or "Limited edition":

[1071] 1. The server collects these banner ads and stores them in a database.

[1072] 2. The text analysis module analyzes the banner text and detects risky keywords.

[1073] 3. The image analysis module analyzes banner images to check for inappropriate content and design.

[1074] 4. The risk assessment module references past data on similar advertisements and calculates the NG rate.

[1075] 5. The judgment reason generation module generates a reason such as "It is highly risky because it contains specific keywords."

[1076] 6. The feedback collection module accepts the judges' review and feedback.

[1077] 7. The priority determination module displays this advertisement on the list as a high priority.

[1078] 8. The final report generation module generates the final audit report and notifies the relevant personnel.

[1079] In this way, the system improves the quality of online advertising by quickly assessing the risks of advertising creatives and encouraging appropriate responses.

[1080] The processing flow will be explained below.

[1081] Step 1: Collect ad creatives

[1082] Server: Collects advertising creatives (text, banners, videos) from multiple advertising platforms and websites on the Internet. Ad creatives are periodically obtained using API integration and web scraping technology.

[1083] Example: The server retrieves "new ads" through the API of an ad distribution platform and stores them in a database.

[1084] Step 2: Saving to the database

[1085] Server: Collected ad creatives are assigned a unique ID and stored in a database along with metadata (acquisition date, advertiser, ad type, etc.).

[1086] Example: The server saves the image file of the advertisement banner it has acquired in storage, and records the file path and related metadata in a database.

[1087] Step 3: Content analysis

[1088] Server: Performs text analysis, image analysis, and video analysis on ad creatives. Natural language processing (NLP) technology is used to analyze text, OCR technology is used to extract text from images, image recognition is performed, video analysis technology is used to analyze the content of each frame, and speech recognition is used to convert audio content into text.

[1089] Example: The server analyzes images in advertising banners to detect risky keywords such as "special price" and "buy now."

[1090] Step 4: Risk Assessment

[1091] Server: Based on the analysis results, the NG rate of the ad creative is calculated by comparing it with past judgment results. Specific risk factors are weighted to determine the overall risk level.

[1092] Example: Calculate the similarity between ads that have been judged to be "illegal" in the past and current ad creatives, and calculate the NG rate.

[1093] Step 5: Generate and save the reason for the decision

[1094] Server: Based on the analysis results and risk level of the advertising creative, the reason for the judgment and the risk level are generated in text format and saved in the database.

[1095] Example: The server generates and saves the reason for the judgment in text format, such as "This ad contains the keyword 'free' and therefore has a high risk of fraud."

[1096] Step 6: Judges gather feedback

[1097] Terminal: Provides an interface where human judges can review the results and make corrections or provide feedback as needed. Judges review the analysis results of the ad creative and the reasons for their decisions, and evaluate their accuracy.

[1098] Example: A judge checks the evaluation results of an advertising banner on a terminal and enters feedback such as "This advertisement is appropriate."

[1099] Step 7: Save the feedback and use it for retraining

[1100] Server: The acquired feedback information is saved as re-learning data. By re-learning the AI ​​model, the accuracy of judgments will be improved from the next time onwards.

[1101] Example: The server adds feedback data from judges to the database and incorporates it into the retraining dataset.

[1102] Step 8: Determine and list priorities

[1103] Server: Calculates the response priority of each ad creative based on risk and profitability, and displays a list of ad creatives in order of priority through the interface.

[1104] Example: The server sets high profitability and high risk advertisements as priority 1 and displays them in a prominent position on the device.

[1105] Step 9: Generate the final report

[1106] Server: Generates a final review report for the ad creatives that have been reviewed, stores it in a database, and notifies the relevant personnel.

[1107] Example: The server generates a final report for the advertisement "New Product Launch" and notifies the person in charge, including results such as "No problems" and "Appropriate wording."

[1108] These are the processing steps of the system, which allows for efficient advertising screening and improves the reliability of media.

[1109] Example 1

[1110] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1111] Some online advertisements contain illegal or fraudulent content, which can have a negative impact on users. In particular, it is difficult to properly evaluate and monitor advertisements in a consistent manner across a wide range of formats (text, images, and videos). Furthermore, there is a lack of a system for effectively feeding back the results of human judges' judgments and using them as retraining data. To address these issues, it is necessary to automate and streamline the entire process, from analyzing advertising creatives to assessing risk, prioritizing, and generating a final report.

[1112] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1113] In this invention, the server includes: means for collecting advertising creatives from the Internet; means for storing the collected advertising creatives in a database; means for analyzing the text, images, and video of the advertising creatives to extract risky elements; means for calculating a rejection rate by comparing past judgment results with current analysis results; means for analyzing the text portion of the advertising creatives using natural language processing technology to detect risky keywords; means for analyzing the content of advertising banners using image recognition technology to determine potential illegality; means for analyzing the content of advertising videos frame by frame using video analysis technology to detect inappropriate messages; and means for generating and storing the judgment reasons and risk levels in a database. This improves the quality of advertising on the Internet and minimizes adverse effects on users. The server also includes means for providing an interface for human reviewers to review the judgment results and make corrections or provide feedback; means for saving the acquired feedback information as re-learning data; means for calculating the response priority of advertising creatives based on risk and profitability; and means for providing an interface for displaying advertising creatives in order of priority, thereby automating and streamlining this entire process.

[1114] "Internet advertising creative" refers to the overall content of advertisements distributed over the Internet, including text, images, and videos.

[1115] "Means of collection" refers to programs and algorithms used to obtain data from the internet, including methods such as APIs, RSS feeds, and web scraping.

[1116] "Means of storing data in a database" refers to systems and algorithms for efficiently storing and managing collected data, including platforms for storing data in a particular format.

[1117] "Means of analysis" refers to technologies and programs that use machine learning and AI technology to evaluate and analyze the text, images, and videos of advertising creatives.

[1118] "Risk elements" refer to specific text, image, or video elements that pose concerns about illegality or fraud, such as specific keywords, icons, or colors.

[1119] The "NG rate" is an indicator that shows how risky an ad creative is compared to past judgment results, and is often expressed as a number or percentage.

[1120] "Natural language processing technology" refers to artificial intelligence technology for analyzing meaning and structure from text data, and specifically includes the use of machine learning algorithms and APIs.

[1121] "Image recognition technology" refers to the technology of analyzing image data to extract and recognize specific features, and computer vision technology and deep learning algorithms are commonly used.

[1122] "Video analysis technology" refers to the technology of analyzing video data frame by frame to extract and evaluate specific content, and includes video processing software and voice recognition technology.

[1123] "Reasons for judgment and risk level" refers to the analytical reasons generated as a result of evaluating the advertising creative and indicators that quantitatively and qualitatively indicate the level of risk.

[1124] "Feedback interface" refers to a user interface that allows human reviewers to review the analysis results and enter corrections and feedback.

[1125] "Retraining data" refers to data used to improve the performance of an AI model based on feedback information, and is used through a continuous learning process.

[1126] The "means for calculating priority" refers to a program for using an algorithm to calculate the priority of a response based on the risk and profitability of an advertising creative.

[1127] "Listing interface" refers to a user interface that displays advertising creatives in order of priority, allowing users to efficiently respond.

[1128] "Final Report" refers to a comprehensive report containing analysis results and feedback on the advertising creatives that have been reviewed, and is a document stored in a database.

[1129] "Means of notification" refers to the system or method for notifying relevant personnel of the generated final report, including email and notification systems.

[1130] MODE FOR CARRYING OUT THE INVENTION

[1131] The present invention is a system for detecting illegal or fraudulent activities by collecting and analyzing advertising creatives on the Internet. Specific embodiments of this system will be described below.

[1132] System configuration

[1133] The system mainly consists of three elements: a server, a terminal, and a user. The server collects, analyzes, and evaluates data, and the terminal provides a user interface. The user checks the information provided by the system and provides feedback as needed.

[1134] Hardware and software used

[1135] 1. The server uses the following technologies to collect, analyze, and evaluate advertising creatives:

[1136] Use APIs and RSS feeds to gather advertising creatives from across the internet. Examples include APIs of ad serving platforms, RSS feeds, and web scraping.

[1137] Natural language processing technology (e.g., Google Cloud Natural Language API) is used to analyze the text and detect risky keywords such as "free" and "limited."

[1138] Image recognition technology (e.g., OpenCV, Google Cloud Vision API) is used to analyze the content of advertising banners and determine whether they may be illegal.

[1139] Video analysis technology (e.g., FFmpeg, Google Cloud Video Intelligence API) is used to analyze the video content frame by frame, and speech recognition technology is used to convert the audio content into text and detect inappropriate messages.

[1140] The database will be used to store collected advertising creatives, analysis results, reasons for judgments, etc.

[1141] 2. The device provides an interface for the user to review the analysis results and provide feedback. Specific elements include:

[1142] The user interface visually displays the analysis results, allowing the user to easily check the judgment results.

[1143] The feedback function provides an interface where the user can enter corrections and feedback on the results.

[1144] 3. The user acts as a judge, reviewing the information provided by the system and providing feedback if necessary.

[1145] Specific examples

[1146] For example, if a banner ad contains text such as "Buy now" or "Limited edition," the system's processing flow is as follows:

[1147] 1. The server collects these banner ads and stores them in a database.

[1148] 2. The server uses a text analysis module to analyze the banner text and detect risky keywords (e.g., "Buy now" or "Limited Edition").

[1149] 3. The server uses an image analysis module to analyze the banner image and check for inappropriate content and design.

[1150] 4. The server uses the risk assessment module to refer to past similar ad data and calculate the NG rate.

[1151] 5. The server uses the judgment reason generation module to generate a reason such as "It is highly risky because it contains specific keywords."

[1152] 6. The device accepts the judges' confirmation and feedback through the feedback collection module.

[1153] 7. The server uses the priority determination module to display this advertisement on the list as a high priority.

[1154] 8. The server uses the final report generation module to generate the final review report and notify the relevant personnel.

[1155] Prompt Sentence Examples

[1156] Here is an example of a prompt to input to the generative AI model:

[1157] Please explain the system that collects and analyzes advertising creatives containing keywords such as "buy now" and "limited edition" to assess their potential for illegality.

[1158] In this way, the system improves the quality of online advertising by quickly assessing the risks of advertising creatives and encouraging appropriate responses.

[1159] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1160] Step 1: Collect ad creatives

[1161] Input: Ad serving platform API, RSS feed, website URL

[1162] How it works: The server collects ad creatives from specified internet sources. The server uses the ad serving platform's API to get the latest ad lists, extracts newly published ads from RSS feeds, and uses web scraping to collect ad banners from specific web pages.

[1163] Output: Collected ad creative data

[1164] Step 2: Saving to the database

[1165] Input: Collected ad creative data

[1166] Specific operation: The server stores the collected ad creatives in a database, assigns a unique ID, and stores metadata for each ad creative (e.g., collection source, date, format, etc.).

[1167] Output: Ad creatives and their metadata stored in a database

[1168] Step 3: Text analysis

[1169] Input: Ad creative text data loaded from the database

[1170] Specific operation: The server analyzes the text using natural language processing technology (e.g., Google Cloud Natural Language API). It detects risky keywords and expressions such as "free" and "limited" and marks them.

[1171] Output: Parsed text data with risk keywords marked

[1172] Step 4: Image analysis

[1173] Input: Ad creative image data loaded from database

[1174] How it works: The server analyzes the content of the ad banner using image recognition technology (e.g., OpenCV, Google Cloud Vision API), extracting icons, text, and colors from the image and determining whether they match indicators of illegality.

[1175] Output: Analyzed image data and a list of potentially illegal elements

[1176] Step 5: Video Analysis

[1177] Input: Video data of the ad creative loaded from the database

[1178] Specific operation: The server analyzes the content of the advertisement video frame by frame using video analysis technology (e.g., FFmpeg, Google Cloud Video Intelligence API), converts the audio content into text using speech recognition technology, and detects inappropriate messages.

[1179] Output: Analyzed frame data and text data based on speech recognition

[1180] Step 6: Risk Assessment

[1181] Input: Analyzed text data, image data, video data

[1182] Specific operation: The server compares past judgment results with the current analysis results to calculate the NG rate. It weights each specific element and evaluates the overall risk level.

[1183] Output: Risk assessment results including NG rate

[1184] Step 7: Generate reasons for the decision

[1185] Input: Risk assessment results

[1186] Specific operation: The server generates the reason for the judgment and the risk level based on the analysis results and saves them in a database in text format. The reason for the judgment is automatically generated and the specific reason is output in text format.

[1187] Output: Reason for judgment and risk level stored in the database

[1188] Step 8: Gather feedback

[1189] Input: Reason for judgment and risk level stored in the database

[1190] Specific operation: The terminal provides an interface that allows the user (judge) to check the judging results and add corrections or feedback. The user enters feedback, and the terminal sends it to the server.

[1191] Output: Feedback data sent to the server

[1192] Step 9: Prioritization

[1193] Input: Risk assessment results, feedback data

[1194] Specific operation: The server calculates the response priority of the ad creative based on the risk and profitability obtained from the analysis results, sorts the ads in order of priority, and displays them on the device.

[1195] Output: A list of ad creatives sorted by priority

[1196] Step 10: Generate the final report

[1197] Input: Final evaluation and feedback results

[1198] Specific operations: The server generates a final review report for the advertising creatives that have been reviewed, stores it in the database, and notifies the relevant personnel.

[1199] Output: Final report stored in database and notification sent to relevant parties through notification system

[1200] In this way, the processes at each step are organically linked to provide a system for assessing and managing the risks of advertising creatives.

[1201] (Application example 1)

[1202] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1203] The Internet is flooded with a wide variety of advertisements, many of which are illegal or involve fraudulent activities. This puts users at high risk of coming into contact with inappropriate advertisements, which could potentially impair their quality of life. Manually checking advertisement content requires a huge amount of time and effort, making it inefficient. Furthermore, there is an inadequate system for detecting advertising risks in real time and notifying users, making it difficult to prevent harm caused by advertisements.

[1204] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1205] In this invention, the server includes means for collecting advertising creatives from the Internet, means for saving the collected advertising creatives in a database, means for analyzing the text, images, and videos of the advertising creatives and extracting risky elements, means for comparing past judgment results with current analysis results to calculate an NG rate, means for generating judgment reasons and risk levels in text format and saving them in a database, means for scanning and analyzing the collected advertisements in real time while browsing, and means for displaying a warning to the user based on the analysis results. This makes it possible to automatically detect illegal or fraudulent advertising and notify the user in real time.

[1206] "Internet advertising creative" refers to advertising materials such as text, images, and video that are distributed via the Internet, such as websites and applications.

[1207] "Means of collection" refers to functions and modules for detecting advertising creatives on the Internet and capturing and storing them as data.

[1208] "Database" means an information management system that enables efficient storage, management, and access of collected advertising creatives and analysis results.

[1209] "Means of analysis" refers to algorithms or programs that examine the content (text, images, video) of advertising creatives and detect risky elements.

[1210] "Risk elements" refer to keywords, images, audio, etc. contained in the content of advertising creatives that may raise suspicions of illegality or fraud.

[1211] The "judgment result" is the evaluation or judgment derived from the analysis of the advertising creative.

[1212] The "NG rate" is a number that indicates the probability that illegal or fraudulent activity will be detected in the analyzed advertising creative.

[1213] "Text format" refers to a format in which analysis results and reasons for judgments are expressed as text information.

[1214] "Real-time scanning while browsing" refers to the ability to instantly detect and analyze advertisements displayed while a user is browsing a website.

[1215] "Means to display a warning to users" refers to a function that notifies users of risky advertising creatives when they are detected, and provides a warning or caution.

[1216] The present invention relates to a system that analyzes advertising creatives on the Internet in real time to detect illegal or fraudulent activities. This system has the function of analyzing the text, images, and videos of advertisements, evaluating the risk level, and displaying a warning to the user. The following describes in detail the embodiments of the present invention.

[1217] System configuration

[1218] Hardware and software used

[1219] 1. Hardware

[1220] Smartphones: Used for real-time analysis when users view ads, using built-in sensors such as the camera, display, and microphone.

[1221] Server: Responsible for major back-end processing such as collecting ad creatives, analyzing, and storing data.

[1222] 2. Software

[1223] Python: A programming language used to implement the analysis program.

[1224] BeautifulSoup: A web scraping library used to scrape ad creatives from the web.

[1225] OpenCV: An image processing library used to analyze advertising images and videos.

[1226] Pytesseract: An OCR library used for analyzing text in images.

[1227] TensorFlow: A machine learning framework used for natural language processing, including text analysis and speech recognition.

[1228] System Operation

[1229] 1. Collecting advertising creatives

[1230] The server collects ad creatives from designated internet sources (websites, APIs, etc.), stores the collected data in a database, and then analyzes it. For example, it retrieves ads from news sites and social media platforms.

[1231] 2. Analysis of advertising creatives

[1232] The server analyzes the text, images, and videos of the stored ad creatives. Text analysis uses natural language processing technology to detect risky keywords and expressions. Image analysis examines the text, icons, and color usage within images to determine the possibility of fraud. Video analysis analyzes each frame of video, converting audio into text to detect inappropriate content.

[1233] 3. Risk assessment and warning signs

[1234] The server compares past judgment results with the analysis results to calculate the NG rate. The analysis results are used to generate the reason for the judgment and the risk level, which are then saved in a text database. When a user views an advertisement in real time while browsing, the advertisement is scanned, the risk level is assessed based on the analysis results, and a warning is displayed if necessary.

[1235] 4. User Interface and Feedback Collection

[1236] The system provides an interface for users to respond to warning messages and collects user feedback. This feedback is sent to the server and saved as retraining data. Once the review is complete, a detailed review report is generated and notified to the relevant personnel.

[1237] Specific examples

[1238] As a concrete example, consider the case where analysis is performed using the following prompt sentence:

[1239] Analyze the ad text below to see if it contains any risky keywords.

[1240] Text: "Buy now, limited time offer, 100% free"

[1241] In this way, the system can efficiently detect illegal or fraudulent advertising creatives and provide users with real-time warnings, thereby improving the quality of advertising on the Internet and ensuring user safety.

[1242] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1243] Step 1:

[1244] The server collects ad creatives from specified internet sources (websites, APIs, etc.). For example, it detects ad containers from news sites or social media platforms and retrieves their contents in HTML format. The input is the website URL or API endpoint, and the output is the ad creative data.

[1245] Step 2:

[1246] The server stores the collected ad creatives in a database. The collected ad creatives are recorded in the database with a unique ID for subsequent analysis. The input is the ad creative data collected in step 1, and the output is a new record added to the database.

[1247] Step 3:

[1248] The server uses natural language processing technology to analyze the text portion of the ad creative stored in the database. Specifically, it extracts risky keywords (e.g., "free," "now," "limited," etc.) from the text data. The input is the text data of the ad creative, and the output is a list of risky keywords and expressions.

[1249] Step 4:

[1250] The server uses image recognition technology to analyze the content of the ad banner. It uses OCR technology to detect text, icons, color usage, etc. in the image and extracts risky elements. The input is the image data of the ad banner, and the output is the analyzed text and risky elements.

[1251] Step 5:

[1252] The server uses video analysis technology to analyze the content of the ad video. It analyzes the text and audio for each frame of the video to detect inappropriate content and expressions. The input is the ad video data, and the output is the extracted text and audio content, as well as risky elements.

[1253] Step 6:

[1254] The server compares past judgment results with the current analysis results to calculate the NG rate. Based on the analysis results for each element of the ad creative (text, images, video), the NG rate is calculated while weighting specific risk elements. The input at this time is the analysis result data, and the NG rate is obtained as the output.

[1255] Step 7:

[1256] The server generates the reason for the judgment and the risk level in text format and saves them in a database. Based on the analysis results, it generates a reason for the judgment, such as "It is highly risky because it contains specific keywords." The input at this time is the analysis result and NG rate, and the reason for the judgment is obtained in text format as an output.

[1257] Step 8:

[1258] The device scans and analyzes the ads displayed while the user is browsing in real time. The collected ads are analyzed on the server and a warning is displayed to the user if necessary. The input is the ad data displayed in real time, and the output is a warning about risky ads.

[1259] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1260] The advertising review system of the present invention analyzes the content of advertising creatives (text, banners, videos) collected from the Internet to detect illegality and fraudulent activities. Furthermore, by incorporating an emotion engine that recognizes user emotions, the present invention performs more advanced risk assessment of advertising and realizes optimization of display methods.

[1261] System configuration

[1262] The system consists of the following main components:

[1263] 1. Ad creative acquisition module:

[1264] The server collects advertising creatives (text, banners, videos) from various internet sources using the APIs of ad serving platforms and web scraping technology.

[1265] Example: A server uses the API of an ad distribution platform to periodically collect the latest ad banners and store them in a database.

[1266] 2. Text Analysis Module:

[1267] The server uses natural language processing technology to analyze the text content of the collected advertising creatives and identify inappropriate language and risky keywords.

[1268] Example: The server uses OCR technology to extract text from banner images and detect keywords such as "Buy now" and "Limited Edition."

[1269] 3. Image Analysis Module:

[1270] The server uses image recognition technology to analyze the visual content of advertising banners, identifying those promoting counterfeit products or false health information.

[1271] Example: The server analyzes the icons and colors in an advertising banner and determines that it may contain false information.

[1272] 4. Video Analysis Module:

[1273] The server uses video analysis technology to analyze each frame of the advertising video and uses voice recognition technology to convert the audio content into text.

[1274] Example: The server extracts and analyzes messages such as "Great deals" from the audio in the video.

[1275] 5. Emotion Engine:

[1276] The server is equipped with an emotion engine that recognizes the user's emotions in real time, detecting their emotional state (happiness, surprise, anger, sadness, etc.) through facial recognition and voice analysis.

[1277] Example: The server analyzes the user's camera footage and microphone input using an emotion engine to determine the emotion the user is expressing while watching an advertisement.

[1278] 6. Risk Assessment Module:

[1279] The server calculates the NG rate and risk level based on the analysis of the ad creative and the user's emotion recognition. For example, if the user expresses negative emotions toward the ad, it will determine that the risk is high.

[1280] Example: The server reflects the fact that "the user showed anger while watching an advertisement" in the risk assessment.

[1281] 7. Reasons for decision generation module:

[1282] The server generates a reason for the judgment based on the risk assessment result and stores it in a database. The reason for the judgment is generated in text format.

[1283] Example: The server generates and stores a reason for the judgment, such as "There is a high risk of fraud because it contains certain keywords."

[1284] 8. Feedback Collection Module:

[1285] The terminal provides an interface that allows users (human judges) to review the results and provide corrections or feedback as needed.

[1286] Example: A judge adds feedback to the evaluation results, such as "This advertisement is not illegal."

[1287] 9. Priority Judgment Module:

[1288] The server determines the priority of the advertising creatives taking into consideration both the risk and profitability, and displays them on the terminal in order of priority.

[1289] Example: The server determines that the advertisements that are both highly profitable and highly risky are given priority 1 and are displayed on the device.

[1290] 10. Final Report Generation Module:

[1291] The server prepares a final review report for the advertising creatives for which review has been completed, and notifies the relevant personnel.

[1292] Example: The server generates a final report for the advertisement "New Product Launch" and notifies the person in charge, including results such as "No problems" and "Appropriate wording."

[1293] This system effectively detects illegal and fraudulent advertising creatives and recognizes user sentiment in real time, thereby improving advertising quality and optimizing the user experience.

[1294] The processing flow will be explained below.

[1295] Step 1: Collect ad creatives

[1296] The server collects advertising creatives (text, banners, videos) from multiple ad distribution platforms and websites on the Internet, periodically retrieving advertisements using API integration and web scraping technology, and storing them in a database.

[1297] Step 2: Saving to the database

[1298] The server stores the collected ad creatives in a database with a unique ID, along with metadata (date and time of acquisition, advertiser, ad type, etc.).

[1299] Step 3: Content analysis

[1300] The server performs the following analysis on the ad creative:

[1301] Text analysis: Analyzes text content using natural language processing (NLP) techniques to detect risky keywords.

[1302] Image analysis: Using image recognition technology to analyze the visual content of advertising banners to identify inappropriate elements or fraudulent products.

[1303] Video analysis: Using video analysis technology, each frame of the advertising video is analyzed and the audio content is converted into text through speech recognition.

[1304] Step 4: Emotion Recognition

[1305] The server analyzes the user's camera footage and microphone input in real time using an emotion engine that recognizes the user's emotions, detecting the emotions (happiness, surprise, anger, sadness, etc.) that the user shows while watching the advertisement.

[1306] Step 5: Risk Assessment

[1307] The server integrates the results of the ad creative analysis and emotion recognition, and calculates the NG rate by comparing it with past judgment results. It also weights specific risk factors and determines the overall risk level.

[1308] Step 6: Generate and save the reason for the decision

[1309] The server generates a reason for the judgment in text format based on the risk assessment result and stores it in the database. The reason for the judgment includes the specific elements that were judged to be high risk.

[1310] Step 7: Collect and store feedback

[1311] The terminal provides an interface for human judges to review the results and provide corrections or feedback as necessary. The judges' feedback is sent to the server and saved as re-learning data.

[1312] Step 8: Determine and list priorities

[1313] The server calculates the response priority of the ad creatives based on the analysis results and profitability information, and displays them on the device in order of priority, allowing high-risk ads to be quickly addressed.

[1314] Step 9: Generate and communicate final report

[1315] The server generates a final review report for the reviewed advertising creatives and stores it in a database, and notifies the relevant personnel of the report.

[1316] Specific examples

[1317] Collection phase: The server retrieves new banner ads from the ad distribution platform via API and stores the collected data in a database.

[1318] Analysis phase: The server uses natural language processing technology to analyze the text of the advertising banner, detecting risky keywords such as "only now" and "limited edition." It also analyzes the image content to assess the possibility that it contains fraudulent products.

[1319] Emotion recognition phase: When a user watches an advertisement, the server uses an emotion engine to analyze the user's facial expressions and detect the emotions shown by the user in real time.

[1320] Evaluation phase: The server integrates the analysis results and emotion recognition results, compares them with past judgment results, calculates the NG rate, and generates the reason for the judgment.

[1321] Feedback phase: The judges check the reasons for the judgement on the terminal and input their feedback on whether the judgement was appropriate. This feedback is sent to the server and used for the next analysis.

[1322] Priority calculation phase: The server calculates the priority based on the profitability and risk of the ad creative, and displays it on the device in order of priority.

[1323] Final report phase: The server generates a final review report, stores it in the database, and notifies the person in charge.

[1324] In this way, the system efficiently detects illegal and fraudulent advertising and, by taking into account evaluations based on user emotions, improves advertising quality and optimizes the user experience.

[1325] Example 2

[1326] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1327] There is a wide variety of advertising creatives on the Internet, some of which may contain inappropriate or illegal content. This poses a risk of adversely affecting users, so there is a need for a system that can efficiently and effectively analyze advertising content and detect illegal or inappropriate content. However, conventional systems have problems such as low analysis accuracy and an inability to take user emotions into account.

[1328] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1329] In this invention, the server includes means for collecting advertising creatives from the Internet, means for storing the collected advertising creatives in a database, means for analyzing the text, images, and videos of the advertising creatives and extracting risky elements, means for recognizing user emotions in real time, means for calculating a risk level based on the results of the advertising creative analysis and emotion recognition, and means for generating the reasons for the determination and the risk level in text format and storing them in the database. This makes it possible to analyze the content of advertising creatives with high accuracy, efficiently detect illegal or inappropriate content, and perform risk assessment taking user emotions into consideration.

[1330] The "Internet" is a global digital communications system that connects computer networks around the world.

[1331] "Ad creative" refers to the elements of an advertisement displayed on the Internet, including content in any media format, such as text, banners, or video.

[1332] A "server" is a computer system that provides information and services over a network and processes data in response to requests from clients.

[1333] A "database" is an integrated data storage system for efficiently managing, searching, and updating large amounts of data.

[1334] "Text analysis" is the process of using natural language processing technology to understand the content of text data and extract inappropriate expressions and risky keywords.

[1335] "Image analysis" is the process of using image recognition technology to analyze the content of image data and detect inappropriate elements or false information.

[1336] "Video analysis" is the process of analyzing video data frame by frame and converting the audio content into text using speech recognition technology.

[1337] "Emotion recognition" is a technology that detects a user's emotional state (joy, surprise, anger, sadness, etc.) in real time through facial recognition technology and voice analysis.

[1338] The "risk level" is an evaluation index regarding the inappropriateness or illegality of an advertisement, calculated based on the content of the advertising creative and the results of user emotion recognition.

[1339] The "reason for assessment" is text that describes the specific reason for the risk assessment, generated based on the analysis results of the ad creative.

[1340] "Feedback" is evaluation information provided by judges who review the results of their assessments and add corrections or opinions as necessary.

[1341] "Profitability" is a metric that measures the degree of profit an advertisement is likely to generate.

[1342] "Priority" is an evaluation indicator that indicates the urgency and importance of a response based on the analysis results of advertising creative.

[1343] "Review Report" means a document that describes the final evaluation results for the advertising creatives that have been reviewed and is notified to the relevant personnel.

[1344] This invention is a system that can effectively detect illegal or inappropriate content in advertisements by analyzing the content of advertising creatives (text, banners, videos) collected on the Internet with high accuracy and recognizing user emotions in real time. Furthermore, it aims to perform risk assessment of advertisements and select appropriate display methods.

[1345] Hardware and software used

[1346] Server: The server, which is the core of this system, collects and analyzes advertising creatives and stores data. Its primary hardware is a high-performance computer with large storage capacity. Software includes APIs from ad distribution platforms, web scraping tools, natural language processing engines (e.g., Python's NLTK or spaCy), OCR engines (e.g., Tesseract), image recognition technology (e.g., OpenCV), speech recognition engines (e.g., Google Speech-to-Text API), and emotion recognition engines (e.g., Microsoft Azure's Emotion API).

[1347] Devices: The devices used by judges display the results of their judgement and provide an interface for providing feedback. These devices can be PCs, tablets, or smartphones.

[1348] Users: The users of this system include advertisers, judges, and general advertising users.

[1349] Specific examples of program processing

[1350] 1. Collecting advertising creatives

[1351] The server collects advertising creatives from various sources on the Internet using the API of the ad serving platform or a web scraping tool. For example, the server periodically sends a request to the ad serving platform to obtain the latest advertising banners and stores the collected data in a database.

[1352] 2. Analysis of advertising creatives

[1353] The server analyzes the collected text, images, and videos of the advertising creative using various analytical techniques, such as using an OCR engine to extract text from banner images and a natural language processing engine to detect specific keywords such as "buy now."

[1354] 3. Emotion recognition

[1355] The server uses an emotion recognition engine to recognize the user's emotions in real time. It analyzes the video and audio data acquired through the user's camera and microphone to recognize emotions such as joy, surprise, anger, and sadness. For example, the server detects the emotion "the user is surprised" from the user's microphone input.

[1356] 4. Risk assessment and risk calculation

[1357] The server calculates the risk level of the ad based on the analysis results of the ad creative and the user's emotion recognition. For example, if the user's anger towards the ad is 50% or more, the risk level is set to "high."

[1358] 5. Generating reasons for judgment

[1359] The server generates a reason for the judgment based on the risk assessment result and stores it in the database. The reason for the judgment is detailed in text format. For example, the server may generate a reason for the judgment that "the risk of fraud is high because a specific keyword is included."

[1360] 6. Gathering User Feedback

[1361] The terminal provides an interface for the judge to check the judging results and send feedback if necessary. For example, the judge can use the terminal to input feedback such as "This ad is OK" and send it to the server.

[1362] 7. Priority determination and display

[1363] The server determines the priority of the ad creative based on the risk and profitability, and displays them on the device in order of priority. For example, the server may set and display ads that are both highly profitable and highly risky as priority 1.

[1364] 8. Generate the final report

[1365] The server creates a final review report for the advertising creative that has been reviewed and notifies the relevant person in charge. For example, the server generates a final report for the advertisement "New Product Launch," concludes that there are no problems, and notifies the person in charge.

[1366] Example prompts to input to the generative AI model

[1367] As an example of a prompt statement, you can enter:

[1368] "Analyze the following ad creative to determine if its content is inappropriate and perform a risk assessment taking into account the results of the sentiment engine. Analyze the following text: 'Shop now! Limited offer!'"

[1369] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1370] Step 1: Collect ad creatives

[1371] Specific actions

[1372] The server collects advertising creatives from online ad distribution platforms and websites, using APIs and web scraping technology to obtain data and store it in a database.

[1373] Input: API endpoint of ad serving platform, target URL for web scraping

[1374] Processing: Sending API requests, parsing HTML, extracting data

[1375] Output: Collected ad creative dataset (text, banner, video)

[1376] Step 2: Text analysis

[1377] Specific actions

[1378] The server analyzes the text content of the collected ad creatives using natural language processing technology, extracts text from images using OCR technology, and detects risky keywords.

[1379] Input: Ad creative dataset (text, banner)

[1380] Processing: Text extraction using OCR, keyword detection using natural language processing technology

[1381] Output: Text data tagged with keywords

[1382] Step 3: Image analysis

[1383] Specific actions

[1384] The server uses image recognition technology to analyze the visual content of advertising banners to detect inappropriate products and false health information.

[1385] Input: Ad creative dataset (banner image)

[1386] Processing: Image recognition algorithms are used to analyze and detect inappropriate content.

[1387] Output: Image tag as analysis result (if inappropriate elements are detected)

[1388] Step 4: Video Analysis

[1389] Specific actions

[1390] The server uses video analysis technology to analyze each frame of the ad video, and speech recognition technology to convert the audio content into text and determine whether it contains inappropriate content.

[1391] Input: Ad creative dataset (video files)

[1392] Processing: Frame-by-frame analysis, speech-to-text recognition, and inappropriate content detection

[1393] Output: Texted audio data and analysis report

[1394] Step 5: Emotion Recognition

[1395] Specific actions

[1396] The server utilizes an emotion recognition engine to recognize the user's emotions in real time, using facial recognition technology and voice analysis to detect the emotional state the user is expressing.

[1397] Input: User's camera and microphone input

[1398] Processing: Analysis by emotion recognition engine, identification of emotional state

[1399] Output: Emotion recognition results (happiness, surprise, anger, sadness, etc.)

[1400] Step 6: Risk Assessment

[1401] Specific actions

[1402] The server calculates the risk level based on the analysis results of the ad creative and the user's emotion recognition. For example, if the user expresses negative emotions toward the ad, it will determine the risk level as high.

[1403] Input: text, image, video analysis results, emotion recognition results

[1404] Processing: Data integration and analysis, risk level calculation

[1405] Output: Risk assessment report

[1406] Step 7: Generate reasons for the decision

[1407] Specific actions

[1408] The server generates a reason for the judgment based on the risk assessment result and stores it in the database. The reason for the judgment is detailed in text format.

[1409] Input: Hazard Assessment Report

[1410] Processing: Generating and formatting the reason for the decision

[1411] Output: Reason for decision in text format

[1412] Step 8: Gather feedback

[1413] Specific actions

[1414] The terminal provides an interface for judges to review the results and provide feedback if necessary.

[1415] Input: Judge feedback information

[1416] Process: Enter and send feedback information

[1417] Output: Corrected evaluation data

[1418] Step 9: Determine and display priorities

[1419] Specific actions

[1420] The server determines the priority of the advertising creatives based on the risk and profitability, and displays them on the terminal in order of priority.

[1421] Inputs: Risk assessment report, profitability data

[1422] Processing: Evaluating and determining priority, generating display data

[1423] Output: A list of ad creatives ordered by priority

[1424] Step 10: Generate the final report

[1425] Specific actions

[1426] The server prepares a final review report for the advertising creatives for which review has been completed, and notifies the relevant personnel.

[1427] Input: All analysis results and feedback information

[1428] Processing: Final report generation, formatting, and notification

[1429] Output: Final review report

[1430] (Application example 2)

[1431] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1432] There is a wide variety of advertising creatives on the Internet, many of which are illegal or fraudulent. Therefore, there is a need for a system that can automatically and effectively analyze advertising content and detect illegal or fraudulent activities. It is also necessary to evaluate the emotional impact of advertising on users and improve advertising quality. Furthermore, it is important to perform these evaluations in real time and improve advertising based on appropriate feedback.

[1433] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1434] In this invention, the server includes means for collecting advertising creatives from the Internet, means for saving the collected advertising creatives in a database, means for analyzing the text, images, and videos of the advertising creatives and extracting risky elements, means for comparing past judgment results with current analysis results to calculate an NG rate, means for evaluating the risk of displaying an advertisement using an emotion engine that recognizes the user's emotional state in real time, means for optimizing the advertising creatives based on the user's emotion recognition results, and means for generating judgment reasons and risk levels in text format and saving them in a database. This makes it possible to automatically detect illegality and fraudulent behavior in advertising creatives in real time and optimize advertising display taking user emotions into consideration.

[1435] "Internet advertising creative" means the content, including text, images, or video, of an advertisement displayed over the Internet.

[1436] "Means of collection" refers to the technology or method used to automatically collect advertising creatives published on the Internet and store them in a database.

[1437] "Means for storing in a database" refers to the technology or method for storing the collected advertising creative content in a digital format and managing it so that it can be subsequently analyzed and searched.

[1438] "Means of analysis and extraction of risky elements" refers to technologies and methods that use natural language processing, image recognition, voice analysis, etc. to analyze the content of advertising creatives and detect elements that may be illegal or fraudulent.

[1439] "Means for calculating NG rate" refers to the technology or method for comparing past judgment results with current analysis results and calculating the percentage of inappropriate advertisements.

[1440] The "emotion engine that recognizes the user's emotional state in real time" is a system that uses facial recognition and voice analysis technology to detect the emotions (joy, surprise, anger, sadness, etc.) of users viewing advertisements in real time.

[1441] "Means for assessing the risk of displaying advertisements" refers to technologies and methods for assessing the risk level of advertising creatives based on the results of user emotion recognition and taking appropriate measures.

[1442] "Means for optimizing advertising creatives based on user emotion recognition results" refers to technologies and methods that adjust the content and display method of advertisements based on user emotion data, thereby improving the user experience.

[1443] "Means for generating the reasons for judgment and risk levels in text format and storing them in a database" refers to the technology and methods for generating the analysis results of advertising creatives as text and storing them in a database.

[1444] "An interface where human judges can review the results of their assessments and make corrections or provide feedback" refers to a user interface that allows human judges to review the analysis results of advertisements on a screen and make corrections or comments as necessary.

[1445] "Means for saving as re-learning data" refers to the technology and methods for collecting and saving feedback information as data for machine learning and using it to improve the system.

[1446] The "means for calculating the response priority of advertising creatives based on risk level and profitability" refers to a technique or method for determining the response priority in consideration of the risk level and profitability of advertising creatives.

[1447] The "interface for displaying a list of advertising creatives in order of priority" refers to a user interface for displaying a list of advertising creatives on a screen according to their corresponding priorities.

[1448] "Means for auditing the legality of advertising creatives in real time using smartphones" refers to technologies and methods that use smartphone functions to analyze and audit the illegality of displayed advertising creatives in real time.

[1449] "Means for generating an examination report" refers to a technology or method for creating a final analysis result for an advertising creative in the form of a report.

[1450] "Means for storing the audit report in a database and notifying relevant personnel" refers to the technology or method for storing the generated audit report in a database and notifying the personnel in charge of the contents of the report.

[1451] "Means for creating an evaluation report based on the results of user emotion recognition" refers to a technology or method for creating an evaluation report that includes an evaluation of the advertising creative and points for improvement, taking into account user emotion data.

[1452] A specific method for implementing the advertisement quality audit application according to the present invention will be described. In this embodiment, an application using a smartphone as a platform is exemplified, but the application can be implemented in a similar manner on other devices.

[1453] System configuration

[1454] 1. How to obtain advertising creatives

[1455] The server automatically collects advertising creatives published on the Internet using the API of the advertising distribution platform. For example, it periodically retrieves the latest advertising banners and videos and stores them in a database.

[1456] 2. Database storage method

[1457] The server stores the collected advertising creatives in a digital database, which is structured to facilitate subsequent analysis and retrieval.

[1458] 3. Text Analysis Methods

[1459] The server analyzes the text of the ad creative using natural language processing techniques, for example, OCR technology to extract text from ad images and detect marketing keywords such as "buy now."

[1460] 4. Image and video analysis methods

[1461] The server uses image recognition and video analysis technology to analyze the content of the ad creative in real time, checking the content and audio data of each frame to identify counterfeit products and false information.

[1462] 5. Use of Emotion Engine

[1463] The server uses the smartphone's camera and microphone to recognize the user's emotions in real time, using facial recognition and voice analysis technology to analyze the emotional state (happiness, surprise, anger, sadness, etc.) displayed by the user while watching the advertisement.

[1464] 6. Advertisement display risk assessment method

[1465] The server evaluates the NG rate and risk level based on the results of ad analysis and user emotion recognition. For example, if a user expresses anger or discomfort while watching an ad, it will be deemed high risk.

[1466] 7. Ad creative optimization methods

[1467] The server adjusts the content and display method of the advertisement based on the user's emotional data to improve the user experience.

[1468] 8. Judgment reason generation / storage means

[1469] The server generates a reason for the judgment and a risk level in text format based on the analysis results and stores this in a database. For example, it may generate a judgment reason such as "There is a high risk of fraud because a specific keyword is included."

[1470] 9. Feedback Collection Methods

[1471] Users can review the results through a provided interface and provide corrections and feedback if necessary, allowing the system to continuously learn and improve its accuracy.

[1472] Examples of examples and prompts to use

[1473] For example, if an ad creative contains the text "Buy Now" and the user expresses anger, the ad will be judged to be highly likely to be fraudulent. Feedback provided by users regarding the ad will also be saved as learning data for the system and reflected in future analyses.

[1474] Prompt Sentence Examples

[1475] We need an application that can detect illegal and fraudulent advertising creatives and analyze user sentiment in real time. This includes the following features:

[1476] 1. Collect advertising creatives from the internet

[1477] 2. Illegal detection through text analysis

[1478] 3. Image and video analysis

[1479] 4. Facial Recognition and Emotion Analysis

[1480] 5. Generate a report of the results

[1481] Based on this requirement, generate the best model.

[1482] The present invention effectively detects advertisements that contain illegal or fraudulent activities and optimizes advertisement display taking into account user emotions, thereby enabling improvements in advertisement quality and user experience.

[1483] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1484] Step 1:

[1485] The server automatically collects advertising creatives (images, videos, text) from the Internet through the advertising distribution platform's API.

[1486] Input: Ad serving platform API

[1487] Data processing and calculation: Obtaining advertising data from the internet and converting it into digital format

[1488] Output: Collected ad creatives

[1489] Step 2:

[1490] The server stores the collected advertising creatives in a database.

[1491] Input: Collected ad creatives

[1492] Data processing and calculation: Store advertising data in the appropriate tables in the database

[1493] Output: Ad creatives stored in a database

[1494] Step 3:

[1495] The server uses OCR technology to extract text from the advertising image and analyzes the text using natural language processing (NLP) technology.

[1496] Input: Ad creative image data stored in a database

[1497] Data processing and calculation: Extract text using OCR and analyze keywords using NLP

[1498] Output: Parsed text and its risk factors

[1499] Step 4:

[1500] The server uses image recognition and video analysis technology to analyze the ad creative and identify elements that may contain counterfeit products or false information.

[1501] Input: Image and video data of advertising creatives stored in the database

[1502] Data processing and calculation: Risk element extraction through image recognition and video frame analysis

[1503] Output: Analyzed image and video data and their risk factors

[1504] Step 5:

[1505] The server uses the smartphone's camera and microphone to collect data to recognize the user's emotions in real time.

[1506] Input: Camera image and microphone audio from the user's smartphone

[1507] Data processing and calculation: Emotion recognition using facial recognition and voice analysis technology

[1508] Output: User's emotional state data

[1509] Step 6:

[1510] The server evaluates the rejection rate and risk level based on the analysis results of the advertising creative and the user's emotion recognition results.

[1511] Input: Parsed advertising data and user emotional state data

[1512] Data processing and calculation: Risk assessment based on risk factors and emotional data

[1513] Output: Evaluation results of NG rate and risk level

[1514] Step 7:

[1515] The server optimizes the content and display method of the advertising creative based on the user's emotional data.

[1516] Input: Emotion recognition results and ad creative

[1517] Data processing and calculation: Determining and adjusting the optimal ad display method

[1518] Output: Optimized ad creative

[1519] Step 8:

[1520] Based on the analysis results, the server generates the reason for the judgment and the risk level in text format and stores this in a database.

[1521] Input: Risk factors, emotional data, risk level

[1522] Data processing and calculation: Generate textual reasons for judgment and risk level

[1523] Output: Text data of the generated judgment reason and risk level

[1524] Step 9:

[1525] The user can check the results of the assessment through the provided interface and submit corrections or feedback as necessary.

[1526] Input: Generated judgment reason and danger level text

[1527] Data processing and calculation: Adding and correcting feedback

[1528] Output: Corrected feedback data

[1529] Step 10:

[1530] The server stores user feedback as re-learning data and uses it to improve the accuracy of the system.

[1531] Input: User feedback data

[1532] Data processing and calculation: Collecting feedback data and adding it to the retraining dataset

[1533] Output: Updated training dataset

[1534] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1535] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1536] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1537] [Fourth embodiment]

[1538] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1539] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1540] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1541] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1542] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1543] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1544] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1545] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1546] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1547] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1548] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1549] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1550] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1551] This invention is a system that detects illegality and fraudulent activities by collecting advertising creatives from the Internet and analyzing their content. This system automatically analyzes the text, banners, videos, etc. of advertising creatives and calculates the NG rate by comparing it with past judgment results. This improves the quality of advertising and prevents the negative impact of illegal advertising on people's lives.

[1552] System configuration

[1553] The system consists of the following main components:

[1554] 1. Ad creative acquisition module:

[1555] The server collects ad creatives from various internet sources, including ad serving platform APIs, RSS feeds, web scraping, etc. The collected data is stored in a database and assigned a unique ID.

[1556] 2. Text Analysis Module:

[1557] The server analyzes the text of the saved ad creative using natural language processing technology, which detects risky keywords and expressions such as "free" and "limited."

[1558] 3. Image Analysis Module:

[1559] The server uses image recognition technology to analyze the content of the advertising banners, extracting icons, text, and color schemes from the images to determine whether they may be illegal.

[1560] 4. Video Analysis Module:

[1561] The server uses video analysis technology to analyze the content of the advertising video frame by frame, and uses voice recognition technology to convert the audio content into text and detect inappropriate messages.

[1562] 5. Risk Assessment Module:

[1563] The server compares past results with the current analysis result and calculates a NG rate, which includes weighting certain factors.

[1564] 6. Reasons for decision generation module:

[1565] The server generates the reason for the judgment and the risk level based on the analysis results and stores them in a database in text format. For example, it outputs a reason such as "The risk is high because it contains specific keywords."

[1566] 7. Feedback Collection Module:

[1567] The terminal provides an interface that allows users (human judges) to check the results of the assessment and input feedback, which is sent to the server and saved as retraining data.

[1568] 8. Priority Judgment Module:

[1569] The server calculates the priority of the ad creative based on the risk and profitability of the analysis results, sorts the ads in order of priority, and displays them on the device.

[1570] 9. Final Report Generation Module:

[1571] The server generates a final review report for the reviewed advertising creatives, which is stored in a database and notified to relevant personnel.

[1572] Specific examples

[1573] For example, if a banner ad contains text like "Buy now" or "Limited edition":

[1574] 1. The server collects these banner ads and stores them in a database.

[1575] 2. The text analysis module analyzes the banner text and detects risky keywords.

[1576] 3. The image analysis module analyzes banner images to check for inappropriate content and design.

[1577] 4. The risk assessment module references past data on similar advertisements and calculates the NG rate.

[1578] 5. The judgment reason generation module generates a reason such as "It is highly risky because it contains specific keywords."

[1579] 6. The feedback collection module accepts the judges' review and feedback.

[1580] 7. The priority determination module displays this advertisement on the list as a high priority.

[1581] 8. The final report generation module generates the final audit report and notifies the relevant personnel.

[1582] In this way, the system improves the quality of online advertising by quickly assessing the risks of advertising creatives and encouraging appropriate responses.

[1583] The processing flow will be explained below.

[1584] Step 1: Collect ad creatives

[1585] Server: Collects advertising creatives (text, banners, videos) from multiple advertising platforms and websites on the Internet. Ad creatives are periodically obtained using API integration and web scraping technology.

[1586] Example: The server retrieves "new ads" through the API of an ad distribution platform and stores them in a database.

[1587] Step 2: Saving to the database

[1588] Server: Collected ad creatives are assigned a unique ID and stored in a database along with metadata (acquisition date, advertiser, ad type, etc.).

[1589] Example: The server saves the image file of the advertisement banner it has acquired in storage, and records the file path and related metadata in a database.

[1590] Step 3: Content analysis

[1591] Server: Performs text analysis, image analysis, and video analysis on ad creatives. Natural language processing (NLP) technology is used to analyze text, OCR technology is used to extract text from images, image recognition is performed, video analysis technology is used to analyze the content of each frame, and speech recognition is used to convert audio content into text.

[1592] Example: The server analyzes images in advertising banners to detect risky keywords such as "special price" and "buy now."

[1593] Step 4: Risk Assessment

[1594] Server: Based on the analysis results, the NG rate of the ad creative is calculated by comparing it with past judgment results. Specific risk factors are weighted to determine the overall risk level.

[1595] Example: Calculate the similarity between ads that have been judged to be "illegal" in the past and current ad creatives, and calculate the NG rate.

[1596] Step 5: Generate and save the reason for the decision

[1597] Server: Based on the analysis results and risk level of the advertising creative, the reason for the judgment and the risk level are generated in text format and saved in the database.

[1598] Example: The server generates and saves the reason for the judgment in text format, such as "This ad contains the keyword 'free' and therefore has a high risk of fraud."

[1599] Step 6: Judges gather feedback

[1600] Terminal: Provides an interface where human judges can review the results and make corrections or provide feedback as needed. Judges review the analysis results of the ad creative and the reasons for their decisions, and evaluate their accuracy.

[1601] Example: A judge checks the evaluation results of an advertising banner on a terminal and enters feedback such as "This advertisement is appropriate."

[1602] Step 7: Save the feedback and use it for retraining

[1603] Server: The acquired feedback information is saved as re-learning data. By re-learning the AI ​​model, the accuracy of judgments will be improved from the next time onwards.

[1604] Example: The server adds feedback data from judges to the database and incorporates it into the retraining dataset.

[1605] Step 8: Determine and list priorities

[1606] Server: Calculates the response priority of each ad creative based on risk and profitability, and displays a list of ad creatives in order of priority through the interface.

[1607] Example: The server sets high profitability and high risk advertisements as priority 1 and displays them in a prominent position on the device.

[1608] Step 9: Generate the final report

[1609] Server: Generates a final review report for the ad creatives that have been reviewed, stores it in a database, and notifies the relevant personnel.

[1610] Example: The server generates a final report for the advertisement "New Product Launch" and notifies the person in charge, including results such as "No problems" and "Appropriate wording."

[1611] These are the processing steps of the system, which allows for efficient advertising screening and improves the reliability of media.

[1612] Example 1

[1613] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1614] Some online advertisements contain illegal or fraudulent content, which can have a negative impact on users. In particular, it is difficult to properly evaluate and monitor advertisements in a consistent manner across a wide range of formats (text, images, and videos). Furthermore, there is a lack of a system for effectively feeding back the results of human judges' judgments and using them as retraining data. To address these issues, it is necessary to automate and streamline the entire process, from analyzing advertising creatives to assessing risk, prioritizing, and generating a final report.

[1615] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1616] In this invention, the server includes: means for collecting advertising creatives from the Internet; means for storing the collected advertising creatives in a database; means for analyzing the text, images, and video of the advertising creatives to extract risky elements; means for calculating a rejection rate by comparing past judgment results with current analysis results; means for analyzing the text portion of the advertising creatives using natural language processing technology to detect risky keywords; means for analyzing the content of advertising banners using image recognition technology to determine potential illegality; means for analyzing the content of advertising videos frame by frame using video analysis technology to detect inappropriate messages; and means for generating and storing the judgment reasons and risk levels in a database. This improves the quality of advertising on the Internet and minimizes adverse effects on users. The server also includes means for providing an interface for human reviewers to review the judgment results and make corrections or provide feedback; means for saving the acquired feedback information as re-learning data; means for calculating the response priority of advertising creatives based on risk and profitability; and means for providing an interface for displaying advertising creatives in order of priority, thereby automating and streamlining this entire process.

[1617] "Internet advertising creative" refers to the overall content of advertisements distributed over the Internet, including text, images, and videos.

[1618] "Means of collection" refers to programs and algorithms used to obtain data from the internet, including methods such as APIs, RSS feeds, and web scraping.

[1619] "Means of storing data in a database" refers to systems and algorithms for efficiently storing and managing collected data, including platforms for storing data in a particular format.

[1620] "Means of analysis" refers to technologies and programs that use machine learning and AI technology to evaluate and analyze the text, images, and videos of advertising creatives.

[1621] "Risk elements" refer to specific text, image, or video elements that pose concerns about illegality or fraud, such as specific keywords, icons, or colors.

[1622] The "NG rate" is an indicator that shows how risky an ad creative is compared to past judgment results, and is often expressed as a number or percentage.

[1623] "Natural language processing technology" refers to artificial intelligence technology for analyzing meaning and structure from text data, and specifically includes the use of machine learning algorithms and APIs.

[1624] "Image recognition technology" refers to the technology of analyzing image data to extract and recognize specific features, and computer vision technology and deep learning algorithms are commonly used.

[1625] "Video analysis technology" refers to the technology of analyzing video data frame by frame to extract and evaluate specific content, and includes video processing software and voice recognition technology.

[1626] "Reasons for judgment and risk level" refers to the analytical reasons generated as a result of evaluating the advertising creative and indicators that quantitatively and qualitatively indicate the level of risk.

[1627] "Feedback interface" refers to a user interface that allows human reviewers to review the analysis results and enter corrections and feedback.

[1628] "Retraining data" refers to data used to improve the performance of an AI model based on feedback information, and is used through a continuous learning process.

[1629] The "means for calculating priority" refers to a program for using an algorithm to calculate the priority of a response based on the risk and profitability of an advertising creative.

[1630] "Listing interface" refers to a user interface that displays advertising creatives in order of priority, allowing users to efficiently respond.

[1631] "Final Report" refers to a comprehensive report containing analysis results and feedback on the advertising creatives that have been reviewed, and is a document stored in a database.

[1632] "Means of notification" refers to the system or method for notifying relevant personnel of the generated final report, including email and notification systems.

[1633] MODE FOR CARRYING OUT THE INVENTION

[1634] The present invention is a system for detecting illegal or fraudulent activities by collecting and analyzing advertising creatives on the Internet. Specific embodiments of this system will be described below.

[1635] System configuration

[1636] The system mainly consists of three elements: a server, a terminal, and a user. The server collects, analyzes, and evaluates data, and the terminal provides a user interface. The user checks the information provided by the system and provides feedback as needed.

[1637] Hardware and software used

[1638] 1. The server uses the following technologies to collect, analyze, and evaluate advertising creatives:

[1639] Use APIs and RSS feeds to gather advertising creatives from across the internet. Examples include APIs of ad serving platforms, RSS feeds, and web scraping.

[1640] Natural language processing technology (e.g., Google Cloud Natural Language API) is used to analyze the text and detect risky keywords such as "free" and "limited."

[1641] Image recognition technology (e.g., OpenCV, Google Cloud Vision API) is used to analyze the content of advertising banners and determine whether they may be illegal.

[1642] Video analysis technology (e.g., FFmpeg, Google Cloud Video Intelligence API) is used to analyze the video content frame by frame, and speech recognition technology is used to convert the audio content into text and detect inappropriate messages.

[1643] The database will be used to store collected advertising creatives, analysis results, reasons for judgments, etc.

[1644] 2. The device provides an interface for the user to review the analysis results and provide feedback. Specific elements include:

[1645] The user interface visually displays the analysis results, allowing the user to easily check the judgment results.

[1646] The feedback function provides an interface where the user can enter corrections and feedback on the results.

[1647] 3. The user acts as a judge, reviewing the information provided by the system and providing feedback if necessary.

[1648] Specific examples

[1649] For example, if a banner ad contains text such as "Buy now" or "Limited edition," the system's processing flow is as follows:

[1650] 1. The server collects these banner ads and stores them in a database.

[1651] 2. The server uses a text analysis module to analyze the banner text and detect risky keywords (e.g., "Buy now" or "Limited Edition").

[1652] 3. The server uses an image analysis module to analyze the banner image and check for inappropriate content and design.

[1653] 4. The server uses the risk assessment module to refer to past similar ad data and calculate the NG rate.

[1654] 5. The server uses the judgment reason generation module to generate a reason such as "It is highly risky because it contains specific keywords."

[1655] 6. The device accepts the judges' confirmation and feedback through the feedback collection module.

[1656] 7. The server uses the priority determination module to display this advertisement on the list as a high priority.

[1657] 8. The server uses the final report generation module to generate the final review report and notify the relevant personnel.

[1658] Prompt Sentence Examples

[1659] Here is an example of a prompt to input to the generative AI model:

[1660] Please explain the system that collects and analyzes advertising creatives containing keywords such as "buy now" and "limited edition" to assess their potential for illegality.

[1661] In this way, the system improves the quality of online advertising by quickly assessing the risks of advertising creatives and encouraging appropriate responses.

[1662] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1663] Step 1: Collect ad creatives

[1664] Input: Ad serving platform API, RSS feed, website URL

[1665] How it works: The server collects ad creatives from specified internet sources. The server uses the ad serving platform's API to get the latest ad lists, extracts newly published ads from RSS feeds, and uses web scraping to collect ad banners from specific web pages.

[1666] Output: Collected ad creative data

[1667] Step 2: Saving to the database

[1668] Input: Collected ad creative data

[1669] Specific operation: The server stores the collected ad creatives in a database, assigns a unique ID, and stores metadata for each ad creative (e.g., collection source, date, format, etc.).

[1670] Output: Ad creatives and their metadata stored in a database

[1671] Step 3: Text analysis

[1672] Input: Ad creative text data loaded from the database

[1673] Specific operation: The server analyzes the text using natural language processing technology (e.g., Google Cloud Natural Language API). It detects risky keywords and expressions such as "free" and "limited" and marks them.

[1674] Output: Parsed text data with risk keywords marked

[1675] Step 4: Image analysis

[1676] Input: Ad creative image data loaded from database

[1677] How it works: The server analyzes the content of the ad banner using image recognition technology (e.g., OpenCV, Google Cloud Vision API), extracting icons, text, and colors from the image and determining whether they match indicators of illegality.

[1678] Output: Analyzed image data and a list of potentially illegal elements

[1679] Step 5: Video Analysis

[1680] Input: Video data of the ad creative loaded from the database

[1681] Specific operation: The server analyzes the content of the advertisement video frame by frame using video analysis technology (e.g., FFmpeg, Google Cloud Video Intelligence API), converts the audio content into text using speech recognition technology, and detects inappropriate messages.

[1682] Output: Analyzed frame data and text data based on speech recognition

[1683] Step 6: Risk Assessment

[1684] Input: Analyzed text data, image data, video data

[1685] Specific operation: The server compares past judgment results with the current analysis results to calculate the NG rate. It weights each specific element and evaluates the overall risk level.

[1686] Output: Risk assessment results including NG rate

[1687] Step 7: Generate reasons for the decision

[1688] Input: Risk assessment results

[1689] Specific operation: The server generates the reason for the judgment and the risk level based on the analysis results and saves them in a database in text format. The reason for the judgment is automatically generated and the specific reason is output in text format.

[1690] Output: Reason for judgment and risk level stored in the database

[1691] Step 8: Gather feedback

[1692] Input: Reason for judgment and risk level stored in the database

[1693] Specific operation: The terminal provides an interface that allows the user (judge) to check the judging results and add corrections or feedback. The user enters feedback, and the terminal sends it to the server.

[1694] Output: Feedback data sent to the server

[1695] Step 9: Prioritization

[1696] Input: Risk assessment results, feedback data

[1697] Specific operation: The server calculates the response priority of the ad creative based on the risk and profitability obtained from the analysis results, sorts the ads in order of priority, and displays them on the device.

[1698] Output: A list of ad creatives sorted by priority

[1699] Step 10: Generate the final report

[1700] Input: Final evaluation and feedback results

[1701] Specific operations: The server generates a final review report for the advertising creatives that have been reviewed, stores it in the database, and notifies the relevant personnel.

[1702] Output: Final report stored in database and notification sent to relevant parties through notification system

[1703] In this way, the processes at each step are organically linked to provide a system for assessing and managing the risks of advertising creatives.

[1704] (Application example 1)

[1705] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1706] The Internet is flooded with a wide variety of advertisements, many of which are illegal or involve fraudulent activities. This puts users at high risk of coming into contact with inappropriate advertisements, which could potentially impair their quality of life. Manually checking advertisement content requires a huge amount of time and effort, making it inefficient. Furthermore, there is an inadequate system for detecting advertising risks in real time and notifying users, making it difficult to prevent harm caused by advertisements.

[1707] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1708] In this invention, the server includes means for collecting advertising creatives from the Internet, means for saving the collected advertising creatives in a database, means for analyzing the text, images, and videos of the advertising creatives and extracting risky elements, means for comparing past judgment results with current analysis results to calculate an NG rate, means for generating judgment reasons and risk levels in text format and saving them in a database, means for scanning and analyzing the collected advertisements in real time while browsing, and means for displaying a warning to the user based on the analysis results. This makes it possible to automatically detect illegal or fraudulent advertising and notify the user in real time.

[1709] "Internet advertising creative" refers to advertising materials such as text, images, and video that are distributed via the Internet, such as websites and applications.

[1710] "Means of collection" refers to functions and modules for detecting advertising creatives on the Internet and capturing and storing them as data.

[1711] "Database" means an information management system that enables efficient storage, management, and access of collected advertising creatives and analysis results.

[1712] "Means of analysis" refers to algorithms or programs that examine the content (text, images, video) of advertising creatives and detect risky elements.

[1713] "Risk elements" refer to keywords, images, audio, etc. contained in the content of advertising creatives that may raise suspicions of illegality or fraud.

[1714] The "judgment result" is the evaluation or judgment derived from the analysis of the advertising creative.

[1715] The "NG rate" is a number that indicates the probability that illegal or fraudulent activity will be detected in the analyzed advertising creative.

[1716] "Text format" refers to a format in which analysis results and reasons for judgments are expressed as text information.

[1717] "Real-time scanning while browsing" refers to the ability to instantly detect and analyze advertisements displayed while a user is browsing a website.

[1718] "Means to display a warning to users" refers to a function that notifies users of risky advertising creatives when they are detected, and provides a warning or caution.

[1719] The present invention relates to a system that analyzes advertising creatives on the Internet in real time to detect illegal or fraudulent activities. This system has the function of analyzing the text, images, and videos of advertisements, evaluating the risk level, and displaying a warning to the user. The following describes in detail the embodiments of the present invention.

[1720] System configuration

[1721] Hardware and software used

[1722] 1. Hardware

[1723] Smartphones: Used for real-time analysis when users view ads, using built-in sensors such as the camera, display, and microphone.

[1724] Server: Responsible for major back-end processing such as collecting ad creatives, analyzing, and storing data.

[1725] 2. Software

[1726] Python: A programming language used to implement the analysis program.

[1727] BeautifulSoup: A web scraping library used to scrape ad creatives from the web.

[1728] OpenCV: An image processing library used to analyze advertising images and videos.

[1729] Pytesseract: An OCR library used for analyzing text in images.

[1730] TensorFlow: A machine learning framework used for natural language processing, including text analysis and speech recognition.

[1731] System Operation

[1732] 1. Collecting advertising creatives

[1733] The server collects ad creatives from designated internet sources (websites, APIs, etc.), stores the collected data in a database, and then analyzes it. For example, it retrieves ads from news sites and social media platforms.

[1734] 2. Analysis of advertising creatives

[1735] The server analyzes the text, images, and videos of the stored ad creatives. Text analysis uses natural language processing technology to detect risky keywords and expressions. Image analysis examines the text, icons, and color usage within images to determine the possibility of fraud. Video analysis analyzes each frame of video, converting audio into text to detect inappropriate content.

[1736] 3. Risk assessment and warning signs

[1737] The server compares past judgment results with the analysis results to calculate the NG rate. The analysis results are used to generate the reason for the judgment and the risk level, which are then saved in a text database. When a user views an advertisement in real time while browsing, the advertisement is scanned, the risk level is assessed based on the analysis results, and a warning is displayed if necessary.

[1738] 4. User Interface and Feedback Collection

[1739] The system provides an interface for users to respond to warning messages and collects user feedback. This feedback is sent to the server and saved as retraining data. Once the review is complete, a detailed review report is generated and notified to the relevant personnel.

[1740] Specific examples

[1741] As a concrete example, consider the case where analysis is performed using the following prompt sentence:

[1742] Analyze the ad text below to see if it contains any risky keywords.

[1743] Text: "Buy now, limited time offer, 100% free"

[1744] In this way, the system can efficiently detect illegal or fraudulent advertising creatives and provide users with real-time warnings, thereby improving the quality of advertising on the Internet and ensuring user safety.

[1745] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1746] Step 1:

[1747] The server collects ad creatives from specified internet sources (websites, APIs, etc.). For example, it detects ad containers from news sites or social media platforms and retrieves their contents in HTML format. The input is the website URL or API endpoint, and the output is the ad creative data.

[1748] Step 2:

[1749] The server stores the collected ad creatives in a database. The collected ad creatives are recorded in the database with a unique ID for subsequent analysis. The input is the ad creative data collected in step 1, and the output is a new record added to the database.

[1750] Step 3:

[1751] The server uses natural language processing technology to analyze the text portion of the ad creative stored in the database. Specifically, it extracts risky keywords (e.g., "free," "now," "limited," etc.) from the text data. The input is the text data of the ad creative, and the output is a list of risky keywords and expressions.

[1752] Step 4:

[1753] The server uses image recognition technology to analyze the content of the ad banner. It uses OCR technology to detect text, icons, color usage, etc. in the image and extracts risky elements. The input is the image data of the ad banner, and the output is the analyzed text and risky elements.

[1754] Step 5:

[1755] The server uses video analysis technology to analyze the content of the ad video. It analyzes the text and audio for each frame of the video to detect inappropriate content and expressions. The input is the ad video data, and the output is the extracted text and audio content, as well as risky elements.

[1756] Step 6:

[1757] The server compares past judgment results with the current analysis results to calculate the NG rate. Based on the analysis results for each element of the ad creative (text, images, video), the NG rate is calculated while weighting specific risk elements. The input at this time is the analysis result data, and the NG rate is obtained as the output.

[1758] Step 7:

[1759] The server generates the reason for the judgment and the risk level in text format and saves them in a database. Based on the analysis results, it generates a reason for the judgment, such as "It is highly risky because it contains specific keywords." The input at this time is the analysis result and NG rate, and the reason for the judgment is obtained in text format as an output.

[1760] Step 8:

[1761] The device scans and analyzes the ads displayed while the user is browsing in real time. The collected ads are analyzed on the server and a warning is displayed to the user if necessary. The input is the ad data displayed in real time, and the output is a warning about risky ads.

[1762] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1763] The advertising review system of the present invention analyzes the content of advertising creatives (text, banners, videos) collected from the Internet to detect illegality and fraudulent activities. Furthermore, by incorporating an emotion engine that recognizes user emotions, the present invention performs more advanced risk assessment of advertising and realizes optimization of display methods.

[1764] System configuration

[1765] The system consists of the following main components:

[1766] 1. Ad creative acquisition module:

[1767] The server collects advertising creatives (text, banners, videos) from various internet sources using the APIs of ad serving platforms and web scraping technology.

[1768] Example: A server uses the API of an ad distribution platform to periodically collect the latest ad banners and store them in a database.

[1769] 2. Text Analysis Module:

[1770] The server uses natural language processing technology to analyze the text content of the collected advertising creatives and identify inappropriate language and risky keywords.

[1771] Example: The server uses OCR technology to extract text from banner images and detect keywords such as "Buy now" and "Limited Edition."

[1772] 3. Image Analysis Module:

[1773] The server uses image recognition technology to analyze the visual content of advertising banners, identifying those promoting counterfeit products or false health information.

[1774] Example: The server analyzes the icons and colors in an advertising banner and determines that it may contain false information.

[1775] 4. Video Analysis Module:

[1776] The server uses video analysis technology to analyze each frame of the advertising video and uses voice recognition technology to convert the audio content into text.

[1777] Example: The server extracts and analyzes messages such as "Great deals" from the audio in the video.

[1778] 5. Emotion Engine:

[1779] The server is equipped with an emotion engine that recognizes the user's emotions in real time, detecting their emotional state (happiness, surprise, anger, sadness, etc.) through facial recognition and voice analysis.

[1780] Example: The server analyzes the user's camera footage and microphone input using an emotion engine to determine the emotion the user is expressing while watching an advertisement.

[1781] 6. Risk Assessment Module:

[1782] The server calculates the NG rate and risk level based on the analysis of the ad creative and the user's emotion recognition. For example, if the user expresses negative emotions toward the ad, it will determine that the risk is high.

[1783] Example: The server reflects the fact that "the user showed anger while watching an advertisement" in the risk assessment.

[1784] 7. Reasons for decision generation module:

[1785] The server generates a reason for the judgment based on the risk assessment result and stores it in a database. The reason for the judgment is generated in text format.

[1786] Example: The server generates and stores a reason for the judgment, such as "There is a high risk of fraud because it contains certain keywords."

[1787] 8. Feedback Collection Module:

[1788] The terminal provides an interface that allows users (human judges) to review the results and provide corrections or feedback as needed.

[1789] Example: A judge adds feedback to the evaluation results, such as "This advertisement is not illegal."

[1790] 9. Priority Judgment Module:

[1791] The server determines the priority of the advertising creatives taking into consideration both the risk and profitability, and displays them on the terminal in order of priority.

[1792] Example: The server determines that the advertisements that are both highly profitable and highly risky are given priority 1 and are displayed on the device.

[1793] 10. Final Report Generation Module:

[1794] The server prepares a final review report for the advertising creatives for which review has been completed, and notifies the relevant personnel.

[1795] Example: The server generates a final report for the advertisement "New Product Launch" and notifies the person in charge, including results such as "No problems" and "Appropriate wording."

[1796] This system effectively detects illegal and fraudulent advertising creatives and recognizes user sentiment in real time, thereby improving advertising quality and optimizing the user experience.

[1797] The processing flow will be explained below.

[1798] Step 1: Collect ad creatives

[1799] The server collects advertising creatives (text, banners, videos) from multiple ad distribution platforms and websites on the Internet, periodically retrieving advertisements using API integration and web scraping technology, and storing them in a database.

[1800] Step 2: Saving to the database

[1801] The server stores the collected ad creatives in a database with a unique ID, along with metadata (date and time of acquisition, advertiser, ad type, etc.).

[1802] Step 3: Content analysis

[1803] The server performs the following analysis on the ad creative:

[1804] Text analysis: Analyzes text content using natural language processing (NLP) techniques to detect risky keywords.

[1805] Image analysis: Using image recognition technology to analyze the visual content of advertising banners to identify inappropriate elements or fraudulent products.

[1806] Video analysis: Using video analysis technology, each frame of the advertising video is analyzed and the audio content is converted into text through speech recognition.

[1807] Step 4: Emotion Recognition

[1808] The server analyzes the user's camera footage and microphone input in real time using an emotion engine that recognizes the user's emotions, detecting the emotions (happiness, surprise, anger, sadness, etc.) that the user shows while watching the advertisement.

[1809] Step 5: Risk Assessment

[1810] The server integrates the results of the ad creative analysis and emotion recognition, and calculates the NG rate by comparing it with past judgment results. It also weights specific risk factors and determines the overall risk level.

[1811] Step 6: Generate and save the reason for the decision

[1812] The server generates a reason for the judgment in text format based on the risk assessment result and stores it in the database. The reason for the judgment includes the specific elements that were judged to be high risk.

[1813] Step 7: Collect and store feedback

[1814] The terminal provides an interface for human judges to review the results and provide corrections or feedback as necessary. The judges' feedback is sent to the server and saved as re-learning data.

[1815] Step 8: Determine and list priorities

[1816] The server calculates the response priority of the ad creatives based on the analysis results and profitability information, and displays them on the device in order of priority, allowing high-risk ads to be quickly addressed.

[1817] Step 9: Generate and communicate final report

[1818] The server generates a final review report for the reviewed advertising creatives and stores it in a database, and notifies the relevant personnel of the report.

[1819] Specific examples

[1820] Collection phase: The server retrieves new banner ads from the ad distribution platform via API and stores the collected data in a database.

[1821] Analysis phase: The server uses natural language processing technology to analyze the text of the advertising banner, detecting risky keywords such as "only now" and "limited edition." It also analyzes the image content to assess the possibility that it contains fraudulent products.

[1822] Emotion recognition phase: When a user watches an advertisement, the server uses an emotion engine to analyze the user's facial expressions and detect the emotions shown by the user in real time.

[1823] Evaluation phase: The server integrates the analysis results and emotion recognition results, compares them with past judgment results, calculates the NG rate, and generates the reason for the judgment.

[1824] Feedback phase: The judges check the reasons for the judgement on the terminal and input their feedback on whether the judgement was appropriate. This feedback is sent to the server and used for the next analysis.

[1825] Priority calculation phase: The server calculates the priority based on the profitability and risk of the ad creative, and displays it on the device in order of priority.

[1826] Final report phase: The server generates a final review report, stores it in the database, and notifies the person in charge.

[1827] In this way, the system efficiently detects illegal and fraudulent advertising and, by taking into account evaluations based on user emotions, improves advertising quality and optimizes the user experience.

[1828] Example 2

[1829] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1830] There is a wide variety of advertising creatives on the Internet, some of which may contain inappropriate or illegal content. This poses a risk of adversely affecting users, so there is a need for a system that can efficiently and effectively analyze advertising content and detect illegal or inappropriate content. However, conventional systems have problems such as low analysis accuracy and an inability to take user emotions into account.

[1831] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1832] In this invention, the server includes means for collecting advertising creatives from the Internet, means for storing the collected advertising creatives in a database, means for analyzing the text, images, and videos of the advertising creatives and extracting risky elements, means for recognizing user emotions in real time, means for calculating a risk level based on the results of the advertising creative analysis and emotion recognition, and means for generating the reasons for the determination and the risk level in text format and storing them in the database. This makes it possible to analyze the content of advertising creatives with high accuracy, efficiently detect illegal or inappropriate content, and perform risk assessment taking user emotions into consideration.

[1833] The "Internet" is a global digital communications system that connects computer networks around the world.

[1834] "Ad creative" refers to the elements of an advertisement displayed on the Internet, including content in any media format, such as text, banners, or video.

[1835] A "server" is a computer system that provides information and services over a network and processes data in response to requests from clients.

[1836] A "database" is an integrated data storage system for efficiently managing, searching, and updating large amounts of data.

[1837] "Text analysis" is the process of using natural language processing technology to understand the content of text data and extract inappropriate expressions and risky keywords.

[1838] "Image analysis" is the process of using image recognition technology to analyze the content of image data and detect inappropriate elements or false information.

[1839] "Video analysis" is the process of analyzing video data frame by frame and converting the audio content into text using speech recognition technology.

[1840] "Emotion recognition" is a technology that detects a user's emotional state (joy, surprise, anger, sadness, etc.) in real time through facial recognition technology and voice analysis.

[1841] The "risk level" is an evaluation index regarding the inappropriateness or illegality of an advertisement, calculated based on the content of the advertising creative and the results of user emotion recognition.

[1842] The "reason for assessment" is text that describes the specific reason for the risk assessment, generated based on the analysis results of the ad creative.

[1843] "Feedback" is evaluation information provided by judges who review the results of their assessments and add corrections or opinions as necessary.

[1844] "Profitability" is a metric that measures the degree of profit an advertisement is likely to generate.

[1845] "Priority" is an evaluation indicator that indicates the urgency and importance of a response based on the analysis results of advertising creative.

[1846] "Review Report" means a document that describes the final evaluation results for the advertising creatives that have been reviewed and is notified to the relevant personnel.

[1847] This invention is a system that can effectively detect illegal or inappropriate content in advertisements by analyzing the content of advertising creatives (text, banners, videos) collected on the Internet with high accuracy and recognizing user emotions in real time. Furthermore, it aims to perform risk assessment of advertisements and select appropriate display methods.

[1848] Hardware and software used

[1849] Server: The server, which is the core of this system, collects and analyzes advertising creatives and stores data. Its primary hardware is a high-performance computer with large storage capacity. Software includes APIs from ad distribution platforms, web scraping tools, natural language processing engines (e.g., Python's NLTK or spaCy), OCR engines (e.g., Tesseract), image recognition technology (e.g., OpenCV), speech recognition engines (e.g., Google Speech-to-Text API), and emotion recognition engines (e.g., Microsoft Azure's Emotion API).

[1850] Devices: The devices used by judges display the results of their judgement and provide an interface for providing feedback. These devices can be PCs, tablets, or smartphones.

[1851] Users: The users of this system include advertisers, judges, and general advertising users.

[1852] Specific examples of program processing

[1853] 1. Collecting advertising creatives

[1854] The server collects advertising creatives from various sources on the Internet using the API of the ad serving platform or a web scraping tool. For example, the server periodically sends a request to the ad serving platform to obtain the latest advertising banners and stores the collected data in a database.

[1855] 2. Analysis of advertising creatives

[1856] The server analyzes the collected text, images, and videos of the advertising creative using various analytical techniques, such as using an OCR engine to extract text from banner images and a natural language processing engine to detect specific keywords such as "buy now."

[1857] 3. Emotion recognition

[1858] The server uses an emotion recognition engine to recognize the user's emotions in real time. It analyzes the video and audio data acquired through the user's camera and microphone to recognize emotions such as joy, surprise, anger, and sadness. For example, the server detects the emotion "the user is surprised" from the user's microphone input.

[1859] 4. Risk assessment and risk calculation

[1860] The server calculates the risk level of the ad based on the analysis results of the ad creative and the user's emotion recognition. For example, if the user's anger towards the ad is 50% or more, the risk level is set to "high."

[1861] 5. Generating reasons for judgment

[1862] The server generates a reason for the judgment based on the risk assessment result and stores it in the database. The reason for the judgment is detailed in text format. For example, the server may generate a reason for the judgment that "the risk of fraud is high because a specific keyword is included."

[1863] 6. Gathering User Feedback

[1864] The terminal provides an interface for the judge to check the judging results and send feedback if necessary. For example, the judge can use the terminal to input feedback such as "This ad is OK" and send it to the server.

[1865] 7. Priority determination and display

[1866] The server determines the priority of the ad creative based on the risk and profitability, and displays them on the device in order of priority. For example, the server may set and display ads that are both highly profitable and highly risky as priority 1.

[1867] 8. Generate the final report

[1868] The server creates a final review report for the advertising creative that has been reviewed and notifies the relevant person in charge. For example, the server generates a final report for the advertisement "New Product Launch," concludes that there are no problems, and notifies the person in charge.

[1869] Example prompts to input to the generative AI model

[1870] As an example of a prompt statement, you can enter:

[1871] "Analyze the following ad creative to determine if its content is inappropriate and perform a risk assessment taking into account the results of the sentiment engine. Analyze the following text: 'Shop now! Limited offer!'"

[1872] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1873] Step 1: Collect ad creatives

[1874] Specific actions

[1875] The server collects advertising creatives from online ad distribution platforms and websites, using APIs and web scraping technology to obtain data and store it in a database.

[1876] Input: API endpoint of ad serving platform, target URL for web scraping

[1877] Processing: Sending API requests, parsing HTML, extracting data

[1878] Output: Collected ad creative dataset (text, banner, video)

[1879] Step 2: Text analysis

[1880] Specific actions

[1881] The server analyzes the text content of the collected ad creatives using natural language processing technology, extracts text from images using OCR technology, and detects risky keywords.

[1882] Input: Ad creative dataset (text, banner)

[1883] Processing: Text extraction using OCR, keyword detection using natural language processing technology

[1884] Output: Text data tagged with keywords

[1885] Step 3: Image analysis

[1886] Specific actions

[1887] The server uses image recognition technology to analyze the visual content of advertising banners to detect inappropriate products and false health information.

[1888] Input: Ad creative dataset (banner image)

[1889] Processing: Image recognition algorithms are used to analyze and detect inappropriate content.

[1890] Output: Image tag as analysis result (if inappropriate elements are detected)

[1891] Step 4: Video Analysis

[1892] Specific actions

[1893] The server uses video analysis technology to analyze each frame of the ad video, and speech recognition technology to convert the audio content into text and determine whether it contains inappropriate content.

[1894] Input: Ad creative dataset (video files)

[1895] Processing: Frame-by-frame analysis, speech-to-text recognition, and inappropriate content detection

[1896] Output: Texted audio data and analysis report

[1897] Step 5: Emotion Recognition

[1898] Specific actions

[1899] The server utilizes an emotion recognition engine to recognize the user's emotions in real time, using facial recognition technology and voice analysis to detect the emotional state the user is expressing.

[1900] Input: User's camera and microphone input

[1901] Processing: Analysis by emotion recognition engine, identification of emotional state

[1902] Output: Emotion recognition results (happiness, surprise, anger, sadness, etc.)

[1903] Step 6: Risk Assessment

[1904] Specific actions

[1905] The server calculates the risk level based on the analysis results of the ad creative and the user's emotion recognition. For example, if the user expresses negative emotions toward the ad, it will determine the risk level as high.

[1906] Input: text, image, video analysis results, emotion recognition results

[1907] Processing: Data integration and analysis, risk level calculation

[1908] Output: Risk assessment report

[1909] Step 7: Generate reasons for the decision

[1910] Specific actions

[1911] The server generates a reason for the judgment based on the risk assessment result and stores it in the database. The reason for the judgment is detailed in text format.

[1912] Input: Hazard Assessment Report

[1913] Processing: Generating and formatting the reason for the decision

[1914] Output: Reason for decision in text format

[1915] Step 8: Gather feedback

[1916] Specific actions

[1917] The terminal provides an interface for judges to review the results and provide feedback if necessary.

[1918] Input: Judge feedback information

[1919] Process: Enter and send feedback information

[1920] Output: Corrected evaluation data

[1921] Step 9: Determine and display priorities

[1922] Specific actions

[1923] The server determines the priority of the advertising creatives based on the risk and profitability, and displays them on the terminal in order of priority.

[1924] Inputs: Risk assessment report, profitability data

[1925] Processing: Evaluating and determining priority, generating display data

[1926] Output: A list of ad creatives ordered by priority

[1927] Step 10: Generate the final report

[1928] Specific actions

[1929] The server prepares a final review report for the advertising creatives for which review has been completed, and notifies the relevant personnel.

[1930] Input: All analysis results and feedback information

[1931] Processing: Final report generation, formatting, and notification

[1932] Output: Final review report

[1933] (Application example 2)

[1934] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1935] There is a wide variety of advertising creatives on the Internet, many of which are illegal or fraudulent. Therefore, there is a need for a system that can automatically and effectively analyze advertising content and detect illegal or fraudulent activities. It is also necessary to evaluate the emotional impact of advertising on users and improve advertising quality. Furthermore, it is important to perform these evaluations in real time and improve advertising based on appropriate feedback.

[1936] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1937] In this invention, the server includes means for collecting advertising creatives from the Internet, means for saving the collected advertising creatives in a database, means for analyzing the text, images, and videos of the advertising creatives and extracting risky elements, means for comparing past judgment results with current analysis results to calculate an NG rate, means for evaluating the risk of displaying an advertisement using an emotion engine that recognizes the user's emotional state in real time, means for optimizing the advertising creatives based on the user's emotion recognition results, and means for generating judgment reasons and risk levels in text format and saving them in a database. This makes it possible to automatically detect illegality and fraudulent behavior in advertising creatives in real time and optimize advertising display taking user emotions into consideration.

[1938] "Internet advertising creative" means the content, including text, images, or video, of an advertisement displayed over the Internet.

[1939] "Means of collection" refers to the technology or method used to automatically collect advertising creatives published on the Internet and store them in a database.

[1940] "Means for storing in a database" refers to the technology or method for storing the collected advertising creative content in a digital format and managing it so that it can be subsequently analyzed and searched.

[1941] "Means of analysis and extraction of risky elements" refers to technologies and methods that use natural language processing, image recognition, voice analysis, etc. to analyze the content of advertising creatives and detect elements that may be illegal or fraudulent.

[1942] "Means for calculating NG rate" refers to the technology or method for comparing past judgment results with current analysis results and calculating the percentage of inappropriate advertisements.

[1943] The "emotion engine that recognizes the user's emotional state in real time" is a system that uses facial recognition and voice analysis technology to detect the emotions (joy, surprise, anger, sadness, etc.) of users viewing advertisements in real time.

[1944] "Means for assessing the risk of displaying advertisements" refers to technologies and methods for assessing the risk level of advertising creatives based on the results of user emotion recognition and taking appropriate measures.

[1945] "Means for optimizing advertising creatives based on user emotion recognition results" refers to technologies and methods that adjust the content and display method of advertisements based on user emotion data, thereby improving the user experience.

[1946] "Means for generating the reasons for judgment and risk levels in text format and storing them in a database" refers to the technology and methods for generating the analysis results of advertising creatives as text and storing them in a database.

[1947] "An interface where human judges can review the results of their assessments and make corrections or provide feedback" refers to a user interface that allows human judges to review the analysis results of advertisements on a screen and make corrections or comments as necessary.

[1948] "Means for saving as re-learning data" refers to the technology and methods for collecting and saving feedback information as data for machine learning and using it to improve the system.

[1949] The "means for calculating the response priority of advertising creatives based on risk level and profitability" refers to a technique or method for determining the response priority in consideration of the risk level and profitability of advertising creatives.

[1950] The "interface for displaying a list of advertising creatives in order of priority" refers to a user interface for displaying a list of advertising creatives on a screen according to their corresponding priorities.

[1951] "Means for auditing the legality of advertising creatives in real time using smartphones" refers to technologies and methods that use smartphone functions to analyze and audit the illegality of displayed advertising creatives in real time.

[1952] "Means for generating an examination report" refers to a technology or method for creating a final analysis result for an advertising creative in the form of a report.

[1953] "Means for storing the audit report in a database and notifying relevant personnel" refers to the technology or method for storing the generated audit report in a database and notifying the personnel in charge of the contents of the report.

[1954] "Means for creating an evaluation report based on the results of user emotion recognition" refers to a technology or method for creating an evaluation report that includes an evaluation of the advertising creative and points for improvement, taking into account user emotion data.

[1955] A specific method for implementing the advertisement quality audit application according to the present invention will be described. In this embodiment, an application using a smartphone as a platform is exemplified, but the application can be implemented in a similar manner on other devices.

[1956] System configuration

[1957] 1. How to obtain advertising creatives

[1958] The server automatically collects advertising creatives published on the Internet using the API of the advertising distribution platform. For example, it periodically retrieves the latest advertising banners and videos and stores them in a database.

[1959] 2. Database storage method

[1960] The server stores the collected advertising creatives in a digital database, which is structured to facilitate subsequent analysis and retrieval.

[1961] 3. Text Analysis Methods

[1962] The server analyzes the text of the ad creative using natural language processing techniques, for example, OCR technology to extract text from ad images and detect marketing keywords such as "buy now."

[1963] 4. Image and video analysis methods

[1964] The server uses image recognition and video analysis technology to analyze the content of the ad cre...

Claims

1. A means of collecting advertising creatives on the Internet; a means for storing the collected advertising creatives in a database; Analyzing the text, images, and videos of advertising creatives to extract risky elements, A means for calculating the NG rate by comparing past judgment results with current analysis results; The system includes a means for generating the reason for the judgment and the risk level in text format and storing them in a database.

2. A means to provide an interface for human judges to review the results and provide corrections and feedback; A means for storing the acquired feedback information as re-learning data; A method for calculating the response priority of advertising creatives based on risk and profitability; 10. The system of claim 1, further comprising means for providing an interface that lists advertising creatives in order of priority.

3. a means for generating a final review report for the reviewed advertising creative; 2. The system according to claim 1, further comprising means for storing the generated final report in a database and notifying relevant personnel.

Images