Generative ai based system for product material evaluation and generation process observation or the like

A system using multiple computers to evaluate and broadcast generative AI product generation processes addresses the challenge of assessing AI prompt quality and skills, offering a competitive platform for fair evaluation and education in AI prompt engineering.

JP2024162018AInactive Publication Date: 2024-11-21株式会社 海馬
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023077153
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-09
Publication Date
2024-11-21
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The quality of products generated by generative AI is dependent on the quality and quantity of AI prompts, making it difficult to assess their appropriateness until they are input, and there is a need for a fair and objective evaluation of AI prompt engineering skills.

Method used

A system comprising multiple computers that evaluate products generated by generative AI based on input prompts, observing the production process, and providing a platform for competition similar to e-sports, using evaluation and transmission devices to assess and broadcast the generation process.

Benefits of technology

The system objectively evaluates AI prompt engineers' skills, contributing to the spread and education of AI prompt engineering, and provides entertaining e-sports-like entertainment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024162018000001_ABST
    Figure 2024162018000001_ABST
Patent Text Reader

Abstract

To provide a place for AI prompt engineers to compete in terms of their skills and a generative AI based system for product evaluation and generation process observation, or the like.SOLUTION: A generative AI based system for product evaluation and generation process observation includes: an evaluation device 11 equipped with reception means 2 for receiving product information 2a including a product generated based on subject material information 1a including a generation target subject material, and evaluation means 3 for outputting evaluation information 3a regarding the evaluation of the product; and a transmission device 13 equipped with screen acquisition means 4 for acquiring screen information 4a including a video of a contestant 12 generating the product and / or a screen display of an operation screen generating the product, and transmission means 5 for transmitting the screen information 4a.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a system for evaluating products produced by generative AI and for watching the process of production. It also relates to a system for evaluating products produced by generative AI through e-sports and for watching the process of production. [Background technology]

[0002] Conventionally, so-called e-sports tournaments have been held in which participants compete in computer games. For example, players compete in gaming skills using fighting games (see Patent Document 1) and puzzle games.

[0003] In recent years, a technology called generative AI (generative artificial intelligence) has been spreading in society. Generative AI is AI that generates new digital products, such as images, videos, audio, music, text, and code. In generative AI, instructions given to an AI, such as "Draw an illustration like..." or "Tell me about...", are called AI prompts or simply prompts (questions or instructions). Just as there is some trick to giving instructions using an AI prompt, the products (answers) generated by the AI ​​change depending on how the instructions are given. Moreover, the person who creates AI prompts to obtain the intended answer from AI is called an AI prompt engineer. In order to make the most of the capabilities of generative AI, AI prompt engineers create appropriate prompts, analyze and evaluate the answers and suggestions obtained from AI, and design more appropriate AI prompts based on the results. The academic field of developing and optimizing these AI prompts is called AI prompt engineering. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-171282 A Summary of the Invention [Problem to be solved by the invention]

[0005] Generally, there are some techniques and know-how whose skills are difficult to grasp without actually trying them out. For example, the quality of the product (answer) of a generative AI varies depending on the quality and quantity of the AI ​​prompt (question or instruction). For this reason, it tends to be difficult to know whether the quality and quantity of the prompt are appropriate unless you actually input them into the generative AI. On the other hand, in e-sports tournaments, competitions are held in public under set rules, and skill is evaluated in an objective and fair manner.

[0006] If an arena for competing in AI prompt creation techniques could be provided as e-sports, it could contribute to the spread of AI prompt engineering, help develop AI prompt engineers, and help discover excellent AI prompt engineers. It could also contribute to expanding employment opportunities for AI prompt engineers, and overall, it could become educational entertainment that combines e-sports with skill evaluation.

[0007] Therefore, the present invention aims to provide a forum for AI prompt engineers to compete against each other in terms of their skills, as well as a system for evaluating the products produced by generative AI and observing the generation process in order to objectively evaluate the skills of AI prompt engineers. [Means for solving the problem]

[0008] (1) The system for evaluating products generated by a generative AI and watching the generation process of the products of the present invention is a system consisting of a plurality of computers for evaluating products generated by a generative AI based on input of an athlete's AI prompts and for watching the generation process of the products, characterized in that it comprises an evaluation device having a reception means for receiving product information including the product generated based on subject information including the subject that is the target of generation, and an evaluation means for outputting evaluation information regarding the evaluation of the product, a transmission device having a screen acquisition means for acquiring screen information including a video of the athlete generating the product and / or a screen display of the operation screen generating the product, and a transmission means for transmitting the screen information.

[0009] The "generation process" refers to the process in which the AI ​​prompt engineer (the competitor) creates an appropriate prompt, analyzes and evaluates the answers and suggestions obtained from the AI, redesigns a more appropriate AI prompt based on the results, and develops and optimizes the AI ​​prompt to generate a product, or information that can be converted into such a product.

[0010] "Subject information" refers to what the contestant wants to generate using generative AI, the image or atmosphere of the thing they want to generate, and the direction for generation. For example, it is information on the subject that will be the subject or theme for the contestant to generate. The information is text, audio, image, video data, or a combination of multiple pieces of such data. Specifically, for example, it is information that presents information on the image, video, audio, music, sentence, or program code that is desired to be generated, or even information that suggests the atmosphere or image of the finished product without giving the image, video, audio, music, sentence, or program to be generated. It is a concept that also includes information that can be converted into the above-mentioned information.

[0011] "Product information" is a concept that corresponds to the product of electronic data generated by the generative AI based on the AI ​​prompt created by the contestant based on the subject information, and includes information that can be converted into that product.

[0012] The "evaluation information" is information indicating the degree to which the product information is close to the subject information / based on the subject information. For example, when specific text, audio, image, or video information is given as the subject information, the similarity between the product information and the subject information is quantified by analysis, and the evaluation information is the similarity. For example, the similarity is obtained by analyzing a plurality of viewpoints. For example, if the subject information is an image, the viewpoints are composition, color, touch, shape, etc. For example, the average value of the similarities in a plurality of viewpoints is used as the evaluation information, or the sum of the similarities in all viewpoints is used as the evaluation information, or other known evaluation techniques are used as the evaluation information. In addition to the similarity, it may be based on numerical information that quantifies the number of votes by a third party or the selection by an expert. The similarity and numerical information may be added to form the evaluation information. The evaluation information is a concept that includes the above-mentioned similarity and / or information that can be converted into numerical information.

[0013] "Screen information" is information on images showing the athlete competing and / or information on the screen display of the operation screen operated by the athlete, and is a concept that includes information that can be converted into these. Note that the images may be moving images or images that are moved over a predetermined period of time.

[0014] In the embodiment, the "receiving means" corresponds to steps S3-S4 (see FIG. 5). The "evaluation means" corresponds to step S5 (see FIG. 5) in this embodiment. In the embodiment, the "screen capture means" corresponds to step T1-T2 (see FIG. 8). In the embodiment, the "transmitting means" corresponds to step T3 (see FIG. 8).

[0015] (2) In such a system for evaluating products and observing the process of their generation by a generative AI, it is preferable that the transmission device is equipped with an addition means for adding related information related to the product information to the screen information.

[0016] "Related information" refers to information that is added to the screen information transmitted to viewers, and is a concept that includes information to explain the skills, techniques, and know-how that the competitors are competing with, as well as information that can be converted into those things. "Additional means" corresponds to step T2 (see FIG. 8) in this embodiment.

[0017] (3) It is also preferable to have a publishing means for publishing subject information including the subject on the Internet.

[0018] In the embodiment, the "disclosure means" corresponds to step S1 (see FIG. 5).

[0019] (4) The evaluation device of the present invention is an evaluation device used in a system that consists of a plurality of computers and evaluates products generated by a generation system AI based on input of AI prompts from a competitor, as well as for watching the process of generating the products, and is characterized in having a reception means for receiving product information including the products generated based on subject information including the subject that is the target of generation, and an evaluation means for outputting evaluation information regarding the evaluation of the products.

[0020] (5) It is preferable that such an evaluation device has a publishing means for publishing the subject information on the Internet.

[0021] (6) It is also preferable that the subject generates an image that is substantially similar to an image that is a target for generation, and that the evaluation information is a similarity calculated by numerically analyzing the subject information and the product information.

[0022] (7) It is preferable that the system further includes a selection means for selecting contestants corresponding to a predetermined degree of similarity or higher, the publication means publishes second subject information for the selected contestants to compete against each other, the reception means receives second product information generated by the selected contestants, and the evaluation means outputs second evaluation information regarding the evaluation of the second product information. In the embodiment, the "selecting means" corresponds to step S3 (see FIG. 4).

[0023] (8) The method of the present invention for evaluating a product generated by a generative AI and observing the generation process is executed on multiple computers and is used in a system for evaluating a product generated by a generative AI based on an athlete's AI prompt input and for observing the generation process of the product, and is characterized in that it accepts product information including a product generated based on subject information including a subject that is the target of generation, outputs evaluation information regarding the evaluation of a product including the subject, and obtains screen information including an image of the athlete transmitted from the athlete's terminal and / or a screen display of the operation screen generating the product.

[0024] (9) The evaluation method of the present invention is an evaluation method that is executed on multiple computers and is used in a system for evaluating a product generated by a generation system AI based on an athlete's AI prompt input and for watching the process of generating the product, and is characterized in that it accepts product information including the product generated based on subject information including the subject that is the target of generation, and outputs evaluation information regarding the evaluation of the product.

[0025] (10) In such an evaluation method, it is preferable that the subject information is made public on the Internet.

[0026] (11) It is also preferable that the subject generates an image that is substantially similar to an image that is a target for generation, and that the evaluation information is a similarity calculated by numerically analyzing the subject information and the product information.

[0027] (12) It is also preferable to select competitors corresponding to a predetermined degree of similarity or higher, publish second subject information for the selected competitors to compete against, accept second product information generated by the selected competitors, and output second evaluation information regarding an evaluation of the second product information.

[0028] (13) It is also preferable that the generation of the product information from the subject information be a preliminary round, and the generation of the second product information from the second subject information be a second round.

[0029] (14) It is also preferable that the system further comprises a specific information means for acquiring specific information for identifying the athlete, linking the information to the corresponding evaluation information, and storing the information in a memory unit.

[0030] In the embodiment, the "specific information means" corresponds to step S6 (see FIG. 5). "Identifying information" refers to information for identifying an athlete, such as ID, user name, name, My Number, pension number, driver's license number, physical characteristics including face, and other information that can identify a person, and is a concept that includes information linked to these or information that can be converted into these.

[0031] (15) It is also preferable that the system further comprises an instruction information storage means for acquiring instruction information including the AI ​​prompt used to generate the product information, and storing the instruction information in a storage unit in association with the product information.

[0032] "Instruction information" refers to textual instructions or commands given to generative AI, called prompts, and includes information that is entered by voice and converted into text, and is a concept that includes information that can be converted into such text. In the embodiment, the "instruction information means" corresponds to step S6 (see FIG. 5). Effect of the Invention

[0033] Systems for evaluating products produced by generative AI and observing the generation process can provide an objective evaluation of the skills of AI prompt engineers while also entertaining third parties as e-sports entertainment. [Brief description of the drawings]

[0034] [Figure 1]FIG. 1 is a schematic diagram of a system for evaluating products generated by a generative AI based on a player's AI prompt input, and for observing the process of generating said products. [Diagram 2] FIG. 2 is a functional block diagram illustrating an embodiment of the system shown in FIG. [Diagram 3] FIG. 2 is a schematic diagram illustrating an example of a hardware configuration of an evaluation device and a distribution device. [Figure 4] 1 is a schematic diagram showing an example of the hardware configuration of an athlete terminal and a viewer terminal. [Diagram 5] 13 is a flowchart showing an embodiment of the process flow of the evaluation device and athlete terminal. [Figure 6] FIG. 2 is a schematic diagram illustrating an example of a data structure of a person information database. [Figure 7] FIG. 4 is a schematic diagram illustrating an example of a data structure of a rule condition. [Figure 8] 13 is a flowchart showing an embodiment of the processing flow of the athlete terminal, the transmitting device, and the viewer terminal. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035] [1. Overview] (System 10) First, a system for evaluating products produced by generative AI and watching the generation process (hereinafter, simply referred to as the system) will be explained using Figure 1. Figure 1 shows a system 10. In the system 10, a given theme is first given to a contestant, the contestant 12 designs an AI prompt based on the theme, a generative AI (generative artificial intelligence (Generative AI)) 16 generates a product based on the designed AI prompt, the generated product is evaluated, and the design of the AI ​​prompt and the process of generating the product are distributed on the Internet, and viewers can watch it as an e-sport. In the following description, AI prompt, AI prompt engineering, and AI prompt engineer will be simply referred to as prompt, prompt engineering, and prompt engineer, respectively.

[0036] Providing a venue for competing in prompt creation techniques as an e-sports tournament where participants can watch the product generation process can contribute to the spread of prompt engineering, help train prompt engineers, and help discover excellent prompt engineers. At the same time, if the prompt skills of prompt engineers can be evaluated by a third party through the tournament, it can contribute to expanding employment opportunities for prompt engineers, and overall it can become educational entertainment that combines e-sports and skill evaluation.

[0037] In the system 10, an athlete terminal 15 operated by an athlete 12, a generative AI 16 that receives instructions from the athlete terminal, an evaluation device 11 that evaluates the product generated by the generative AI, a transmission device 13 that transmits the display screen of the athlete terminal 15, and a listener terminal 17 of a viewer 14 to which the display screen is transmitted are all connected together so as to be able to communicate with each other via a communication network 18.

[0038] (Type of competition) As the types of competition, for example, the following types of products are assumed to be generated. However, other types may also be generated. "AI manga (e.g. 4-panel manga)" as Text-to-COMIC "Product photo / logo" as Text-to-Image "AI Composition" as Text-to-Music "AI Video" as Text-to-Video "AI-generated programming" as Text-to-TEXT "Image generation AI" as Text-to-Image "Photo Art" as Text-to-Photo

[0039] (Type of topic) There are two types of topics: one where a clear goal is set and participants are asked to produce something that is almost identical to the goal, and another where a vague goal is presented and participants are asked to express it freely. In the case of the type that sets a clear goal, for example, a target image is presented as a goal, and an image that is almost identical to the target image is generated. The similarity between the target image and the generated image can be processed by AI image recognition, so the evaluation can be performed automatically and the time required for evaluation can be shortened. On the other hand, in the case of a type that presents vague goals, competitors are allowed to express themselves freely, and the evaluation method is determined by taking into account the number of votes from third parties, experts, or sponsors.

[0040] (Communications Network 18) The communication network 18 is, for example, a communication network of the Internet, and is also constructed by a mobile phone network, a wireless communication path, Ethernet (registered trademark), and the like.

[0041] (Generation system AI16) The generating AI 16 is, for example, an AI that generates new digital products such as images, videos, sounds, music, text, and codes. In this embodiment, the processing for generating products is performed on a server device on the cloud as a cloud AI. The processing may be performed on the athlete terminal, or part of the processing may be performed on the athlete terminal 15. The generative AI 16 generates text and images of a certain level of quality by, for example, machine learning a large amount of existing text and images on the Internet, and combining this with reinforcement learning. It generates products based on instructions called prompts.

[0042] (prompt) A prompt is a textual instruction or command to the generative AI. The contestant 12 inputs the prompt via an input device such as a keyboard. It may also be input by voice, converted to text, and used to command the generative AI.

[0043] [2.Each configuration] (Evaluation device 11) 2 shows one embodiment of the evaluation device 11. The evaluation device 11 mainly includes a publishing means 1 that publishes subject information 1a including a subject to be generated via the Internet (communication network) 18, a receiving means 2 that receives product information 2a including a product, and an evaluation means 3 that outputs evaluation information 3a regarding the evaluation of the product.

[0044] (Publication Method 1) The publishing means 1 publicly discloses the subject information 1a via the Internet. The publishing may be made public by uploading it to a specified web page by the evaluation device 11, or may be sent to a destination registered in advance by the contestant 12. The subject information 1a is data of text, audio, images, and videos, or a combination of multiple of these data.

[0045] (Method of reception 2) The reception means 2 receives the product information 1a created by the contestant 12. In this embodiment, the product information 1a is transmitted by the contestant 12 to the evaluation device 11, for example.

[0046] (Evaluation method 3) The evaluation means 3 outputs evaluation information 3a regarding the evaluation of the product. The evaluation is performed in reference to the subject information 1a. The evaluation is performed, for example, by calculating the similarity with the subject information 1a using AI. The evaluation may also be performed by tallying up the number of votes cast by the viewers 14. Furthermore, the similarity calculated by AI and the number of votes cast may be combined to produce the evaluation information 3a.

[0047] (Transmitter 13) The transmitting device 13 mainly comprises a screen acquisition means 4 for acquiring screen information 4a including an image of the athlete 12 generating the product and / or a screen display of the operation screen generating the product, and a transmitting means 5 for transmitting the screen information 4a.

[0048] (Screen acquisition means 4) The screen acquisition means 4 acquires screen information 4a from the athlete terminal 15. In this embodiment, the screen information 4a is transmitted to the transmission device 13 by the athlete terminal 15, for example.

[0049] (Transmission means 5) The transmission means 5 transmits the image information 4a to the viewer terminal 17. In this embodiment, the image information 4a is uploaded to a distribution server (not shown) communicatively connected to the Internet 18. The viewer 14 uses the viewer terminal 17 to view the image information 4a via the distribution server. The viewer 14 may watch the image information 4a in almost real time, or may view the image information 4a stored in the distribution server by streaming playback several minutes or hours after real time, or after the event. The image information 4a may also be downloaded and played back for viewing.

[0050] (Additional Means 6) In this embodiment, the display device further includes an adding unit 6 for adding related information 6a related to the product information 2a to the screen information 4a.

[0051] (Competitor terminal 15, viewer terminal 17) In this embodiment, the athlete terminal 15 and the viewer terminal 17 are, for example, personal computers or smartphones. However, they may also be mobile phones or tablet terminals. Athlete terminal 15 and viewer terminal 17 each include a display unit 19a and an input / output unit 19b.

[0052] (Display section 19a) As the display unit 19a (see FIG. 4), for example, a device that displays a screen, such as a liquid crystal display (LCD), a plasma display panel (PDP), or an organic electroluminescence (EL) display, is used.

[0053] (Input / output section 19b) Input / output unit 19b (see FIG. 4) is, for example, a microphone, a speaker, or a keyboard. Instead of a keyboard, text may be input by voice via a microphone, or by touching display unit 19a. Input / output unit 19b may be built into athlete terminal 15 or viewer terminal 17, or may be connected to athlete terminal 15 or viewer terminal 17 as an external device. Any other conventionally known input / output unit may be used.

[0054] (Image transmission means 7) The athlete terminal 15 is equipped with an image transmission means 7. The image transmission means 7 transmits the screen display operated by the athlete 12 via the display unit 19a as screen information 7a. In this embodiment, the voice of the athlete 12 during the competition is transmitted by a microphone 19b together with the screen information 7a.

[0055] (Voting means 8) The viewer terminal 17 is equipped with a voting means 8. The voting means 8 transmits voting information 8a to the evaluation device 11. The voting information 8a includes the number of votes, and is a concept that can be converted into the number of votes. In this embodiment, the viewer accesses the web page of the evaluation device 11 to vote. It is also possible to access the web page of a voting server to vote.

[0056] [3. Hardware configuration] Next, the hardware configurations of evaluation device 11, transmission device 12, competitor terminal 12 and viewer terminal 17 will be described with reference to FIGS.

[0057] (Hardware configuration of evaluation device 11 and transmission device 12) Next, we will explain the hardware configuration of the evaluation device 11. Note that the hardware configuration of the transmission device 12 is almost the same as that of the evaluation device 11, so the explanation will be omitted. As shown in FIG. 3, the evaluation device 11 of this embodiment uses, for example, a computer. The evaluation device 11 includes a CPU (or GPU) 30. The CPU 30 is connected to, for example, a memory (hereinafter, referred to as a storage unit) 31, a connection port 33 for connecting / reading a storage device 32, and a communication circuit 34 for communicating with the outside via a network, via a bus line 35. The storage unit 31 stores an evaluation device program 36 for processing the system 10 (a transmission device program 36a for the transmission device 13). A browser program 37 and an OS 38 (operating system) may also be stored.

[0058] (Hardware configuration of competitor terminal 12 and viewer terminal 17) The hardware configuration of the competitor terminal 12 and the viewer terminal 17 is almost the same as that of the aforementioned evaluation device 11, so the same parts are given the same reference numerals and their explanations are omitted. The competitor terminal 12 of this embodiment uses, for example, a computer. 4, display unit 19a and input / output unit 19b are connected to bus line 35 of athlete terminal 12. Also, athlete terminal program 39 (viewer terminal program 39a for viewer terminal 17) for processing system 10 is stored in storage unit 31 of the computer of athlete terminal 12. Furthermore, a browser program 37 and even an OS 38 (operating system) may also be stored.

[0059] In this embodiment, the device programs 36, 36a and the terminal programs 39, 39a may operate in cooperation with each other by utilizing the functions of the OS 38 and the browser program 37. Note that the device programs 36, 36a and the terminal programs 39, 39a may operate independently without utilizing the browser program 37 and the OS 38, respectively.

[0060] In the hardware configuration of the device programs 36, 36a and terminal programs 39, 39a described above, the functions shown in the function book diagram of FIG. 2 are realized, for example, by using a CPU 30, the device programs 36, 36a, and the terminal programs 39, 39a, but some or all of them may be sequence-controlled using a logic circuit such as a microcomputer, or a PLC (programmable logic controller).

[0061] [4. Program] (Flowchart for evaluating the product) Fig. 5 is a flow chart showing an embodiment of the processing of the evaluation device program 36 and the athlete terminal program 39 used in the evaluation device 11 and the athlete terminal 15, respectively, of the system 10. Fig. 5 is a flow chart mainly showing the process up to evaluation of the product. The flow chart shows an evaluation method 40 used in the system 10.

[0062] (S0: User registration) In this embodiment, the competitors 12 register as users before the start of the competition, and the viewers 14 register as users before watching. Note that registration may be performed after the competition has started or after watching, or the system 10 may be used without registration.

[0063] (Login Registration) A contestant 12 or a viewer 14 who wishes to use the system 10 requests the transmission of a registration form for user registration to a registration server through their respective terminals. In this embodiment, for example, a web browser function is used. The evaluation device 11 or the transmission device 13 may be used as the registration server. The registration form is entered with a user name and password used for login processing, a handle name to be used in the system 10, user information, and the like. These registration items may be registered all at once, or may be registered each time. The server acquires the information entered from the registration form. The information entered in the registration form is registered in the memory unit 31 as a personal information database (hereinafter referred to as personal information DB) 20 (see FIG. 6).

[0064] (Data structure of personal information DB20) 6 shows the data structure of the person information DB 20. In the person information DB 20 shown in the figure, for example, reference numeral 21 denotes a user's identification number (ID), reference numeral 22 denotes contact information such as the user's email address, reference numeral 23 denotes evaluation information, and reference numeral 24 denotes information on the attributes of a person (attribute information), such as job title, expert, sponsor, etc. Evaluation information is registered when a user participates as a contestant 12 (to be described later in the third embodiment). The evaluation information to be registered may record the highest score, or may be overwritten with the latest score.

[0065] (Login process) The contestant 12 and viewer 14 who wish to log in, for example, start the terminal program 39, 39a (see FIG. 3) of the system 10, respectively, and input the user name and password registered in the login information input screen. The server refers to a user registration database (not shown) and searches whether there is a registration identical to the transmitted user name and password. If there is no identical registration, the login information input screen is displayed and a request is made to re-enter. Note that the contestant 12 may also log in using a mechanism called social login. If the identical registration is confirmed, the contestant logs in to the system 10. When the contestant 12 logs in, the terminal program 39 (see FIG. 4) for transmitting the product information 1a and image information 4a is started. On the other hand, when the viewer 14 logs in, the terminal program 39a (see FIG. 4) for transmitting the voting information 7a is started.

[0066] (S1: Reveal the topic) When a predetermined time arrives, the CPU 30 (see FIG. 3) of the evaluation device 11 uploads the subject information 1a as a subject to a predetermined web page published on the Internet 12. The contestants 12 access the web page and check the subject information 1a. In this embodiment, for example, the predetermined time is stored in the storage unit 31 as a rule condition 25.

[0067] (Rule condition 25) 7 shows the data structure of a condition table showing rule conditions 25. A plurality of conditions are stored in advance in the condition table shown in the figure. In the figure, for example, conditions A and B are shown. Items of conditions that can be set include, for example, a disclosure rule 26 for the subject information 1a, a start rule 27 for the competition, a finish rule 28 for the competition, and other rules 29.

[0068] (Public Rule 26) The disclosure rules 26 relate to rules for disclosing the subject information 1a. Condition A states the upload time to the public web page. Condition B states that when the disclosure time reaches 0:00, the subject information 1a is sent to the contact information 22 of the competitor 12 that has been registered in advance. Other methods may be used for the disclosure rules 26 as long as the competitor 12 is able to confirm the subject information 1a.

[0069] (S2: Create the product) Returning to Figure 5, the contestant 12 starts the competition based on the start rule 27. The contestant 12 analyzes and evaluates the answers and suggestions obtained from the generative AI 16 via the contestant terminal 15, and designs more appropriate AI prompts using the contestant terminal 15 based on the results. The contestant terminal 15 obtains products from the generative AI 16 based on the prompts created by the contestant 12. As will be described later, when the competition starts, the athlete terminal 12 transmits the screen information 4a to the transmission device 13.

[0070] (Starting rule 27) Start rule 27 (see FIG. 7) is a rule regarding the start of the competition. Under condition A, the competition starts almost at the same time as the announcement. Under condition B, the competition starts a predetermined time after the announcement. By leaving a time, the competitor 12 is given time to prepare, and the ability of the competitor 12 is evaluated, including how the competitor 12 has prepared in advance. Other rules may be used as start rule 27, provided that the start of the competition can be confirmed by the competitor 12. Whether or not start rule 27 is being observed can be confirmed from the screen information 4a of the competitor 12. The screen information 4a may be transmitted to the transmitting device 13 at or before the start.

[0071] (S3: Send the product, S4: End reception) 5, the contestant 12 transmits the completed product information 2a to the evaluation device 11. At this time, instruction information 42a including a prompt may also be transmitted to the evaluation device 11. Next, the evaluation device 11 finishes accepting the product information 2 a based on, for example, the rule condition 25 .

[0072] (End Rule 28) The end rule 28 (see FIG. 7) is a rule regarding the end of the competition. Under condition A, the competition ends when the end time is reached, that is, when a predetermined time has elapsed or a predetermined time has arrived. In this embodiment, when the competition ends, the evaluation means 3 stops accepting product information 2a. Alternatively, the transmission time of the product information 2a from the terminal 15 or the reception time of the evaluation device 11 may be checked, and product information 2a that has exceeded the end time may not be moved to the processing of the next step S5. In condition B, the test ends five minutes after it started. This is to measure the ability to improvise by having the participants create the test in a short period of time. The test may be shorter or longer than five minutes. If it is longer, it is possible to measure the ability to carefully consider the test. On the other hand, it is best to set the time so that the viewers 14 do not become bored.

[0073] (S5: Evaluate the product) Returning to Fig. 5, the evaluation device 11 outputs evaluation information 3a that evaluates the product information 2a. In this embodiment, the evaluation information 3a is sent to the athlete terminal 15. The evaluation information 3a may be made public on a predetermined web page, or may be sent to a pre-registered destination.

[0074] (Numerical analysis of similarity) In this embodiment, the subject is "generate an image similar to a published target image." The evaluation device 11 calculates the similarity between the subject information 1a and the product information 2a. A matching technique is used to calculate the similarity of the images. Matching is a process of searching for a certain pattern image from within the entire image. Matching techniques include, for example, template matching and feature matching. Note that the matching technique is not particularly limited. In this embodiment, for example, feature matching is performed to evaluate the similarity. For feature detection, for example, the AKAZE (Accelarated KAZE) method is used. Feature detection involves extracting characteristic parts from an image. For feature matching, for example, a Brute-Force Matcher method is used.

[0075] (Other evaluation methods) The similarity may be acquired by analyzing a plurality of viewpoints. For example, if the subject information is an image, the viewpoints may be color, touch, composition, shape, etc. For example, the evaluation information may be an average value of the similarities in the plurality of viewpoints, a total of the similarities in all viewpoints, or any other known evaluation technique. The color similarity may be determined, for example, by taking the color difference of a pixel in the two images being compared as the pixel similarity, calculating the pixel similarity for all pixels, and adding up the similarities of all pixels to determine the similarity between the two images. The touch similarity is calculated, for example, by calculating the difference in line thickness from a specific pixel of the two images to be compared. The similarity is calculated for all pixels, and the similarity between the two images is calculated by adding up all the similarities. In addition, a conventionally known method of analyzing the similarity of the composition, shape, and the like may also be used.

[0076] In addition to the similarity, the evaluation may be based on voting information 7a, which is a numerical representation of the number of votes from a third party. In the case of voting, it is suitable for evaluation of a type of contest that presents a vague goal (subject) and allows the contestant to express it freely. By vague, it is meant that the atmosphere and image of the final product are suggested, without giving specific instructions. The similarity and the voting information 7a may also be combined to form the evaluation information. The numerical values ​​may be weighted based on the attribute information 24 (see FIG. 6) entered at the time of login registration, for example, so that votes from experts or sponsors receive a higher evaluation point.

[0077] (Flowchart of how viewers watch / vote) Fig. 7 is a flow chart showing an embodiment of the processing of the transmission device program 36a, the athlete terminal program 39, and the viewer terminal program 39a used respectively in the transmission device 13, the athlete terminal 15, and the viewer terminal 17 of the system 10. Fig. 7 is a flow chart mainly showing the process up to when a viewer casts a vote. The flow charts shown in Figs. 5 and 7 show a watching method 41 used in the system 10.

[0078] (T1) The athlete terminal 15 transmits image information 4a. The transmission device 13 acquires the image information 4a. The athlete 12 sets the screen information 4a in a state in which it can be transmitted to the transmission device at or before the start of the competition.

[0079] (T2) Related information 6a is added to image information 4a. In this embodiment, for example, audio information by a prompt engineering expert and / or a presenter, and further video information of the expert and presenter are added to screen information 4a as related information 4a.

[0080] (T3) The transmitting device 13 publishes the screen information 4a to which the related information 6a has been added on a predetermined server on the Internet. The viewer 14 accesses the server via the viewer terminal 17 to view the screen information 4a. The transmitting device 13 may be a server for distributing the screen information 4a to which the related information 6a has been added.

[0081] (T4) Voting information 7a based on the input of the viewer 14 is transmitted to the evaluation device 11. In this embodiment, for example, the voting information 7a, which is the vote for the contestant 12 to be evaluated by the viewer 14, is transmitted to the evaluation device 11 via a voting web page downloaded from a predetermined voting web server.

[0082] 5. Other embodiments Next, a description will be given of other embodiments of the system 10. The modifications and other embodiments described below are substantially similar to the system 10 described above, so the same parts are given the same reference numerals and the description thereof will be omitted.

[0083] (System 10 Variation 1) Modifications of the system 10 will now be described. In Modification 1, the subject information 1a is not made public by the evaluation device 11. For example, it is made public through a third party such as radio or television. Furthermore, it may be made public by a method other than digital data, for example, by using type or photographs printed on the pages of newspapers and magazines.

[0084] (System 10 Variation 2) In variant 2, a prompt to be input to the generative AI is generated by the second generative AI based on the input of the second prompt. The contestant 12 designs the second prompt. For example, if the generative AI (e.g., the first generative AI) 16 generates an image, the second generative AI designs a prompt to be input to the first generative AI. In addition, the number of generative AIs can be increased in a chain-like manner, such as by using a third generative AI to design the second prompt to be input to the second generative AI.

[0085] (System 10 Variation 3) In the third modification, the viewer terminal 17 does not include the voting means 8. The evaluation means 3 outputs the evaluation information 3a without relying on the votes of the viewer 14. For example, when the theme is "Create an image similar to the target image", the evaluation means 3 can output the similarity by numerical analysis.

[0086] Second embodiment Next, another embodiment of the system 10 will be described. Returning to Fig. 2, the system 10a according to the second embodiment includes an instruction information means 42 (see two-dot chain line) that acquires instruction information 42a including a prompt that created the product information 1a, and a storage unit 31 that associates the instruction information 42a with the product information 1a acquired from the evaluation device 11 and stores the product information 1a. The instruction information 42a is a textual instruction or command for the generative AI, called a prompt, and is a concept including information that can be converted into the prompt. The process by the instruction information means 18 corresponds to step S6 of the terminal program 39a (see FIG. 5).

[0087] Third embodiment Next, a description will be given of still another embodiment of the system 10. The system 10b according to the third embodiment includes a specific information means 43 (see two-dot chain line) that acquires specific information 43a (see FIG. 6) that identifies the athlete 12, and stores the information in the storage unit 31 in association with the corresponding evaluation information 3a. The identification information 43a is information for identifying the athlete 12, and is information recorded in the person identification DB 20. For example, it is information that can identify a person, such as ID, user name, name, My Number, pension number, driver's license number, and physical features including face, and is a concept that includes information linked to them and information that can be converted into them. The process by the specific information means 19 corresponds to step S6 of the terminal program 39a (see FIG. 5).

[0088] (Fourth embodiment) Next, other embodiments of the system 10 will be described. In a system 10c according to the fourth embodiment, the evaluation device 11 further includes a selection means 9 (see the two-dot chain line). The subject in the system 10c is to generate an image that is substantially similar to a target image. Furthermore, the evaluation information 3a in the system 10c is a similarity calculated by numerically analyzing the subject information 1a and the product information 2a. The selection means 9 selects the athlete 12 corresponding to a similarity equal to or higher than a predetermined level. The selection means 9 identifies the athlete 12 to be selected based on the identification information 43a (see reference numeral 23 in FIG. 6) of the athlete 12 linked to the evaluation information 3a. The process by the selection means 9 corresponds to step S7 of the terminal program 39a (see FIG. 5).

[0089] [5. Competition method] (Qualifying) Next, the system 10c of the fourth embodiment will be described in more detail with reference to FIG. 2 and FIG. 5. In the system 10c, the preliminary challenge is first presented with the question, "Create an image similar to a target image." The degree of similarity of the images can be calculated in a short time using the image recognition function of AI. This allows the skills of a large number of people to be evaluated in a short time in the preliminary. For example, if the contestants 12 are asked to create images in 3-5 minutes and the similarity can be calculated in a few minutes, this prevents excessive burden on the viewers 14 as a competition to watch.

[0090] (Multiple Qualifiers) After that, competitors 12 whose evaluation information 3a results are above a predetermined level are selected. Then, a final match is held with a predetermined number of people, for example, 2 to 8 people. On the other hand, similar preliminary rounds may be repeated with the selected competitors 12 to select competitors 12 with even higher skills. If the preliminary rounds are held multiple times, each round may be based on a different viewpoint. Furthermore, the evaluation information 3a obtained from multiple preliminary rounds may be totaled or the totals may be averaged, and competitors 12 may be selected in order of their performance.

[0091] (Semi-finals, finals, etc.) In the semi-finals and finals, if the theme (second theme information) is a type that presents an ambiguous goal, it is possible to encourage the contestants 12 to create products (second product information) that demonstrate their expressiveness. When evaluating expressiveness, it is preferable to use an evaluation by voting (second evaluation information). Furthermore, the evaluation by voting may be used as part of the evaluation of the similarity described above.

[0092] [6.Other] In the above embodiment, the evaluation device 11 and the transmission means 13 are provided separately, but they may be provided integrally. The items described as "other" in the above-described embodiment can be used in appropriate combination. When transmitting the product information 1a from the competitor terminal 15 to the evaluation device 11, information for identifying the competitor 12 may be transmitted separately. If the competitor 12 is logged in, the user identification number (ID) 21 is transmitted. The order of steps S6 and S7 shown in FIG. 5 may be reversed.

[0093] [7. Summary] (1)(4)(8)(9) System 10 (evaluation device 11, evaluation method 40, viewing method 41) is composed of a plurality of computers, and evaluates a product generated by a generation system AI based on prompt input from an athlete 12, and also views the process of generating the product. It is characterized by comprising evaluation device 11 having reception means 2 for receiving product information 2a including the product generated based on subject information 1a including a subject that is the target of generation, and evaluation means 3 for outputting evaluation information 3a regarding the evaluation of the product, a transmission device 13 having screen acquisition means 4 for acquiring screen information 4a including an image of the athlete generating the product and / or a screen display of the operation screen generating the product, and a transmission means 5 for transmitting the screen information. Therefore, the system 10 can objectively evaluate the skills of prompt engineers while entertaining viewers 14 (third parties) as e-sports entertainment. By providing a forum for competing in prompt creation skills, it can contribute to the spread of prompt engineering, help train prompt engineers, and help discover excellent prompt engineers. It can also contribute to expanding employment for AI prompt engineers, and overall, it can become educational entertainment that combines e-sports and skill evaluation.

[0094] (2) In such a system 10, the transmitting device 13 is provided with an adding means 6 for adding related information related to the product information 2a to the screen information, thereby helping the viewer 12 to understand the status of the production process.

[0095] (3)(5)(10) Also, since the system is provided with a publishing means 1 for publishing on the Internet the subject information 1a including the subject to be generated, the system can receive evaluations from anywhere.

[0096] (6)(11) Furthermore, since the subject generates an image that is nearly identical to the target image, and the evaluation information 3a is the similarity calculated by numerically analyzing the subject information 1a and the product information 2a, a large number of competitors can be evaluated fairly in a short period of time.

[0097] (7)(12) The system further includes a selection means 9 for selecting contestants 12 corresponding to a predetermined degree of similarity or higher, a publishing means 1 for publishing second subject information for the selected contestants to compete with, a receiving means 2 for receiving second product information generated by the selected contestants, and an evaluation means 3 for outputting second evaluation information regarding the evaluation of the second product information, so that the selected contestants can compete with each other using new second subject information.

[0098] (13) Since the generation of product information 2a from subject information 1a constitutes a preliminary round and the generation of second product information from second subject information constitutes a second round, it is possible to select skilled competitors from a large number of competitors 12 and to show a competition between the skilled competitors. [Explanation of symbols]

[0099] 1. Means of Publication 1a Subject information 2. Method of reception 2a Product information 3. Evaluation methods 3a Evaluation Information 4 Screen acquisition method 4a Screen Information 5. Transmission Method 6 Additional Means 6a Related Information 7 Screen transmission method 8 Voting means 8a Voting information 9. Selection methods 9a Instruction information 9b Specific information 10. Evaluation of products and observation system of the creation process 10a Second embodiment 10b Third embodiment 10c Fourth embodiment 11 Evaluation equipment 12 Competitors 13 Transmitting device 14 Viewers 15 Competitor Terminal 16 Generative artificial intelligence (generative AI) 17 Viewer terminal 18. Communications Networks 19a Display section 19b Input / output section 20 Person information DB 21 Identification number (ID) 22 Contact details 23 Evaluation Information 24 Person Attributes 25 Rule Conditions 26 Publication Rules 27 Starting Rules 28 Rules for Finishing the Competition 29 Other Rules 30 CPU 31 Memory 32 Recording Devices 33 Connection Ports 34 Communication Circuits 35 Bus Line 36 Evaluation Equipment Program 36a Transmitter program 37 Browser Programs 38 OS 39 Competitor Terminal Program 39a Viewer terminal program 40 Evaluation Method 41 How to watch 42 Instruction information acquisition means 42a Instruction information 43 Specific information acquisition means 43a Specific information

Claims

1. A system for evaluating a product generated by a generative AI based on an input of an AI prompt from a player, comprising a plurality of computers, and for observing the process of generating the product, an evaluation device including: a receiving means for receiving product information including a product generated based on subject information including a subject to be generated; and an evaluation means for outputting evaluation information regarding an evaluation of the product; a transmission device including a screen acquisition means for acquiring screen information including an image of the athlete generating the product and / or a screen display of an operation screen generating the product, and a transmission means for transmitting the screen information; A system for evaluating products and observing the creation process using generative AI.

2. The system for evaluating products and observing the creation process by a generative AI as described in claim 1, wherein the transmitting device is equipped with an adding means for adding related information related to the product information to the screen information.

3. A system for evaluating products and observing the creation process by generative AI as described in claim 2, further comprising a publishing means for publishing the subject information on the Internet.

4. An evaluation device for use in a system for evaluating a product generated by a generation system AI based on an input of an AI prompt from a player, the evaluation device comprising: A receiving means for receiving product information including the product to be generated based on subject information including a subject to be generated; and an evaluation means for outputting evaluation information regarding the evaluation of the product.

5. 5. The evaluation device according to claim 4, further comprising publishing means for publishing said subject information on the Internet.

6. 6. The evaluation device according to claim 4, wherein the subject generates an image that is substantially similar to an image targeted for generation, and the evaluation information is a similarity calculated by numerically analyzing the subject information and the product information.

7. The method further includes a selection means for selecting a player who corresponds to a similarity degree equal to or higher than a predetermined degree, The publishing means publishes second subject information for the selected competitors to compete; the accepting means accepts second product information generated by the selected player; 7. The evaluation device according to claim 6, wherein the evaluation means outputs second evaluation information regarding the evaluation of the second product information.

8. 1. A method for use in a system for evaluating a product generated by a generative AI based on input of AI prompts from a player, the system comprising: receiving product information including a product to be generated based on subject information including a subject to be generated; Outputting evaluation information regarding an evaluation of a product including the subject matter; A method for evaluating a product produced by a generative AI and observing the production process, which involves acquiring screen information including an image of the athlete and / or a screen display of the operation screen producing the product, transmitted from the athlete's terminal.

9. 1. An evaluation method for use in a system for evaluating a product generated by a generative AI based on an input of an AI prompt from a player, the system comprising: receiving product information including a product to be generated based on subject information including a subject to be generated; and outputting evaluation information relating to the evaluation of the product.

10. 10. The evaluation device according to claim 9, wherein the subject information is made public on the Internet.

11. 11. The evaluation method according to claim 9 or 10, wherein the subject is to generate an image that is approximately similar to an image to be generated, and the evaluation information is a similarity calculated by numerically analyzing the subject information and the product information.

12. Selecting contestants who have a similarity level equal to or higher than a predetermined level; publishing second subject information for the selected competitors to compete; accepting second product information generated by the selected contestants; The evaluation method according to claim 11 , further comprising the step of outputting second evaluation information relating to the evaluation of the second product information.

13. A preliminary step is to generate the product information from the subject information; The evaluation method according to claim 12 , further comprising the step of generating the second product information from the second subject information.

Citation Information

Patent Citations

  • Computer system, viewer terminal, and program

    JP2018171282A