Electronic devices, control methods, programs, storage media

Abstract

We provide electronic devices that use generative AI to more easily generate content that aligns with the user's intentions. [Solution] The electronic device includes a receiving means for receiving input of the characteristics of content to be generated from a user; an acquisition means for acquiring information on the output destination of the content to be generated; an input means for controlling the content generation means to input a query that includes information corresponding to the characteristics of the content to be generated received by the receiving means and the information on the output destination acquired by the acquisition means; and an output means for controlling the content generation means to output the content generated based on the query to the output destination.

Description

Technical Field

[0001] The present invention relates to an electronic device having an input system for generative AI, a control method, a program, and a storage medium.

Background Art

[0002] In recent years, generative AI (Artificial Intelligence) has become known. Generative AI is one type of machine learning model that can generate new data by utilizing learned data, and interactive AIs that can communicate in a chat format and image generation AIs that can generate images are widely used.

[0003] Patent Document 1 describes specifying a plurality of conversion candidate images included in the same category from a plurality of conversion candidate images determined according to user input. Patent Document 2 describes a printing apparatus capable of executing a printing process by appropriately selecting a server corresponding to a connection partner device.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] There is a need for a mechanism that can more easily generate content according to the user's intention using generative AI.

[0006] An object of the present invention is to provide an electronic device, a control method, a program, and a storage medium that can more easily generate content according to the user's intention using generative AI.

Means for Solving the Problems

[0007] To solve the above problems, the electronic device according to the present invention is characterized by comprising: a receiving means for receiving input of characteristics of content to be generated from a user; an acquisition means for acquiring information on the output destination of the content to be generated; an input means for controlling the content generation means to input a query including information corresponding to the characteristics of the content to be generated received by the receiving means and the output destination information acquired by the acquisition means; and an output means for controlling the content generation means to output the content generated based on the query to the output destination. [Effects of the Invention]

[0008] According to the present invention, content that aligns with the user's intent can be generated more easily using a generative AI. [Brief explanation of the drawing]

[0009] [Figure 1] This diagram shows the system configuration. [Figure 2] This is a diagram showing the block configuration of the device. [Figure 3] This is a diagram showing the block configuration of the device. [Figure 4] This is a flowchart showing the processing in the first application. [Figure 5] This is a flowchart showing the processing in the second application. [Figure 6] This is a diagram showing the queries. [Figure 7] This is a diagram showing the user interface screen. [Figure 8] This is a diagram showing the user interface screen. [Figure 9] This is a diagram showing the user interface screen. [Modes for carrying out the invention]

[0010] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims. While the embodiments describe multiple features, not all of these features are essential to the invention, and the features may be combined in any way. Furthermore, in the attached drawings, identical or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0011] Figure 1 shows an example of the configuration of the content generation system in this embodiment. In the content generation system 100, the PC 105 and the printer 101 are connected to each other so that they can communicate with one another via a wireless LAN 102. Furthermore, the PC 105 and the printer 101 can access the internet 104 via an access point 103. With such a configuration, the PC 105 and the printer 101 can communicate with a generative AI (Artificial Intelligence) server 106 that provides generative AI services connected to the internet 104. Generative AI is a machine learning model built using deep learning, and generative AI services can output more creative results for content such as images, text, videos, and audio. As an example of a generative AI service, ChatGPT, which uses a text generation language model, is known and realizes a conversational AI service that enables human-like conversation. Note that the generative AI server 106 may be a single server device or a configuration in which multiple server devices cooperate, as long as it is an external system that uses generative AI.

[0012] PC105 is an information processing device with communication functions such as wireless LAN and wired LAN. Wireless LAN is sometimes referred to as WLAN (Wireless LAN). Examples of devices used as PC105 include smartphones, notebook PCs, tablet devices, and PDAs (Personal Digital Assistants). PC105 can communicate with the printer 101 via wireless LAN 102. For example, PC105 can be instructed to perform printing or scanning functions via wireless LAN 102. Alternatively, PC105 and the printer 101 may be directly connected without going through access point 103. That is, PC105 can communicate with the printer 101 via a direct connection. A wired network may be used as part of the network between PC105, printer 101, and access point 103.

[0013] The printing device 101 is an example of a printing device having a printing function. The printing device 101 may be configured as a multifunctional printer (MFP) having a scanning function (scanner), a fax function, and a telephone function. The printing device 101 also has a communication function that allows it to communicate wirelessly with the PC 105. In this embodiment, the printing device 101 is described as an example, but a device of a different form than the printing device 101 may be used. For example, a facsimile machine, scanner, projector, mobile terminal, smartphone, notebook PC, tablet terminal, PDA, digital camera, music playback device, television, smart speaker, AR (Augmented Reality) glasses, etc., which have a communication function may be used. The printing device 101 receives print data including image data from the PC 105 connected via the access point 103, for example, and forms an image based on that data. Alternatively, the printing device 101 transmits image data read by the scanner function, for example, to the PC 105 connected via the access point 103. The printing device 101 can also communicate other control information and other data with the network connected via the access point 103.

[0014] Access point 103 is a communication device that is provided separately (externally) from PC 105 and printer 101 and operates as a WLAN base station. Access point 103 may also be referred to as the external access point 103 or external wireless base station. Communication devices with WLAN communication capabilities can communicate in WLAN infrastructure mode via access point 103. In this embodiment, PC 105 and printer 101 are also examples of communication devices. Wireless infrastructure mode may also be referred to as "wireless infrastructure mode." In other words, wireless infrastructure mode is a mode for, for example, printer 101 to communicate with PC 105 via access point 103 to which printer 101 is connected. Access point 103 communicates with communication devices that it has authorized to connect to (authenticated) and relays wireless communication between those communication devices and other communication devices. Furthermore, access point 103 is connected to a wired LAN communication network and relays communication between communication devices connected to that network and other communication devices wirelessly connected to access point 103. Furthermore, if the network configured by access point 103 uses an authentication server (i.e., access point 103 supports an authentication method that uses an authentication server), access control is performed by authenticating communication devices connected to the network in cooperation with the authentication server (not shown). Note that access point 103 may also support an authentication method that does not use an authentication server.

[0015] The PC 105 and the printing device 101 can perform wireless communication using their respective WLAN communication functions in a wireless infrastructure mode via an external access point 103 or in a peer-to-peer mode without using the external access point 103. Note that the peer-to-peer mode may be referred to as the "P2P mode" or the "wireless direct mode" for the wireless infrastructure mode. The P2P mode is, in other words, a mode for the printing device 101 to communicate directly with the PC 105 without using the access point 103. The P2P mode includes, for example, the Wi-Fi Direct (registered trademark) mode and the software access point (soft AP) mode. Note that Wi-Fi Direct (registered trademark) may be referred to as WFD. That is, the wireless direct mode can also be said to be a communication mode compliant with the IEEE 802.11 series.

[0016] FIG. 2 is a diagram showing an example of the configuration of the PC 105. The PC 105 includes a main board 220 that controls the entire device, a wireless communication unit 213 that performs WLAN communication, a display unit 212, an operation unit 211, and a short-range wireless communication unit 214 that performs wireless communication different from that of the wireless communication unit 213. The main board 220 includes, for example, a CPU 201, a ROM 202, a RAM 203, an image memory 204, a data conversion unit 205, a camera unit 206, a non-volatile memory 207, a data storage unit 208, a speaker unit 209, and a power supply unit 210. Each functional unit in the main board 220 is interconnected via a system bus 215. Also, between the main board 220 and the wireless communication unit 213 and between the main board 220 and the short-range wireless communication unit 214 are connected via, for example, dedicated buses. Also, between the main board 220 and the display unit 212 and between the main board 220 and the operation unit 211 are each connected via, for example, dedicated buses.

[0017] The CPU 201 is the system control unit and controls the entire PC 105. The operation of the PC 105 described in this embodiment is realized, for example, by the CPU 201 reading a program stored in the ROM 202 into the RAM 203 and executing it. Dedicated hardware may be provided for each process. The ROM 202 stores control programs and embedded operating system (OS) programs executed by the CPU 201. The CPU 201 executes each control program stored in the ROM 202 under the management of the embedded OS stored in the ROM 202, thereby performing software control such as scheduling and task switching. The ROM 202 also stores applications (printing applications) that generate information that the printing device 101 can interpret. Information that the printing device 101 can interpret is information corresponding to the functions that the printing device 101 can execute, and the application can set settings such as printing and scanning, or instruct the printing device 101 to execute each function. The RAM 203 is composed of SRAM (Static RAM) or the like. RAM 203 stores data such as program control variables, user-registered settings, and PC 105 management data. RAM 203 may also be used as a buffer for various tasks. Image memory 204 is composed of memory such as DRAM (Dynamic RAM). Image memory 204 temporarily stores image data received via the wireless communication unit 213 and image data read from the data storage unit 208 for processing by the CPU 201. Non-volatile memory 207 is composed of memory such as flash memory and continues to store data even when the PC 105 is powered off. Note that the memory configuration of PC 105 is not limited to the above configuration. For example, image memory 204 and RAM 203 may be shared, or data backup may be performed using the data storage unit 208. Also, while DRAM is given as an example of image memory 204, other storage media such as hard disks or non-volatile memory may be used.

[0018] The data conversion unit 205 performs analysis of data in various formats and data conversions such as color conversion and image conversion. The camera unit 206 has a function of electronically recording and encoding an image input through a lens. The image data obtained by imaging with the camera unit 206 is stored in the data storage unit 208. The speaker unit 209 performs control for realizing the function of inputting or outputting voice. The power supply unit 210 is, for example, a portable battery and controls the power supply to the device. The display unit 212 electronically controls the display content and executes control for displaying various input contents, the operation status of the PC 105, the status status, etc. The operation unit 211 executes control such as generating an electrical signal corresponding to the operation and outputting it to the CPU 201 when receiving a user operation.

[0019] The PC 105 performs wireless communication using the wireless communication unit 213 and conducts data communication with other communication devices such as the printing device 101. The wireless communication unit 213 converts data into packets and transmits the packets to other communication devices. Also, the wireless communication unit 213 restores the packets from an external other communication device to the original data and outputs it to the CPU 201. The wireless communication unit 213 is a unit for realizing communication compliant with a standard such as WLAN. The short-range wireless communication unit 214 communicates by a communication method different from the wireless communication unit 213, for example, Bluetooth (registered trademark). The configuration of the PC 105 and the main board 220 is not limited to the above. For example, each function of the main board 220 realized by the CPU 201 may be realized by a processing circuit such as an ASIC (application-specific integrated circuit), and may be realized by either hardware or software.

[0020] Figure 3 is a block diagram showing an example of the configuration of the printing device 101. The printing device 101 includes a main board 320 that controls the entire device, a USB communication unit 307, a wireless communication unit 309, a wired communication unit 310, an operation display unit 312, a power button 313, a print unit 315, and a scan unit 317.

[0021] The main board 320 is equipped with a microprocessor-type CPU 301. The CPU 301 controls the printing device 101 according to the control program stored in the ROM-type program memory 302, which is connected via the internal bus 318, and the contents stored in the RAM-type data memory 303. The operation of the printing device 101 described in this embodiment is realized, for example, by the CPU 301 reading the program stored in the program memory 302 into the data memory 303 and executing it. The CPU 301 controls the scan control unit 316 to optically read the document in the scan unit 317 and stores the read data in the image memory in the data memory 303. The scan control unit 316 is an interface for connecting the scan unit 317 and the main board 320, and performs functions such as converting the format of the scanned image data. The CPU 301 controls the print control unit 314 to print the image of the read data stored in the image memory in the data memory 303 onto a recording medium in the print unit 315 (copy function). The print control unit 314 is an interface for connecting the print unit 315 and the main board 320, and performs image data conversion, etc. The data conversion unit 304 performs data conversion such as analysis of various data formats, color conversion, and conversion from image data to print data. The code decoding processing unit 305 performs encoding and decoding processing of image data (JPEG, PNG, etc.) handled by the printing device 101, as well as scaling processing.

[0022] The CPU 301 controls the USB communication unit 307 via the USB communication control unit 306 to perform USB communication with the external PC 105 via a USB connection. The CPU 301 controls the operation control unit 311 to receive operation information from the power button 313 and the operation display unit 312. The CPU 301 controls the operation control unit 311 to display, for example, the status of the printer 101 and the function selection menu on the operation display unit 312. The CPU 301 controls the wireless communication unit 309 and the wired communication unit 310 via the communication control unit 308 according to the operation information received by the operation display unit 312. For example, the CPU 301 changes the communication method settings and configures network connection settings according to the operation information.

[0023] The wireless communication unit 309 is a unit capable of providing WLAN communication functionality. Specifically, the wireless communication unit 309 converts data into packets according to the WLAN standard and transmits the packets to other communication devices. The wireless communication unit 309 also restores packets from other external communication devices back to their original data and outputs it to the CPU 301. The wireless communication unit 309 is configured to perform data (packet) communication in a WLAN system compliant with, for example, the IEEE 802.11 standard series (IEEE 802.11a / b / g / n / ac / ax, etc.). However, it is not limited to this configuration, and the wireless communication unit 309 may also be capable of performing communication in WLAN systems compliant with other standards. Furthermore, the wireless communication unit 309 can perform communication in WFD mode, P2P mode, wireless infrastructure mode, etc. Note that the PC 105 and the printer 101 are capable of wireless communication based on WFD mode, and the wireless communication unit 309 has a soft AP function or a group owner function. In other words, the wireless communication unit 309 is capable of constructing a communication network in P2P mode and determining the channel to be used for communication in P2P mode.

[0024] The wired communication unit 310 is a unit for performing wired communication. The wired communication unit 310 can perform data (packet) communication in a wired LAN (Ethernet) system compliant with, for example, the IEEE 802.3 series. Furthermore, wired communication using the wired communication unit 310 is possible in wired communication mode. The wired communication unit 310 is connected to the main board 320 via a bus cable or the like.

[0025] The printer 101 has an OCR (Optical Character Recognition) function that recognizes character information on an image read by the scanning function and extracts text data. The OCR function may be implemented by, for example, the scan control unit 316. The OCR function creates data in a text-searchable file format. Such data may include PDF, XPS (XML Paper Specification), etc. The printer 101 can save (or send) the file generated by the OCR function to a specified internal or external storage location. Instructions for setting and executing such OCR functions may be made on the operation panel on the printer 101, or they may be made to the printer 101 from an application installed on the PC 105. The printer 101 is not limited to the configuration shown in Figure 3, and may have a configuration appropriate to the functions that the device to which it is applied as a printer 101 can perform.

[0026] Figure 4 is a flowchart showing the processing performed by a predetermined application (hereinafter referred to as the predetermined app) in this embodiment. The predetermined app is, for example, a printing app that has a printing function to generate an image and print it on the printing device 101. In this embodiment, the predetermined app and the generation AI app are assumed to be installed on the PC 105. In other words, the operation of the predetermined app and the generation AI app is the operation of the PC 105. However, the predetermined app and the generation AI app may also be programs installed on the printing device 101. In that case, in other words, the operation of the predetermined app and the generation AI app is the operation of the printing device 101.

[0027] Furthermore, in this embodiment, a generative AI application is installed on the PC105 as an application that can cooperate with a predetermined application. The generative AI application may be embedded in the predetermined application or installed separately from the predetermined application. In this embodiment, cooperation between the predetermined application and the generative application is achieved by storing data from each application in a cooperation area reserved in the operating system (OS). For example, the generative AI application is started when the predetermined application stores a startup instruction to start the generative AI application in the cooperation area. Also, for example, the predetermined application can acquire data by storing data acquired by the generative application in the cooperation area. Also, for example, the generative application can acquire data by storing data acquired by the predetermined application in the cooperation area (this is also called data transfer in this embodiment). In this way, cooperation between the predetermined application and the generative application becomes possible. In this embodiment, cooperation between the predetermined application and the generative AI application is described as being performed via a cooperation area reserved in the OS, but cooperation between the predetermined application and the generative application may be performed without going through the cooperation area. For example, in a configuration where a generation application is embedded in a designated application, the designated application may be configured to directly instruct the generation application to start.

[0028] A generative AI application is, for example, an application that is called (launched) when a user selects a predetermined item related to image generation on a designated application. The generative application displays a screen like the one shown in Figure 7, which will be described later, and accepts input of image features from the user. This input can be text input or voice input. For example, as text input, the application accepts input such as "A landscape painting with windmills" in response to a dialog screen that asks "What kind of image would you like to generate?". The generative AI application is an application that can communicate with an external generative AI server 106. When the generative AI application obtains input of the image features to be generated from the user, it creates a query to send to the generative AI server 106. This query is automatically accompanied by information (resolution, image size, etc.) necessary to enable the image to be output by an output device (printer, display device, etc.), without user intervention. The created query is then sent to the generative AI server 106. The generative AI server 106 generates an image based on the query sent from PC 105 and sends the generation result to PC 105. The generative AI application receives the generation results from the generative AI server 106, and further, a designated application can acquire and output those generation results.

[0029] In this embodiment, the application creates a query to send to the generation AI server 106 based on the image features input by the user. During the creation of this query, information necessary to enable image output by the output device is automatically added without user intervention. This provides the following benefits. For example, consider a case where the user inputs the features of the image to be generated in the application, sends that input to the generation AI server to generate an image, and then uses that image in the application. In this case, there is a possibility that an image that does not conform to the application's intended use may be generated. For example, if the application intended to print borderless on a postcard, an image with a different aspect ratio may be generated. In this case, if the generated image is used as is, problems may arise such as not being able to print borderless across the entire postcard, or even if it is printed borderless across the entire postcard, a large portion of it being cropped and deleted. Furthermore, if an image with an aspect ratio that matches the postcard needs to be created again, the user has to go through the trouble of inputting detailed conditions such as the aspect ratio and requesting image generation again. According to this embodiment, when creating a query, information necessary to enable image output by the output device is automatically added without user intervention. Therefore, the user only needs to input the features of the image they want to generate, thus improving user convenience.

[0030] The process shown in Figure 4 is achieved, for example, by the CPU 201 reading a program stored in the computer-readable ROM 202 into the RAM 203 and executing it. The process shown in Figure 4 begins when the PC 105 is started up, the home screen is displayed on the display unit 212, and the system is ready to accept user selections of various applications.

[0031] In S400, the CPU 201 identifies the application selected by the user via the display unit 212. If the application selected by the user is a predetermined application, the subsequent processing is executed. On the other hand, if the application selected by the user is not a predetermined application, the process shown in Figure 4 is terminated, and the application is launched. The explanation for the case where the application selected by the user is not a predetermined application is omitted.

[0032] In S401, CPU201 determines whether or not a call to a generative AI application has been made. For example, in S401, if the user selects a predetermined item, it is determined that a call to a generative AI application has been made. If it is determined that a call to a generative AI application has been made, the process proceeds to S402. On the other hand, if it is determined that there has been no call to a generative AI application, the process shown in Figure 4 is terminated.

[0033] In S402, CPU201 launches the generative AI application. Specifically, for example, CPU201 launches the generative AI application by storing a launch instruction to launch the generative AI application in the area allocated in the operating system (OS) for inter-application communication. After processing in S402, the screen is displayed by the generative AI application, which will be described later in Figure 5, and user operations are performed on that screen.

[0034] In S403, CPU201 waits for the processing results of the generative AI application. Specifically, for example, the application may receive a notification from the OS indicating that the generative AI application has completed its processing.

[0035] In S404, the CPU 201 acquires and outputs the processing result (generation result) of the generative AI application. Specifically, for example, first, the generative AI application displays the image received as a generation result from the generative AI server 106 on the display unit 212. Then, the generative AI application stores the information of the image selected by the user on that screen in the OS's communication area. A predetermined application, triggered by receiving a notification from the OS indicating that the generative AI application's processing is complete, acquires and outputs the image information (image data, etc.) from the OS's communication area as the processing result of the generative AI application. At this time, the output device to which the output is sent is, for example, a printer 101 or a display device such as a display. Processing such as color space conversion and resolution conversion is performed on the image data etc. acquired from the OS's communication area, depending on the output destination.

[0036] Figure 5 is a flowchart showing the process executed by the generative AI application in this embodiment. The process in Figure 5 is realized, for example, by the CPU 201 reading a program stored in the computer-readable ROM 202 into the RAM 203 and executing it. The process in Figure 5 starts when the PC 105 is powered on and a predetermined application is being used by the user.

[0037] In the S500, CPU201 launches a generative AI application when a launch command is received from a designated application. Specifically, for example, the generative AI application is launched based on the fact that a launch command for the generative AI application has been stored in the OS's communication area from the designated application.

[0038] In S501, the CPU 201 receives user input via the display unit 212. Specifically, for example, a generative AI application displays a UI screen 700 on the display unit 212, as shown in Figure 7. As shown in Figure 7, the UI screen 700 displays the message "What kind of image would you like to generate?" and an input area 701 for receiving the characteristics of the image the user wants to generate. Figure 7 shows an example where the input area 701 receives input such as "A landscape painting with windmills."

[0039] In S502, CPU201 refers to RAM203 to obtain information held by a specified application. Specifically, for example, a generative AI application may access the OS's communication area where the information held by the specified application is stored, or it may be triggered by a notification from the OS to obtain the information held by the specified application.

[0040] For example, the OS's integration area may be divided for each application. When a designated application is launched by user operation or other means, it may store information that identifies itself and information that should be passed on to a generative AI application in the OS's integration area. Here, the information that should be passed on to the generative AI application is the information held by the designated application. Information that identifies the designated application may include, for example, the application name and the type of application (whether it is a printing application, an SNS (Social Networking Service) application, an image viewing application, etc.).

[0041] When the application type indicates a printing application, the information held by the specified application is, for example, information about the colorants used for printing, which are installed in the printing device 101 that is registered with the application and with which the PC 105 can communicate. Such information might include, for example, that there is little cyan ink and sufficient amounts of magenta, yellow, and black ink. In addition, the information held by the specified application is, for example, information about the paper set in the printing device 101 (including at least one of the type, size, and aspect ratio).

[0042] If the app type indicates a social networking app, the information held by the app would include, for example, the conditions for images posted to the app (aspect ratio, image size limits, etc.). If the app type indicates an image viewing app, the information held by the app would include, for example, the image size and the capabilities of the output device (display, etc.) that will display the image.

[0043] In S503, CPU201 creates a query to send to the generation AI server 106 based on the user input obtained in S501 and the information held by the designated application obtained in S502, and sends the created query to the generation AI server 106.

[0044] Figure 6 shows an example of a query created in S503. The entire code shown in Figure 6 is created as a query. However, the two-digit numbers in each line of Figure 6 are for illustrative purposes only and do not constitute the actual code. Also, the strings after "#" in Figure 6 are comments indicating the meaning of the code and do not constitute the actual code.

[0045] The "01" and "03" lines in Figure 6 indicate that packages related to generative AI tools are loaded and made available for use. For example, the package names of open-source code are listed.

[0046] In line "05" of Figure 6, the variable name for storing the return value of the generate function is specified. The name is not fixed to "response"; it can be anything.

[0047] In line "06" of Figure 6, the image generation AI model is specified.

[0048] In line "07" of Figure 6, the prompt is specified. The content entered in input area 701 of Figure 7 is specified as the prompt. That is, the user input content obtained in S501 is specified.

[0049] In lines "08" and "09" of Figure 6, the information held by the predetermined application acquired in S502 is specified. In Figure 6, it is assumed that the predetermined application is a printing application. Therefore, the image size corresponding to the paper size and the image quality set in the printing device 101 are specified. In this way, in this embodiment, the setting information necessary to output the image to the output device is automatically added to the query without user operation. For example, the setting information necessary to display the image on a display device (image size, aspect ratio, etc.) is automatically added to the query. Also, for example, the setting information necessary to print the image on the printing device 101 (image quality, paper size, etc.) is automatically added to the query. With such a configuration, the user can acquire an image from the generation system AI server 106 that satisfies the setting conditions for outputting the image on the device, simply by specifying the characteristics of the image as content.

[0050] In line "10" of Figure 6, the number of images to be requested from the generation AI server 106 is specified. For example, in Figure 6, 4 images are specified. The number of images may also be a predetermined value.

[0051] In line "13" of Figure 6, the URL of the image created by the generative AI server 106 is specified.

[0052] In line 14 of Figure 6, the image output format is specified according to the type of application. For example, in Figure 6, printing the image is specified.

[0053] When a query is sent via S503, the generative AI server 106 generates images based on the sent query. The number of images specified in the query is generated. The generative AI server 106 then sends information about the generated images to the PC 105. Here, the generated images are images. These images can be still images or moving images. Furthermore, the information about the generated images can be the image data itself or a URL indicating the location of the image data.

[0054] In S504, the CPU 201 receives information about the generated product, which is the result of the transmitted query. Specifically, for example, it receives still image data as the information about the generated product. Then, the CPU 201 displays the generated product on the display unit 212 based on the received information about the generated product.

[0055] Figure 8 shows an example of a screen displayed in S504. Screen 800 includes image 801. Image 801 is an image generated by the generation AI server 106 based on the characteristics of the image to be generated, which have been received by a predetermined application. There may be one or more such images. Figure 8 shows an example in which four images have been generated by the generation AI server 106. A checkbox 802 is provided in the upper left corner of each image, allowing the user to select the image. A message prompting the user to select an image is then displayed on screen 800. In Figure 8, a message such as "Please select the image to use" is displayed. When the user selects the desired image 801 using the checkbox 802 and presses the OK button 803, the information of the selected image 801 is stored in the OS's communication area. Screen 800 also displays a button 804 for receiving instructions to run the image generation again, and a button 805 for adding characteristics to the image to be generated.

[0056] Image 801 satisfies both the content input by the user and the conditions for outputting the image by the device. Specifically, for example, Image 801 is a landscape painting depicting a windmill, input by the user in the input area 701, and is an image that satisfies the conditions (print settings) for a predetermined application, which is a printing application, to print on the printing device 101. For example, the query is automatically specified to match the aspect ratio of a postcard, which is the print output, so the aspect ratio of the generated image will be a ratio that matches a postcard (e.g., 100mm x 148mm). Therefore, the user can use the generated image as is, that is, without cropping, and print it nicely on the postcard without any borders. Thus, it is possible to suppress situations where it becomes impossible to print on the postcard without any borders, or where a large part is cropped and deleted even if it is printed without any borders on the postcard. In this way, according to this embodiment, a suitable image can be generated more easily using a generative AI application.

[0057] In S505, the CPU 201 determines whether or not to perform image generation again based on the user operation received via the display unit 212. Specifically, for example, if button 804 on screen 800 is pressed, it is determined to perform image generation again. If it is determined to perform image generation again, the process from S501 is repeated. On the other hand, if it is determined not to perform image generation again, the process proceeds to S506. At this time, the features that were input in input area 701 are maintained.

[0058] In S506, the CPU 201 determines whether or not to add image features based on user input received via the display unit 212. Specifically, for example, if button 805 on screen 800 is pressed, it is determined to add image features. If it is determined to add image features, the process proceeds to S507. On the other hand, if it is determined not to add image features, the process proceeds to S508. The process proceeds to S508, for example, when the user selects a desired image 801 using checkbox 802 and presses the OK button 803.

[0059] In S507, the CPU 201 receives input of image features from the user via the display unit 212. Specifically, for example, the CPU 201 displays a UI screen 900 on the display unit 212, as shown in Figure 9. As shown in Figure 9, the UI screen 900 displays the message "What features would you like to add?" and an input area 901 for receiving additional image features that the user wants to generate. Figure 9 shows an example where the input area 901 receives input such as "at dusk". In this way, the user can view an image that has been generated once, input additional features, and have the image generation run again.

[0060] In S508, the CPU 201 determines whether or not the confirm button 803 was pressed. If it is determined that the confirm button 803 was not pressed, the process from S505 is repeated. If it is determined that the confirm button 803 was pressed, the process proceeds to S509.

[0061] In S509, the CPU 311 stores the information of the selected image 801 in the OS's communication area. This allows the OS to notify a designated application, which can then retrieve the information of the selected image 801 from the OS's communication area.

[0062] The example above illustrates how to automatically add a condition to a query based on the paper information set in the printing device (e.g., the aspect ratio of a postcard). However, this is not the only example; any output condition that a generative AI application can obtain in advance can also be automatically added to the query. For example, based on the fact that a given application is a specific type of social networking service (SNS) application, the query could be automatically updated to generate images with a 1:1 aspect ratio.

[0063] Furthermore, the specified application may add queries that depend on the status of the device that outputs the image. For example, if the specified application stores information indicating that the cyan ink in the communicable printer 101 is low, the query may be modified to exclude cyan ink from the colors that make up the output image. Alternatively, the query may be modified to limit the colors that make up the output image to magenta, yellow, black, white, and mixtures thereof. If the paper size set in the communicable printer 101 is A4, the query may be automatically modified to generate the image with an aspect ratio of A4. Additionally, if the specified application stores information that the output destination for the generated image is a display with a 16:9 aspect ratio and 1920 pixels x 1080 pixels (i.e., output destination capability information) in the OS's communication area, the aspect ratio and pixel count (resolution) may be automatically modified to modify the query. Furthermore, if the specified application stores information about the number of colors and dynamic range that can be reproduced at the output destination in the OS's communication area, the number of colors and dynamic range may be modified to modify the query. Similarly, if a designated application stores information obtained from the capabilities of the output destination, such as brightness, bit depth, frame rate, color gamut, and whether it can display in HDR (High Dynamic Range) or SDR (Standard Dynamic Range), in the OS's communication area, that information may be automatically added to the query.

[0064] Furthermore, this embodiment is not limited to generative AI systems that generate images, but can also be applied to generative systems that generate other types of content. For example, in a configuration that sends a query to a generative AI system (generative AI server) that generates audio, in addition to the prompts entered by the user, information such as the dynamic range and time may be acquired as information held by a predetermined application and automatically added to the query. Also, for example, in a configuration that sends a query to a generative AI system that generates videos, in addition to the prompts entered by the user, information such as the time limit as information held by a predetermined application as information about the output destination (for example, a video posting site to which the output will be placed) may be acquired as information held by a predetermined application and automatically added to the query. Also, for example, in a configuration that sends a query to a generative AI system that generates text, in addition to the prompts entered by the user, information such as the character limit and usable character types may be acquired as information about the output destination and automatically added to the query.

[0065] Furthermore, the various controls described above, which are assumed to be performed by the CPU, may be performed by a single piece of hardware, or multiple pieces of hardware (for example, multiple processors or circuits) may share the processing to control the entire device.

[0066] Furthermore, although the present invention has been described in detail based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention. Moreover, each of the embodiments described above is merely one embodiment of the present invention, and it is possible to combine each embodiment as appropriate.

[0067] Furthermore, although the above-described embodiments explained the application of the present invention to a printing device 101 and a PC 105 as an example, the invention is not limited to this example and can be applied to any electronic device capable of running a predetermined application. In other words, the present invention can be applied to personal computers, PDAs, mobile phone terminals, portable image viewers, printer devices equipped with displays, digital photo frames, e-book readers, cameras, and the like.

[0068] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0069] This embodiment includes the following electronic devices, methods, programs, and storage media. (Item 1) A means of receiving input from the user regarding the characteristics of the content to be generated, An acquisition means for acquiring information on the output destination of the content to be generated, An input means controls the content generation means to input a query that includes information representing the characteristics of the content to be generated, which has been received as input by the receiving means, and information about the output destination, which has been acquired by the acquisition means. An output means that controls the content generated by the content generation means based on the query to be output to the output destination, An electronic device characterized by having the following features. (Item 2) The electronic device according to item 1, characterized in that the content is an image. (Item 3) The electronic device according to item 2, characterized in that the output destination information includes the aspect ratio. (Item 4) The electronic device according to item 3, characterized in that the query is a query that requests the content generation means to generate an image corresponding to the aspect ratio. (Item 5) The aforementioned output destination is a printer. The electronic device according to item 3 or 4, characterized in that the aspect ratio is information based on the paper set in the printer. (Item 6) The aforementioned output destination is an SNS (Social Networking Service) application. The electronic device according to item 3 or 4, characterized in that the aspect ratio is information based on conditions for posting to the SNS application. (Item 7) The electronic device according to item 2, characterized in that the output destination information includes resolution. (Item 8) The electronic device according to item 7, characterized in that the query is a query that requests the content generation means to generate an image corresponding to the resolution. (Item 9) The electronic device according to item 2, characterized in that the output destination information includes information regarding at least one of the following: paper, colorants, dynamic range, brightness, bit depth, frame rate, color gamut, whether or not it can be displayed in HDR (High Dynamic Range), and whether or not it can be displayed in SDR (Standard Dynamic Range). (Item 10) The aforementioned image is an electronic device according to item 2, characterized in that it includes still images and moving images. (Item 11) The electronic device according to item 1, characterized in that the content includes audio. (Item 12) The electronic device according to item 10 or 11, characterized in that the output destination information includes information related to time. (Item 13) The electronic device described in item 1, characterized in that the content includes text. (Item 14) The electronic device according to item 13, characterized in that the output destination information includes information regarding at least one of the following: character limits and usable characters. (Item 15) The electronic device has a first application and a second application that is different from the first application. The first application has the output means, The second application comprises the receiving means, the acquisition means, and the input means. The output destination information is information held by the first application. An electronic device as described in any one of items 1 to 14, characterized by the features described herein. (Item 16) The first application stores the output destination information in the storage means of the electronic device, The acquisition means acquires the output destination information stored in the storage means. The electronic device described in item 15, characterized by the features described herein. (Item 17) The electronic device according to item 16, characterized in that the first application acquires content generated by the content generation means, and the output means is controlled to output the acquired content. (Item 18) The second application stores the content generated by the content generation means in the storage means. The first application retrieves the content generated by the content generation means, which is stored in the storage means. The electronic device described in item 17, characterized by its features. (Item 19) The electronic device according to any one of items 16 to 18, characterized in that the storage means is included in an operating system that is different from the first application and different from the second application. (Item 20) The electronic device according to any one of items 1 to 19, characterized in that the content generation means is a generation AI (Artificial Intelligence) system. (Item 21) The electronic device according to any one of items 1 to 20, characterized in that the content generation means is a system located outside the electronic device. (Item 22) A method for controlling electronic equipment that is performed in an electronic device, A reception process that receives input from the user regarding the characteristics of the content to be generated, The acquisition step involves obtaining information about the output destination of the content to be generated, An input step controls the content generation means to input a query that includes information corresponding to the characteristics of the content to be generated, which was received in the reception step, and the information of the output destination obtained in the acquisition step. An output step that controls the content generated by the content generation means based on the query to be output to the output destination, A control method characterized by having the following features. (Item 23) A program for causing a computer to function as one of the means of electronic equipment described in any one of items 1 through 21. (Item 24) A computer-readable storage medium on which a program for causing a computer to function as one of the means of electronic equipment described in any one of items 1 through 21 is stored.

[0070] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of Symbols]

[0071] 101 Printing devices: 105 PCs: 106 Generative AI servers: 201, 301 CPUs

Claims

1. A means of receiving input from the user regarding the characteristics of the content to be generated, An acquisition means for acquiring information on the output destination of the content to be generated, An input means controls the content generation means to input a query that includes information representing the characteristics of the content to be generated, which has been received as input by the receiving means, and information about the output destination, which has been acquired by the acquisition means. An output means that controls the content generated by the content generation means based on the query to be output to the output destination, An electronic device characterized by having the following features.

2. The electronic device according to claim 1, characterized in that the content is an image.

3. The electronic device according to claim 2, characterized in that the output destination information includes the aspect ratio.

4. The electronic device according to claim 3, characterized in that the query is a query that requests the content generation means to generate an image corresponding to the aspect ratio.

5. The aforementioned output destination is a printer. The electronic device according to claim 3, characterized in that the aspect ratio is information based on the paper set in the printer.

6. The aforementioned output destination is an SNS (Social Networking Service) application. The electronic device according to claim 3, characterized in that the aspect ratio is information based on conditions for posting to the SNS application.

7. The electronic device according to claim 2, characterized in that the output destination information includes resolution.

8. The electronic device according to claim 7, characterized in that the query is a query that requests the content generation means to generate an image corresponding to the resolution.

9. The electronic device according to claim 2, characterized in that the output destination information includes information relating to at least one of the following: paper, colorants, dynamic range, brightness, bit depth, frame rate, color gamut, whether or not it can be displayed in HDR (High Dynamic Range), and whether or not it can be displayed in SDR (Standard Dynamic Range).

10. The electronic device according to claim 2, characterized in that the aforementioned image includes a still image and a moving image.

11. The electronic device according to claim 1, characterized in that the content includes audio.

12. The electronic device according to claim 10, characterized in that the output destination information includes information about time.

13. The electronic device according to claim 1, characterized in that the content includes text.

14. The electronic device according to claim 13, characterized in that the output destination information includes information regarding at least one of the following: character limits and usable characters.

15. The electronic device has a first application and a second application that is different from the first application. The first application has the output means, The second application comprises the receiving means, the acquisition means, and the input means. The output destination information is information held by the first application. The electronic device according to feature 1.

16. The first application stores the output destination information in the storage means of the electronic device, The acquisition means acquires the output destination information stored in the storage means. The electronic device according to feature 15.

17. The electronic device according to claim 16, characterized in that the first application acquires content generated by the content generation means, and the output means is controlled to output the acquired content.

18. The second application stores the content generated by the content generation means in the storage means. The first application retrieves the content generated by the content generation means, which is stored in the storage means. The electronic device according to feature 17.

19. The electronic device according to claim 16, characterized in that the storage means is included in an operating system that is different from the first application and different from the second application.

20. The electronic device according to claim 1, characterized in that the content generation means is an Artificial Intelligence (AI) generation system.

21. The electronic device according to claim 1, characterized in that the content generation means is a system located outside the electronic device.

22. A method for controlling electronic equipment that is performed in an electronic device, A reception process that receives input from the user regarding the characteristics of the content to be generated, The acquisition step involves obtaining information about the output destination of the content to be generated, An input step controls the content generation means to input a query that includes information corresponding to the characteristics of the content to be generated, which was received in the reception step, and the information of the output destination obtained in the acquisition step. An output step that controls the content generated by the content generation means based on the query to be output to the output destination, A control method characterized by having the following features.

23. A program for causing a computer to function as one of the means of the electronic device according to any one of claims 1 to 21.

24. A computer-readable storage medium having a program stored in it that causes a computer to function as one of the means of the electronic device according to any one of claims 1 to 21.

Images