Non-transitory computer-readable storage medium, information processing apparatus, method for controlling information processing apparatus

Abstract

To appropriately convert design data.SOLUTION: A program executed by a second information processing apparatus capable of communicating with a first information processing apparatus causes the second information processing apparatus to function as display control means for displaying converted design data obtained by translating a first element, which is a text element included in design data, and converting a second element, which is a non-text element included in the design data, on the basis of control executed by the first information processing apparatus.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present disclosure relates to a program, an information processing device, a control method for an information processing device, and an information processing system. [Background technology]

[0002] There are systems that support the creation of designs such as posters and flyers. Some of these systems have a function for translating text in design data. In this case, the text in the design data is acquired and translated by machine translating the acquired text into a language specified by an operator. Patent Document 1 discloses a technology for acquiring and translating text in the design data, as well as converting original text elements that are visualized in the design data into text elements corresponding to the translation language. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-026398 Summary of the Invention [Problem to be solved by the invention]

[0004] Techniques for more appropriately converting design data are desired. [Means for solving the problem]

[0005] The program disclosed herein is a program executed by a second information processing device capable of communicating with a first information processing device, and is characterized in that it causes the second information processing device to function as a display control means that, based on control executed by the first information processing device, translates a first element, which is a text element included in design data, and displays converted design data in which a second element, which is a non-text element included in the design data, is converted. [Effects of the Invention]

[0006] According to the technology of the present disclosure, design data can be appropriately converted. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing the overall configuration of the present system. [Figure 2] Hardware configuration diagram of the operator computer (PC) and service server. [Figure 3] Software configuration diagram of this system. [Figure 4] FIG. 4 is a diagram showing an example of a data table of design data. [Figure 5] FIG. 1 is a diagram showing an example of processing a multimodal large-scale language model. [Figure 6] FIG. 10 is a diagram showing an example of processing of an image generation model. [Figure 7] FIG. 10 is a diagram showing an example of a design data editing screen. [Figure 8] FIG. 4 is a sequence diagram showing processing between devices. [Figure 9] 10A and 10B are diagrams showing examples of design conversion on a design data editing screen. [Figure 10] 10 is a flowchart showing details of processing by an operator PC. [Figure 11] 10 is a flowchart showing details of processing by a service server. [Figure 12] 10A and 10B are diagrams showing examples of design conversion on a design data editing screen. [Figure 13] 10 is a flowchart showing details of processing by a service server. [Figure 14] 10A and 10B are diagrams showing examples of design conversion on a design data editing screen. [Figure 15] 10 is a flowchart showing details of processing by a service server. [Figure 16] 10 is a flowchart showing details of processing by a service server. [Figure 17] 10 is a flowchart showing details of processing by a service server. [Figure 18] 10 is a flowchart showing details of processing by a service server. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that the following embodiments do not limit the present disclosure, and not all combinations of features described in the embodiments are necessarily essential to the solutions of the present disclosure. Note that identical configurations will be described using the same reference numerals. Furthermore, each process (step) in a flowchart will be indicated by a reference numeral beginning with "S." In the following description, text elements that are rendered as images in design data will be referred to as text elements in an image or text elements. Furthermore, non-text elements that are rendered as images in design data will be referred to as non-text elements in an image or non-text elements.

[0009] [First embodiment] <<Overall system configuration>> FIG. 1 is a diagram illustrating the overall configuration of an information processing system according to the present disclosure. The information processing system includes an operator computer (PC) 101 and a service server 102. The operator PC 101 and the service server 102 are connected to each other via a network 100 so that they can communicate with each other. Examples of this network include the Internet, a wired local area network (LAN), or a wireless LAN. The operator PC 101 is a computer that executes an application for editing design data such as posters or flyers. The operator PC 101 transmits requests for partial editing, data processing, and rendering of the design data to the service server 102 via the application. The service server 102 is a server that executes editing, data processing, and rendering of the design data. The information processing system according to the present embodiment includes the operator PC 101 and the service server 102, but is not limited to this. For example, the operator PC 101 may also function as the service server 102. Alternatively, there may be two or more operator PCs and two or more service servers. The operator PC 101 and the service server 102 are information processing devices, and may be the above-mentioned PC or a tablet terminal.

[0010] <Hardware configuration of operator's PC> 2(a) is a hardware configuration diagram of the operator PC 101. In the control unit 200, a CPU 201, a ROM 202, a RAM 203, a HDD 204, a network I / F 205, and a display / operation I / F 206 are communicably connected to one another via a system bus 210. The control unit 200 including the CPU 201 controls the overall operation of the operator PC 101.

[0011] The CPU 201 reads various programs such as control programs, system programs, and application programs stored in the storage devices (ROM 202, RAM 203, and HDD 204). The CPU 201 executes the various programs that it reads, thereby functioning as a means for executing print control, reading control, image processing, and the like.

[0012] The storage device stores and holds various programs, image data, etc. The storage device includes a ROM 202 which is a non-volatile memory, a RAM 203 which is a volatile memory, and an HDD (Hard Disk Drive) 204 which is a large-capacity storage area.

[0013] The ROM 202 is a non-volatile memory that stores various programs, and the CPU 201 reads out the various programs and executes the processes. The RAM 203 is a volatile memory used as the main memory of the CPU 201 or as a temporary storage area such as a work area. The HDD 204 is a non-volatile memory used as a large-capacity storage area for storing images and the like to be processed by the CPU 201. The HDD 204 may be an SSD (Solid State Drive), flash memory, or cloud storage.

[0014] The network I / F 205 connects the control unit 200 of the operator PC 101 to the network 100, transmits image data and information to external devices on the network 100, and receives various information from external devices on the network 100. The display / operation I / F 206 connects an output device such as a display and an input device such as a keyboard and a mouse via the system bus 210. In this embodiment, a display, a keyboard, and a mouse are described as examples, but this is not limiting. The output device and the input device may be integrated using a touch panel.

[0015] <<Hardware configuration of the service server>> 2(b) is a diagram showing the hardware configuration of the service server 102. In the control unit 220, a CPU 221, a ROM 222, a RAM 223, a HDD 224, and a network I / F 225 are connected via a system bus 230 so as to be able to communicate with one another.

[0016] The control unit 220, including the CPU 221, controls the overall operation of the service server 102. The CPU 221 reads various programs, such as control programs, system programs, and application programs, stored in storage devices (ROM 222, RAM 223, HDD 224), and executes various processes. The ROM 222 is a non-volatile memory that holds various programs, and the CPU 221 reads the various programs and executes the processes. The RAM 223 is used as the main memory of the CPU 221 or as a temporary storage area such as a work area. The HDD 224 stores document data and various programs. The HDD 224 may be an SSD, flash memory, or cloud storage. The network I / F 225 connects the control unit 220 of the service server 102 to the network 100. The service server 102 receives processing requests via applications from other devices, such as the operator PC 101, via the network I / F 225, and transmits and receives various information.

[0017] <<Software configuration of this system>> FIG. 3 is a software configuration diagram of the information processing system. The software configurations of the operator PC 101 and the service server 102 will be described. The operator PC 101 includes an application 300. The application 300 is a program module that executes editing tasks for design data, such as posters or flyers. The program for the application 300 is stored in a storage device (ROM 202 or HDD 204). The CPU 201 loads the program for the application 300 into RAM 203, whereby the program is executed. The application 300 also includes a display control unit 301, an operation unit 302, a data transmission unit 303, and a data reception unit 304. The display control unit 301 displays an application screen on the display via the display / operation I / F 206. The operation unit 302 accepts user input from a mouse and keyboard via the display / operation I / F 206. The data transmission unit 303 transmits design data processing instructions to the service server 102 via the network I / F 205, based on instructions from the operation unit 302. The data receiving unit 304 receives the processing results of the design data from the service server 102 via the network I / F 205 .

[0018] The service server 102 includes a design data editing unit 310, a data transmission unit 311, a data reception unit 312, a text translation unit 313, and an image conversion unit 314. The service server 102 also includes a design database 321, a multimodal large-scale language model 322, an image generation model 323, and a conversion database 324. The design data editing unit 310 edits design data such as posters or flyers based on processing instructions from an application 300 of the operator's PC 101. For example, the design data editing unit 310 adds or deletes objects such as text, images, or shapes from the design data, and adjusts the coordinates, size, and color of each object. The design data is saved in the design database 321 for each application 300 of the operator's PC 101 (or for each account if multiple user accounts exist).

[0019] FIG. 4 shows an example of a data table for design data stored in the design database 321 and managed by the service server 102. The data table includes parameters such as an ID 401 for uniquely identifying an object in the design data, an object 402, a type 403, coordinates 404, an area size 405, color 406, a font 407, a font size 408, and metadata 409. The object 402 contains a value for each object, such as text, image, or graphic, in the design data. The type 403 contains information indicating the type of each object. The coordinates 404 contain a value indicating the position of each object in the design data. The area size 405 contains a value indicating the size of each object. In this embodiment, if the object type is text, the coordinates 404 and the area size 405 represent the coordinates and size of the text area provided for the text. The color 406 contains a value indicating the color of each object as an RGB value. The color 406 does not need to have a set value for image objects. The color 406 may also be set as an HSV value instead of an RGB value. Font 407 contains a value indicating the font type for each text object. Font size 408 contains a value indicating the font size for each text object. Metadata 409 holds metadata for each object. In this embodiment, if the object type is image or text, a description or attribute of the image or text can be set as metadata. For example, an image description can be written as in item 410, or text attributes can be written as in item 411. Furthermore, if the object type is background, a description of the document size of the background can be set as metadata. For example, a description such as "A2 size" can be written as in item 412. Furthermore, metadata may not be set for some objects. Note that the data table shown in FIG. 4 exists for each design data. A data table may exist for each object type, and may include parameter types other than those described above.

[0020] The data transmission unit 311 transmits the results of design data processing to the application 300 of the operator PC 101 via the network I / F 225. The data reception unit 312 receives design data processing instructions from the application 300 of the operator PC 101 via the network I / F 225. The text translation unit 313 machine-translates text objects in the design data into a specified language. The conversion database 324 stores setting information for design conversion. The setting information for design conversion includes translatable languages, locales representing countries or regions, and information associating each locale with translatable languages. The image conversion unit 314 converts image objects in the design data using a multimodal large-scale language model 322 and an image generation model 323. In this embodiment, the multimodal large-scale language model 322 and the image generation model 323 are located in the service server 102, but this is not limited thereto. If the service server 102 can use a multimodal large-scale language model and an image generation model managed by an external server or other device, the service server 102 may use these to convert the design data. Therefore, the service server 102 may include a conversion control unit (not shown) that controls the text translation unit 313 and the image conversion unit 314.

[0021] FIG. 5 is a diagram illustrating an example of processing by the multimodal large-scale language model 322. Before describing the multimodal large-scale language model, we will first explain the large-scale language model. The large-scale language model is a language model composed of multiple neural network layers and trained using a large amount of text. The parameters of each neural network layer are adjusted by training the model to predict the word that is likely to follow a given word. This allows the large-scale language model to generate text that is highly likely to match the input text. Specifically, the large-scale language model divides the input text into tokens, which are the basic processing units, and then converts each token into an embedding representation that numerically represents the meaning of a word or sentence. By converting into an embedding representation, the context of the input text can be expressed numerically. The large-scale language model then generates text that is highly likely to match the embedding representation based on the converted embedding representation.

[0022] A multimodal large-scale language model is a large-scale language model that can handle multiple types of information, including not only text but also images, audio, and video. In the case of a multimodal large-scale language model that inputs images, an image encoder is used to calculate image features from the input image. The calculated image features are then mapped to an embedded representation in the large-scale language model, enabling the large-scale language model to handle image input. This allows the multimodal large-scale language model to generate text that is highly likely to match the input text or image. The multimodal large-scale language model can also output the input text or image as an embedded representation. Furthermore, by changing the input text or image, the multimodal large-scale language model can perform various tasks that are not present in the training data. Specifically, the multimodal large-scale language model can translate text as shown in Figure 5(a) and describe images as shown in Figure 5(b). As shown in Figure 5(a), when the multimodal large-scale language model 322 receives a prompt 501 representing an instruction for a production, it outputs a text production 502 that is highly likely to match the prompt 501. In the example of FIG. 5(a), an input prompt 503 is input to the multimodal large-scale language model 322, and an output product 504 is output. Alternatively, as shown in FIG. 5(b), the multimodal large-scale language model 322 can input an image 505 and a prompt 506 to output a text product 507 that is highly likely to match the image 505 and the prompt 506. In the example of FIG. 5(b), an input image 508 and an input prompt 509 are input to the multimodal large-scale language model 322, and an output product 510 is output. Specific examples of the multimodal large-scale language model 322 include ChatGPT (https: / / chat.openai.com) and Gemini (https: / / gemini.google.com). Note that any multimodal large-scale language model can be used in this embodiment.

[0023] Figure 6 illustrates an example of the processing of the image generation model 323. The image generation model is composed of multiple neural network layers and is constructed by learning from a large amount of text and images. The parameters of each neural network layer are adjusted by training the model to perform the task of generating an image corresponding to input text. This allows the image generation model to generate images that are highly likely to match the input text. Specifically, the image generation model divides the input text into tokens, which are the basic processing units, and then converts each token into an embedded representation that numerically represents the meaning of words and sentences. By converting to an embedded representation, the context of the input text can be expressed numerically. The image generation model then generates an image that is highly likely to match the embedded representation based on the converted embedded representation. Furthermore, by inputting an embedded representation to the image generation model, it is also possible to generate an image that is highly likely to match the embedded representation.

[0024] As shown in FIG. 6(a), when a prompt 601 is input, the image generation model 323 outputs an output product 604, which is an image that is likely to match the prompt 601. In the example of FIG. 6(a), an input prompt 603 is input to the image generation model 323, and the output product 604 is output. Also, as shown in FIG. 6(b), when a prompt 605 and a negative prompt 606 are input to the image generation model 323, the image generation model 323 can output an output product 610, which is an image that is likely to match the prompt 605 and the negative prompt 606. Here, the negative prompt refers to an instruction regarding an element that is not to be included in the generated image. In the example of FIG. 6(b), an input prompt 608 and an input negative prompt 609 are input to the image generation model 323, and the output product 610 is output. As shown in FIG. 6(b), the negative prompt 606 may be input to the image generation model 323 separately from the prompt 605, or may be combined with the prompt 605 and input to the image generation model 323 as a single prompt. When combining the negative prompt 606 with the prompt 605, the instruction in the negative prompt 606 is negated, such as "Do not include helmet." Examples of the image generation model 323 include StableDiffusion (https: / / ja.stability.ai / stable-diffusion) and Midjourney (https: / / www.midjourney.com). Note that any image generation model can be used in this embodiment.

[0025] FIG. 7 illustrates an example of a design data editing screen. The design data editing screen is displayed on the display of the operator's PC 101 via the application 300 by the display control unit 301. The design data editing screen includes a design editing area 700, a translation button 701, a save button 702, an output button 703, an add image button 704, and an add text button 705. The design data being edited is displayed in the design editing area 700. The design data editing unit 310 edits each object displayed in the design editing area 700, such as adjusting its position or size, based on user input received through the operation unit 302. To add an image or text object to the design editing area 700, the user presses the add image button 704 or the add text button 705. When the add image button 704 is pressed, a file dialog box is invoked, and the file path is specified to perform import processing. Of course, additional buttons corresponding to other object types may be added, and the import source may also be specified as an external cloud service or SNS service. It is also possible to accept addition of content by dragging and dropping it into the design editing area 700. To convert the design, press the translation button 701. To save the design data being edited, press the save button 702 to save the design data in the design database 321. Pressing the output button 703 also allows the design data being edited to be output in PDF or JPEG format.

[0026] 3 illustrates an example of a configuration in which the service server 102 includes the design data editing unit 310, the text translation unit 313, and the image conversion unit 314. This embodiment is not limited to this configuration. The application 300 of the operator PC 101 may include the functions of the design data editing unit 310, the text translation unit 313, and the image conversion unit 314.

[0027] <<Processing Sequence>> The processing sequence of the information processing system in this embodiment will be described with reference to FIG. 8. The following describes the processing of each device when design conversion instruction information received by application 300 of operator PC 101 is sent to service server 102 and the design display is updated based on the design conversion results. Note that the design data being edited by operator PC 101 is stored in design database 321 of service server 102 and is data managed by service server 102. In this embodiment, the design data is stored in design database 321 of service server 102, but is not limited to this. The design data may be stored in, for example, an external storage device or cloud storage managed by service server 102. Furthermore, the instruction information is managed by operator PC 101.

[0028] In S801, the operation unit 302 of the application 300 accepts a design conversion instruction from the user via the display / operation I / F 206. The design conversion instruction includes setting values ​​such as an instruction as to whether to translate text, an instruction as to whether to convert images, a specification of the source language, a specification of the target language, and a specification of the locale representing the country or region. Furthermore, the design conversion instruction may include location information of the operator PC 101.

[0029] FIG. 9 shows an example of design conversion on the design data editing screen of this embodiment. FIG. 9(a) shows an example of the screen display when the translation button 701 in FIG. 7 is pressed. FIG. 9(a) includes a translation setting field 901. The translation setting field 901 includes a setting item 904 for specifying whether to translate the language of text objects in the design data, and a setting item 905 for specifying whether to convert image objects in the design data. The translation setting field 901 also includes a setting item 902 for the source language and a setting item 903 for the target language. When the execute button 906 is pressed, a design conversion instruction is accepted based on the setting values ​​set in each setting item. When the close button 907 is pressed, the translation setting field 901 is closed. FIG. 9(a) shows a state in which a text translation instruction has been made, an image conversion instruction has been made, the source language is "English," and the target language is "Japanese."

[0030] In S802, when the execute button 906 is pressed and a design conversion instruction is accepted, the data transmission unit 303 of the application 300 transmits design conversion instruction information to the service server 102. The design conversion instruction information includes setting values ​​such as an instruction as to whether to translate text, an instruction as to whether to convert images, a designation of the source language, a designation of the target language, and a designation of a locale representing a country or region. Furthermore, the design conversion instruction information may include location information of the operator PC 101.

[0031] In S803, the service server 102 converts the design data based on settings such as the language to be translated or the locale. Details of the design conversion process will be described with reference to FIG. 11. In S804, the data transmission unit 311 of the service server 102 transmits the design conversion result to the application 300 of the operator PC 101. In S805, the display control unit 301 of the application 300 updates the design data being edited on the design editing screen to display the design conversion result. Through the above processing sequence, it is possible to transmit the design conversion instruction information received by the application 300 of the operator PC 101 to the service server 102 and update the design display based on the design conversion result (converted design data) (see FIG. 9(b)).

[0032] <<Processing flow of operator PC 101>> 10 is a flowchart illustrating a conversion instruction receiving process and a design display update process executed by the application 300 of the operator PC 101 in this embodiment. This flowchart includes steps S802 and S805 in FIG. 8. S801 in FIG. 8 corresponds to steps S1001, S1002, S1003, S1004, S1005, S1006, S1007, and S1008 in this flowchart. The process of this flowchart is realized, for example, by the CPU 201 reading an application program stored in the ROM 202 into the RAM 203 and executing it. The process of this flowchart is also started when the translation button 701 is pressed on the design data editing screen.

[0033] In S1001, the display control unit 301 displays a translation setting field on the design data editing screen. In S1002, the operation unit 302 determines whether the close button in the translation setting field has been pressed. If it is determined that the close button in the translation setting field has been pressed, the process proceeds to S1003. If it is determined that the close button in the translation setting field has not been pressed, the process proceeds to S1004. In S1003, the display control unit 301 closes the translation setting field on the design data editing screen, and ends the processing of this flowchart.

[0034] In S1004, the operation unit 302 determines whether the execute button in the translation setting field has been pressed. If it is determined that the execute button in the translation setting field has been pressed, the process proceeds to S1005. If it is determined that the execute button in the translation setting field has not been pressed, the process returns to S1001. In S1005, the operation unit 302 acquires setting values ​​related to design conversion instructions, and the process proceeds to S1006. The setting values ​​include an instruction as to whether to translate text, an instruction as to whether to convert images, a designation of the source language, a designation of the language to be translated, and a designation of the locale representing the country or region. For example, FIG. 9(a) shows a state in which a text translation instruction has been made, an image conversion instruction has been made, the source language is "English," and the target language is "Japanese."

[0035] In S1006, the operation unit 302 verifies whether or not there is an error in the setting value, and the process proceeds to S1007. Specifically, if there is no instruction to translate text and no instruction to convert images, the operation unit 302 determines that there is an error in the setting value. Furthermore, if the source language or the language to be translated is "unselected," the operation unit 302 determines that there is an error in the setting value. In S1007, the operation unit 302 determines whether or not there is an error in the setting value. If there is an error in the setting value, the process proceeds to S1008. If there is no error in the setting value, the process proceeds to S802. In S1008, the display control unit 301 displays the content of the error in the translation setting field, and the process returns to S1001. For example, if the language to be translated is "unselected," a dialog box containing an error message such as "No language to be translated has been selected" is displayed.

[0036] In S802, the data transmission unit 303 transmits design conversion instruction information to the service server 102, and the process proceeds to S1009. The design conversion instruction information includes setting values ​​such as an instruction as to whether to translate text, an instruction as to whether to convert images, a designation of the source language, a designation of the target language, and a designation of a locale representing a country or region. Furthermore, the design conversion instruction information may include location information of the operator PC 101. In S1009, the data receiving unit 304 receives the design conversion result (converted design data) from the service server 102, and the process proceeds to S805. The processing of S1009 corresponds to receiving the design conversion result transmitted by the data transmission unit 311 in S804.

[0037] In S805, the display control unit 301 updates the design data being edited on the design editing screen to display the design conversion result, and then returns to S1001. Specifically, the display control unit 301 updates the design editing screen by replacing the design data being edited with the design conversion result. Note that the design data being edited may be retained and the design conversion result may be added as a new page, thereby displaying both the design data being edited and the design conversion result on the design editing screen. Alternatively, the design conversion result may be printed directly to a printer without being displayed on the display.

[0038] <<Processing flow of the service server 102>> FIG. 11 is a flowchart illustrating the design conversion process executed by the service server 102 in this embodiment. This flowchart includes step S804 in FIG. 8. S803 in FIG. 8 corresponds to steps S1102, S1103, S1104, S1105, S1106, S1107, S1108, S1109, S1110, S1111, S1112, S1113, and S1114 in this flowchart. The process of this flowchart is realized, for example, by the CPU 221 reading a program stored in the ROM 222 into the RAM 223 and executing it. The process of this flowchart is initiated when the data receiving unit 312 of the service server 102 receives design conversion instruction information from the application 300 of the operator PC 101.

[0039] In S1101, the data receiving unit 312 receives design conversion instruction information from the application 300 of the operator PC 101, and proceeds to S1102. S1101 corresponds to receiving the design conversion instruction information sent by the data sending unit 303 in S802. In S1102, the text translation unit 313 determines whether there is a text translation instruction in the design conversion information. If it is determined that there is a text translation instruction, the process proceeds to S1103. If it is determined that there is no text translation instruction, the process proceeds to S1107.

[0040] In S1103, the text translation unit 313 acquires a text object in the design data, and proceeds to S1104. In S1104, the text translation unit 313 determines whether a translation target language is specified in the design conversion instruction information. If it is determined that a translation target language is specified, the process proceeds to S1105. If it is determined that a translation target language is not specified, the process proceeds to S804.

[0041] In S1105, the text translation unit 313 translates the text of the acquired text object, and the process proceeds to S1106. The multimodal large-scale language model 322 is used for the text translation. The text translation unit 313 generates a prompt to cause the multimodal large-scale language model 322 to execute the translation. For example, when the text object 913 in FIG. 9(a) is acquired, the following prompt is generated.

[0042] Please translate the following sentence: "Join Our Automotive Manufacturing Team" By inputting the above prompt into the multimodal large-scale language model 322, the following translated text is obtained:

[0043] "Join our automotive manufacturing team." The text translation unit 313 replaces the text of the acquired text object with the text of the translation result output from the multimodal large-scale language model 322. At this time, if the number of characters in the translated text is greater than the pre-translation text, the text may overflow the area of ​​the object. Therefore, a process of reducing the font size of the text object according to the rate of increase in the number of characters may be added. Note that the text translation method is not limited to the above-described method. For example, the text translation unit 313 may store a machine translation model trained on a large number of pairs of pre-translation text and translated text, and translation may be performed using the machine translation model. Furthermore, pairs of pre-translation text and translated text are registered in the conversion database 324. Then, when the text of the text object is registered as pre-translation text, translation may be performed by replacing the text of the text object with the paired translated text.

[0044] In S1106, the text translation unit 313 determines whether all text objects in the design data have been processed. If it is determined that all text objects have been processed, the process proceeds to S1107. If it is determined that there are unprocessed text objects, the process returns to S1103. In S1107, the image conversion unit 314 determines whether there is an image conversion instruction in the design conversion instruction information. If it is determined that there is an image conversion instruction, the process proceeds to S1108. If it is determined that there is no image conversion instruction, the process proceeds to S804.

[0045] In S1108, the image conversion unit 314 acquires an image object in the design data, and the process proceeds to S1109. In S1109, the image conversion unit 314 interprets the image of the acquired image object, and the process proceeds to S1110. Specifically, the content of the image (the content shown in the image) is expressed as text. The multimodal large-scale language model 322 is used for image interpretation. The image conversion unit 314 generates a prompt to cause the multimodal large-scale language model 322 to interpret the image. For example, when the image object 911 in FIG. 9(a) is acquired, the following prompt is generated.

[0046] "What does this image show?" "Please respond with a prompt expression that will allow you to recreate this image in image generation." If your answer contains text that was taken from an image, please enclose it in quotation marks. By inputting the above prompt and the image of the image object 911 into the multimodal large-scale language model 322, the following interpretation result text is obtained.

[0047] "A mechanic is standing in front of the factory wearing work clothes. He is holding a sign that says 'Welcome'." The multimodal large-scale language model 322 is used to perform image interpretation of text and non-text elements in an image. Non-text elements include images, illustrations, and graphics. In the above example, "Welcome" in quotation marks indicates a text element in the image. The image interpretation method is not limited to the above. For example, the image conversion unit 314 may retain a generic object recognition model trained on the types of objects appearing in the image, and use the generic object recognition model to perform image interpretation of non-text elements in the image. Regarding text elements in an image, optical character recognition (OCR) may be applied to the image to extract the text and its position and size. When the image conversion unit 314 applies OCR to extract text, it may add a new text object in front of the image object, the text object having the extracted text as its value. When the image conversion unit 314 applies OCR to extract text and a translation target language is specified, the text translation unit 313 translates the text of the text object added by the image conversion unit 314 into the translation target language.

[0048] In S1110, the image conversion unit 314 determines whether a target language for translation is specified in the design conversion instruction information. If it is determined that a target language for translation is specified, the process proceeds to S1111. If it is determined that a target language for translation is not specified, the process proceeds to S804. In S1111, the image conversion unit 314 generates a prompt for the target language for translation using the target language for translation and the image interpretation result as image generation instruction data, and then proceeds to S1112. Specifically, the prompt is generated to generate a new image that is appropriate for the cultural sphere of the text language while maintaining the image content of the acquired image object. If a text element exists in the image interpretation result and a target language for translation is specified, the text translation unit 313 translates the text element into the target language for translation. The image conversion unit 314 then generates a prompt for the target language for translation. For example, if the image object 911 in FIG. 9(a) is acquired and the target language for translation is "Japanese," the following prompt is generated.

[0049] "A mechanic is wearing work clothes and standing in front of a factory. He is holding a sign that says 'Welcome.' Please make the image suitable for the Japanese cultural sphere." In S1112, the image conversion unit 314 generates a converted image using the prompt for the language to be translated generated in S1111, and the process proceeds to S1113. The image generation model 323 is used to generate the converted image. The image conversion unit 314 inputs the prompt for the language to be translated generated in S1111 to the image generation model 323. For example, when the image object 911 in FIG. 9(a) is acquired, the following prompt is input.

[0050] "A mechanic is wearing work clothes and standing in front of a factory. He is holding a sign that says 'Welcome.' Please make the image suitable for the Japanese cultural sphere." By inputting the above prompt into the image generation model 323, a transformed image of the image object 921 in FIG. 9(b) is generated.

[0051] In S1113, the image conversion unit 314 replaces the image of the acquired image object with the converted image generated in S1112, and the process proceeds to S1114. If the image size of the acquired image object differs from the image size of the converted image, the image size of the converted image is resized to the same size as the image of the acquired image object before the replacement process is performed. In S1114, the image conversion unit 314 determines whether all image objects in the design data have been processed. If it is determined that all image objects have been processed, the process proceeds to S804. If it is determined that an unprocessed image object exists, the process returns to S1108. In S804, the data transmission unit 311 transmits the design conversion result to the application 300 of the operator PC 101, and when transmission of the design conversion result is complete, the process of this flowchart ends.

[0052] In the example of Figure 9, the images of image objects 911 and 912 in Figure 9(a) are changed to converted images of image objects 921 and 922 in Figure 9(b), respectively. At this time, the text elements and non-text elements (people, buildings, etc.) in the images are changed to images that are suitable for the Japanese cultural sphere. In addition, text objects 913 and 914 in Figure 9(a) are translated into Japanese, and translated text objects 923 and 924 are newly placed as shown in Figure 9(b).

[0053] As described above, according to this embodiment, images in design data are interpreted and prompts for the target language are generated based on the interpretation results. The images in the design data are then replaced with converted images generated based on the prompts for the target language. This allows the design data to be converted appropriately.

[0054] [Modification of the first embodiment] In the first embodiment, a prompt for the target language is generated to change the image in the design data to an image suited to the cultural sphere of the text language. Meanwhile, the step of generating a prompt for the target language in S1111 in the first embodiment may be replaced with another step of generating image generation instruction data. For example, it may be replaced with a step of generating an impression prompt or a step of generating a generation prompt. Furthermore, the design conversion instruction setting values ​​may include a setting for specifying an impression or a setting for specifying a target age. This allows the operator to easily change the image in the design data to an image suited to the target attribute, such as the impression or generation, that the operator expects. For example, to change to an image suited to the impression of "cool," the impression prompt is generated as follows:

[0055] "A mechanic is wearing work clothes and standing in front of a factory. He is holding a sign that says 'Welcome.' Please create an image that gives a cool impression." Also, if you want to change the image to one that is appropriate for the "teenager" generation, you can generate a generation prompt as follows.

[0056] "A mechanic is standing in front of a factory wearing overalls. He is holding a sign that says 'Welcome'. Please make the image suitable for teenagers." Furthermore, the step of generating a prompt for the target language may be replaced with a step of generating an embedded expression for the target language. Specifically, in S1109, the image conversion unit 314 receives the image interpretation result from the multimodal large-scale language model 322 as an embedded expression rather than text. Then, the image conversion unit 314 inputs the text "Japanese cultural sphere" into the multimodal large-scale language model 322 to receive the embedded expression. The embedded expression for the target language is generated by performing an addition operation on the embedded expression of the image interpretation result and the embedded expression meaning "Japanese cultural sphere." Then, in S1112, the image conversion unit 314 inputs the embedded expression for the target language into the image generation model 323 to generate a converted image. This allows the image in the design data to be changed to an image suited to the cultural sphere of the text language by using the embedded expression without using a prompt.

[0057] [Second embodiment] In the first embodiment, non-text elements such as images in design data are interpreted, and a prompt for the target language is generated based on the interpretation results. Then, non-text elements such as images in the design data are replaced with non-text elements converted based on the prompt for the target language. This allows an operator to easily change non-text elements such as images in the design data to non-text elements that are appropriate for the cultural sphere of the text language. However, since all non-text elements in the design data are converted, even non-text elements such as images that the operator does not want to change are replaced, reducing convenience. Therefore, in this embodiment, an example will be described in which only selected non-text elements in the design data are converted to non-text elements that are appropriate for the cultural sphere of the text language. Note that only differences from the first embodiment will be described below. It is also assumed that multiple non-text elements are included.

[0058] <<Processing Sequence>> In S801 of FIG. 8, the operation unit 302 of the application 300 receives a design conversion instruction from the user via the display / operation I / F 206. In this embodiment, the design conversion instruction information includes a setting value indicating whether or not the element is a conversion target element for image conversion. FIG. 12 shows an example of design conversion on the design data editing screen of this embodiment. FIG. 12(a) shows a screen displaying a translation setting field 1201. The translation setting field 1201 includes a setting item 1202 for specifying whether or not to convert only the conversion target elements when converting image objects in the design data. When the setting to convert only the conversion target elements is selected, the operation unit 302 of the application 300 receives the selection of one or more conversion target elements from the image objects in the design editing area 1200. FIG. 12(a) shows a state in which there is a text translation instruction, an image conversion instruction, an instruction to convert only the conversion target elements, the translation source language being "English," and the translation target language being "Japanese." FIG. 12(a) shows a state in which an image object 1211 is designated as an element to be converted, and an image object 1212 is not designated as an element to be converted.

[0059] <<Processing flow of operator PC 101>> In S1005 of FIG. 10, the operation unit 302 acquires the setting values ​​in the design conversion instruction. The setting values ​​include an instruction as to whether or not the element is a target for image conversion. In S1006, the operation unit 302 verifies whether or not there is an error in the setting values. Specifically, if there is no instruction to translate text and no instruction to convert images, the operation unit 302 determines that there is an error in the setting values. If the source language or the target language is "unselected," the operation unit 302 determines that there is an error in the setting values. Furthermore, if there is an instruction to convert only the target elements for conversion and the target elements for conversion are not specified, the operation unit 302 determines that there is an error in the setting values.

[0060] 10, the display control unit 301 displays the content of the setting value error in the translation setting field. For example, if there is an instruction to convert only the conversion target element and the conversion target element is not specified, a dialog box containing an error message such as "No conversion target image has been selected" is displayed.

[0061] <<Processing flow of the service server 102>> FIG. 13 is a flowchart illustrating processing executed by the service server 102 in this embodiment. S803 in FIG. 8 corresponds to S1102, S1103, S1104, S1105, S1106, S1107, S1108, S1109, S1110, S1111, S1112, S1113, S1114, and S1301 in this flowchart. The processing of this flowchart is realized, for example, by the CPU 221 reading a program stored in the ROM 222 into the RAM 223 and executing it. The processing of this flowchart is started based on the reception by the data receiving unit 312 of the service server 102 of design conversion instruction information from the application 300 of the operator PC 101. The following describes the differences between the processing of the flowchart in FIG. 13 and the processing of the flowchart in the first embodiment shown in FIG. 11.

[0062] 13, the process of S1301 is added to the flow of the first embodiment. In S1301, the image conversion unit 314 determines whether the acquired image object is designated as a conversion target element in the design conversion instruction information. If it is determined that the acquired image object is designated as a conversion target element, the process proceeds to S1109. If it is determined that the acquired image object is not designated as a conversion target element, the process proceeds to S1114.

[0063] In the example of FIG. 12, conversion is performed on image object 1211 in FIG. 12(a). Image object 1211 is converted into image object 1221 shown in FIG. 12(b). On the other hand, conversion is not performed on image object 1212 in FIG. 12(a), which is not an element to be converted. In the above example, a case of a non-text element including an image was described, but this is not limited to this. When multiple text elements are included, it is also possible to select only the translation target elements to be translated for the text elements. Translation is performed on the selected translation target elements, and translation is not performed on the unselected translation target elements.

[0064] As described above, according to this embodiment, only non-text elements such as selected images in design data are converted into non-text elements that are suited to the cultural sphere of the text language. This allows the design data to be converted appropriately, improving convenience for the operator.

[0065] [Third embodiment] In the first embodiment, non-text elements such as images in design data are interpreted, and a prompt for the target language is generated based on the interpretation results. Then, non-text elements such as images in the design data are replaced with non-text elements converted based on the prompt for the target language. This allows an operator to easily change non-text elements such as images in the design data to non-text elements that are appropriate for the cultural sphere of the text language. However, when non-text elements converted based on the prompt for the target language are replaced, non-text elements that do not match the cultural sphere expected by the operator may be generated. For example, this can be the case when targeting an English-speaking Asian region. Therefore, in this embodiment, non-text elements such as images in the design data are interpreted, and a prompt for the locale is generated based on the interpretation results. An example of replacing non-text elements such as images in the design data with non-text elements converted based on the prompt for the locale is described below. Note that only differences from the first embodiment are described below.

[0066] <<Processing Sequence>> In S801 of FIG. 8, the operation unit 302 of the application 300 receives a design conversion instruction from the user via the display / operation I / F 206. In this embodiment, the design conversion instruction information includes a setting value for specifying a locale representing a country or region. FIG. 14 shows an example of design conversion on the design data editing screen of this embodiment. FIG. 14(a) shows a screen displaying a translation setting field 1401. The translation setting field 1401 includes a locale setting item 1403 representing a translation destination country or region. For the locale setting item 1403, the design data editing unit 310 may store a previously set setting value in the conversion database 324 and reflect that setting value in the screen display of the setting item 1403. Also, in this embodiment, as shown in FIG. 14(a), the setting item 1402 for the translation target language may be in an unselected state. FIG. 13(a) shows a state in which a text translation instruction is made, an image conversion instruction is made, the translation source language is "English," the translation target language is "unselected," and the locale is "Brazil."

[0067] <<Processing flow of operator PC 101>> In S1005 of FIG. 10, the operation unit 302 acquires the setting values ​​in the design conversion instruction. The setting values ​​include a locale designation representing a country or region. In S1006, the operation unit 302 verifies whether or not there is an error in the setting values. Specifically, if there is no text translation instruction and no image conversion instruction, the operation unit 302 determines that there is an error in the setting values. If the source language is "unselected," the operation unit 302 determines that there is an error in the setting values. Furthermore, if the translation target language is "unselected" and the locale is "unselected," the operation unit 302 determines that there is an error in the setting values.

[0068] 10, the display control unit 301 displays the content of the setting value error in the translation setting field. For example, if the language to be translated is "not selected" and the locale is "not selected," a dialog box including an error message such as "Please select the translation language or locale" is displayed.

[0069] <<Processing flow of the service server 102>> FIG. 15 is a flowchart illustrating processing executed by the service server 102 in this embodiment. S803 in FIG. 8 includes S1102, S1103, S1104, S1105, S1106, S1107, S1108, S1109, S1110, S1111, S1112, S1113, and S1114 in this flowchart. Furthermore, S803 in FIG. 8 includes S1501, S1502, S1503, S1504, and S1505 in this flowchart. The processing of this flowchart is realized, for example, by the CPU 221 reading a program stored in the ROM 222 into the RAM 223 and executing it. Furthermore, the processing of this flowchart is initiated when the data receiving unit 312 of the service server 102 receives design conversion instruction information from the application 300 of the operator PC 101. The difference between the processing in the flowchart of FIG. 15 and the processing in the flowchart of the first embodiment shown in FIG. 11 will be described.

[0070] 15, in addition to the flow in the first embodiment shown in FIG. 11, the processes of S1501, S1502, S1503, S1504, and S1505 are added. In S1501, the text translation unit 313 determines whether a locale is specified in the design conversion instruction information. If it is determined that a locale is specified, the process proceeds to S1502. If it is determined that a locale is not specified, the process proceeds to S804.

[0071] In S1502, the text translation unit 313 identifies the target language from the locale setting value. Specifically, a combination of a locale and a corresponding target language is registered in the conversion database 324. For example, "Locale: Brazil, Target Language: Portuguese" is registered. The text translation unit 313 then searches the conversion database 324 for the locale setting value to identify the target language corresponding to the locale setting value. Note that multiple target languages ​​may exist for one locale. If multiple target languages ​​exist, the same number of design conversion results are generated. The method for identifying the target language is not limited to the above. For example, the multimodal large-scale language model 322 may be used to input the locale setting value and output the language used in that locale to identify the target language. In S1503, the text translation unit 313 determines whether all target languages ​​have been processed. If it is determined that all target languages ​​have been processed, the process proceeds to S1106. If it is determined that there is an unprocessed target language, the process returns to S1105.

[0072] In S1504, the image conversion unit 314 determines whether a locale is specified in the design conversion instruction information. If it is determined that a locale is specified, the process proceeds to S1505. If it is determined that a locale is not specified, the process proceeds to S1110. In S1505, the image conversion unit 314 generates a locale prompt using the locale setting value and the image interpretation result. Specifically, the image conversion unit 314 generates a prompt for generating a new image that preserves the image content of the acquired image object and is appropriate for the cultural sphere of the locale. If a text element is present in the image interpretation result, the text translation unit 313 translates the text element into the translation target language. The image conversion unit 314 then generates a locale prompt. For example, if the image object 1411 in FIG. 14(a) is acquired and the locale setting value is "Brazil," the following prompt is generated.

[0073] "A mechanic standing in front of a factory wearing work clothes. Please make the image suitable for the Brazilian culture." In S1112, the image conversion unit 314 generates a conversion image using the prompt for the language to be translated generated in S1111 or the prompt for the locale generated in S1505. In the example of Fig. 14, the image of image object 1411 in Fig. 14(a) is converted into a conversion image of image object 1421 in Fig. 14(b). At this time, the image is converted into an image suitable for the Brazilian cultural sphere. In addition, text objects 1413 and 1414 in Fig. 14(a) are translated into Portuguese as shown by text objects 1423 and 1424 in Fig. 14(b), respectively.

[0074] As described above, according to this embodiment, non-text elements such as images in design data are interpreted and a locale prompt is generated based on the interpretation result. The non-text elements such as images in the design data are then replaced with non-text elements converted based on the locale prompt. This allows the operator to easily change non-text elements such as images in the design data to non-text elements that are appropriate for the cultural sphere of the locale that the operator expects, thereby improving operator convenience.

[0075] [Fourth embodiment] In the third embodiment, non-text elements such as images in design data are interpreted and a locale prompt is generated based on the interpretation results. Then, non-text elements such as images in the design data are replaced with non-text elements converted based on the locale prompt. This allows an operator to easily change non-text elements such as images in the design data to non-text elements that are appropriate for the cultural sphere of the locale the operator expects. However, if elements that are not intended to be included in the cultural sphere of the locale are not explicitly stated in the locale prompt, these elements may be included in the converted non-text elements. For example, there may be cases where elements that are inappropriate for the cultural sphere of the locale, such as tobacco, alcohol, or pork, are not intended to be included. Therefore, in this embodiment, a negative prompt is generated based on the locale. An example of replacing non-text elements such as images in design data with non-text elements converted based on the locale prompt and negative prompt is described below. Note that only differences from the third embodiment are described below.

[0076] <<Processing flow of the service server 102>> FIG. 16 is a flowchart illustrating processing executed by the service server 102 in this embodiment. S803 in FIG. 8 includes S1102, S1103, S1104, S1105, S1106, S1107, S1108, S1109, S1110, S1111, S1112, S1113, and S1014 in this flowchart. Furthermore, S803 in FIG. 8 includes S1501, S1502, S1503, S1504, S1505, and S1601 in this flowchart. The processing of this flowchart is realized, for example, by the CPU 221 loading a program stored in the ROM 222 into the RAM 223 and executing it. The processing of this flowchart is initiated when the data receiving unit 312 of the service server 102 receives design conversion instruction information from the application 300 of the operator PC 101. The difference between the processing in the flowchart of FIG. 16 and the processing in the flowchart of the third embodiment shown in FIG. 15 will be described.

[0077] The flowchart shown in FIG. 16 includes the additional processing of S1601 in addition to the flow of the flowchart of the third embodiment. In S1601, the image conversion unit 314 generates a negative prompt using a locale setting value. Specifically, a combination of a locale and a word for a negative prompt corresponding to the locale is registered in the conversion database 324. For example, a combination such as "locale: Brazil, word for negative prompt: tobacco" is registered. The image conversion unit 314 searches the conversion database 324 using the locale setting value to identify the word for a negative prompt corresponding to the locale setting value. The image conversion unit 314 then generates a negative prompt. For example, if the locale setting value is "Brazil," the following negative prompt is generated.

[0078] "tobacco" Note that there may be multiple words for negative prompts corresponding to one locale. Furthermore, the method for generating negative prompts is not limited to the above. For example, a negative prompt may be generated by inputting a locale setting value using the multimodal large-scale language model 322 and outputting a list of words that should not be included in the design for that locale.

[0079] In S1112, the image conversion unit 314 generates a converted non-text element using the prompt for the language to be translated generated in S1111, or the prompt for the locale generated in S1505 and the negative prompt generated in S1601.

[0080] As described above, according to this embodiment, a negative prompt is generated based on the locale. Non-text elements, such as images, in the design data are then replaced with non-text elements converted based on the locale prompt and the negative prompt. This allows the operator to easily change non-text elements, such as images, in the design data to converted non-text elements that match the cultural sphere of the locale and do not include elements that are not desired in the cultural sphere of the locale, thereby improving convenience for the operator.

[0081] [Fifth embodiment] In the third embodiment, non-text elements such as images in design data are interpreted, and a locale prompt is generated based on the interpretation results. Then, non-text elements such as images in the design data are replaced with non-text elements converted based on the locale prompt. This allows an operator to easily change non-text elements such as images in the design data to non-text elements that are appropriate for the cultural sphere of the operator's intended locale. However, this requires the operator to accept the translation target language and locale settings on the design editing screen, which reduces operator convenience. Therefore, in this embodiment, an example is described in which the translation target language and locale are determined by obtaining the language and locale setting information set on the operator's PC. Note that the following describes only the differences from the third embodiment.

[0082] <<Processing flow of the service server 102>> FIG. 17 is a flowchart illustrating processing executed by the service server 102 in this embodiment. S803 in FIG. 8 includes S1102, S1103, S1104, S1105, S1106, S1107, S1108, S1109, S1110, S1111, S1112, S1113, and S1114 in this flowchart. Furthermore, S803 in FIG. 8 includes S1501, S1502, S1503, S1504, S1505, S1701, and S1702 in this flowchart. The processing of this flowchart is realized, for example, by the CPU 221 loading a program stored in the ROM 222 into the RAM 223 and executing it. The processing of this flowchart is initiated when the data receiving unit 312 of the service server 102 receives design conversion instruction information from the application 300 of the operator PC 101. The difference between the processing in the flowchart of FIG. 17 and the processing in the flowchart of the third embodiment shown in FIG. 15 will be described.

[0083] <<Processing flow of the service server 102>> The flowchart shown in FIG. 17 includes the additional steps of S1701 and S1702 in addition to the flow of the flowchart of the third embodiment. In S1701, the text translation unit 313 acquires language setting information of the operator PC 101. At this time, the processing sequence between the service server 102 and the operator PC 101 is as follows. First, the data transmission unit 311 of the service server 102 transmits an instruction to transmit language and locale setting information to the application 300 of the operator PC 101. The data reception unit 304 of the application 300 of the operator PC 101 receives the instruction to transmit language and locale setting information from the service server 102. The data transmission unit 303 of the application 300 of the operator PC 101 transmits the language and locale setting information set in the operator PC 101 to the service server 102. The data reception unit 312 of the service server 102 receives the language and locale setting information from the application 300 of the operator PC 101. If the design conversion instruction information includes language and locale setting information, the following setting may be performed without executing the above processing sequence. That is, the language and locale setting information set on the operator PC 101 may be set as the language and locale setting information of the design conversion instruction information.

[0084] The language and locale setting information is expressed in the format of "language_locale." The text translation unit 313 of the service server 102 acquires the language setting information from the language and locale setting information received by the data receiving unit 312. For example, if the language and locale setting information of the operator PC 101 is "en_IN," the language setting information is acquired as "English."

[0085] In S1702, the image conversion unit 314 acquires the locale setting information of the operator PC 101. At this time, the service server 102 receives the language and locale setting information from the application 300 of the operator PC 101 in a processing sequence similar to S1701. Alternatively, if the language and locale setting information set in the operator PC 101 is set in the design conversion instruction information, the set language and locale setting information may be used. The image conversion unit 314 of the service server 102 acquires the locale setting information from the received language and locale setting information. For example, if the language and locale setting information of the operator PC 101 is "en_IN", the locale setting information acquired is "India".

[0086] As described above, according to this embodiment, the target language and locale are determined by acquiring the language and locale setting information of the operator's PC. This eliminates the need to accept the target language and locale setting values ​​entered by the operator on the design editing screen, improving convenience for the operator.

[0087] [Sixth embodiment] In the third embodiment, non-text elements such as images in design data are interpreted, and a locale prompt is generated based on the interpretation results. Then, non-text elements such as images in the design data are replaced with non-text elements converted based on the locale prompt. This allows an operator to easily change non-text elements such as images in the design data to non-text elements that are appropriate for the cultural sphere of the operator's intended locale. However, this requires the operator to accept the translation target language and locale settings on the design editing screen, which reduces operator convenience. Therefore, in this embodiment, an example is described in which the translation target language and locale are determined based on the location information of the operator's PC. Note that the following describes only the differences from the third embodiment.

[0088] <<Processing flow of operator PC 101>> In S1006 of Fig. 10, the operation unit 302 verifies whether or not there is an error in the setting value. Specifically, if there is no instruction to translate text and no instruction to convert images, the operation unit 302 determines that there is an error in the setting value. If the source language is "unselected," the operation unit 302 determines that there is an error in the setting value. If the language to be translated is "unselected," or if the locale is "unselected," the operation unit 302 determines that there is no error in the setting value.

[0089] <<Processing flow of the service server 102>> 18 is a flowchart illustrating processing executed by the service server 102 in this embodiment. S803 in FIG. 8 includes S1102, S1103, S1104, S1105, S1106, S1107, S1108, S1109, S1110, S1111, S1112, S1113, and S1114 in this flowchart. Furthermore, S803 in FIG. 8 includes S1501, S1502, S1503, S1504, S1505, S1801, S1802, S1803, and S1804 in this flowchart. The processing in this flowchart is realized, for example, by the CPU 221 reading a program stored in the ROM 222 into the RAM 223 and executing it. The processing of this flowchart starts when the data receiving unit 312 of the service server 102 receives design conversion instruction information from the application 300 of the operator PC 101. The difference between the processing of the flowchart of Fig. 18 and the processing of the flowchart of the third embodiment shown in Fig. 15 will be described.

[0090] 18, in addition to the flow of the flowchart of the third embodiment, the processes of S1801, S1802, S1803, and S1804 are added. In S1801, the text translation unit 313 acquires the location information of the operator PC 101. At this time, the processing sequence between the service server 102 and the operator PC 101 is as follows. First, the data transmission unit 311 of the service server 102 transmits a location information transmission instruction to the application 300 of the operator PC 101. The data reception unit 304 of the application 300 of the operator PC 101 receives the location information transmission instruction from the service server 102. The data transmission unit 303 of the application 300 of the operator PC 101 transmits the location information of the operator PC 101 to the service server 102. The data reception unit 312 of the service server 102 receives the location information from the application 300 of the operator PC 101. The location information is expressed in the format of "latitude, longitude." The text translation unit 313 of the service server 102 acquires the location information received by the data receiving unit 312. In the above processing sequence, a case has been described in which the design conversion instruction information does not include location information of the operator PC 101. If the design conversion instruction information includes location information of the operator PC 101, the location information of the operator PC 101 may be acquired from the design conversion instruction information without executing the above processing sequence.

[0091] In S1802, the text translation unit 313 identifies a locale based on the acquired location information. Specifically, combinations of locale and location information ranges are registered in the conversion database 324. The image conversion unit 314 searches the conversion database 324 using the location information to identify the locale corresponding to the location information. For example, if the location information is "135 degrees east longitude, 35 degrees north latitude," the locale is identified as "Japan."

[0092] In S1803, the image conversion unit 314 acquires the location information of the operator's PC 101. At this time, the service server 102 receives the location information from the application 300 of the operator's PC 101 in the same processing sequence as in S1801. The image conversion unit 314 of the service server 102 acquires the received location information. In S1804, the image conversion unit 314 identifies the locale based on the acquired location information. The method of identification is the same as the processing in S1802.

[0093] As described above, according to this embodiment, the target language and locale are determined based on the location information of the operator's PC. This improves convenience for the operator because it is not necessary to accept the target language and locale settings made by the operator on the design editing screen.

[0094] [Other embodiments] The present disclosure can also be realized by providing a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0095] The disclosure of the above-described embodiment includes the following configurations.

[0096] (Configuration 1) A program executed by a second information processing device capable of communicating with a first information processing device, characterized in that the program causes the second information processing device to function as a display control means that displays converted design data in which a first element, which is a text element included in design data, is translated and a second element, which is a non-text element included in the design data, is converted based on control executed by the first information processing device.

[0097] (Configuration 2) The program described in Configuration 1 further comprises a transmitting means for transmitting instruction information used to convert design data managed by the first information processing device to the first information processing device, and the control executed by the first information processing device is executed based on the instruction information.

[0098] (Configuration 3) The program described in Configuration 2 further comprises a receiving means for receiving the converted design data in which the first element is translated and the second element is converted based on control executed by the first information processing device based on the instruction information, and the display control means displays the converted design data received by the receiving means.

[0099] (Configuration 4) The program according to any one of configurations 1 to 3, wherein the first element is an element in which a text portion included in the design data is converted into text.

[0100] (Configuration 5) The program according to any one of configurations 1 to 4, wherein the second element is an image, illustration, or graphic element.

[0101] (Configuration 6) The program according to Configuration 3, wherein the instruction information includes a language used to translate the first element and convert the second element, or a locale representing a country or region.

[0102] (Configuration 7) The program described in Configuration 6, characterized in that the receiving means receives the converted design data in which the first element is translated into the language transmitted by the transmitting means based on control executed by the first information processing device, and the second element is converted to become an element corresponding to the language.

[0103] (Configuration 8) The program described in Configuration 6 is characterized in that the receiving means receives the converted design data in which the first element is translated by inputting the first element and a prompt for translating the first element into the language into a large-scale language model based on control executed by the first information processing device, and the second element is converted by inputting the second element and a prompt for converting the second element based on the language into a generative model based on control executed by the first information processing device.

[0104] (Configuration 9) The program described in Configuration 6, characterized in that the receiving means receives the converted design data in which the first element is translated in the language determined based on the locale sent by the sending means based on control executed by the first information processing device, and the second element is converted to become an element corresponding to the locale.

[0105] (Configuration 10) The program described in Configuration 6 is characterized in that the receiving means receives the converted design data in which the first element is translated by inputting the first element and a prompt for translating the first element into the language into a large-scale language model based on control executed by the first information processing device, and the second element is converted by inputting the second element and a prompt for converting the second element based on the locale into a generative model based on control executed by the first information processing device.

[0106] (Configuration 11) The program described in Configuration 6, characterized in that in the instruction information, the language and the locale are the language and locale set in the setting values ​​of the second information processing device, and the receiving means receives the converted design data in which the first element is translated in the language sent by the sending means based on control executed by the first information processing device and the second element is converted to become an element corresponding to the locale.

[0107] (Configuration 12) The program described in Configuration 6, characterized in that the instruction information further includes location information of the second information processing device, and the receiving means, based on control executed by the first information processing device, receives the converted design data in which the first element is translated in the language determined based on the locale identified from the location information of the second information processing device sent by the sending means, and the second element is converted to become an element corresponding to the locale.

[0108] (Configuration 13) The program described in Configuration 6, characterized in that the receiving means receives the converted design data in which elements that are inappropriate to display in the locale have been deleted from the second element by inputting the second element and a negative prompt corresponding to the element that is inappropriate to display in the locale into a generative model based on control executed by the first information processing device.

[0109] (Structure 14) The program described in Structure 6, characterized in that the first elements are included in multiple numbers and include a first target element that is the target of translation and a first non-target element that is not the target of translation, the second elements are included in multiple numbers and include a second target element that is the target of conversion and a second non-target element that is not the target of conversion, and the instruction information includes the first target element and the second target element.

[0110] (Configuration 15) A second information processing device capable of communicating with a first information processing device, characterized in that the second information processing device has a display control means for displaying converted design data in which a first element, which is a text element included in design data, is translated and a second element, which is a non-text element included in the design data, is converted based on control executed by the first information processing device.

[0111] (Structure 16) A control method for a second information processing device capable of communicating with a first information processing device, characterized in that the control method includes a step of displaying converted design data in which a first element, which is a text element included in design data, is translated based on control executed by the first information processing device, and a second element, which is a non-text element included in the design data, is converted.

[0112] (Configuration 17) An information processing system for communicating between a first information processing device and a second information processing device, wherein the first information processing device has a conversion control means for causing a conversion means to translate a first element, which is a text element, contained in the design data based on instruction information used to convert design data managed by the first information processing device, and for causing the conversion means to convert a second element, which is a non-text element, contained in the design data, and a transmission means for transmitting the converted design data converted based on the control of the conversion control means to the second information processing device, and the second information processing device has a receiving means for receiving the converted design data from the first information processing device and a display control means for displaying the converted design data.

Claims

1. A program executed by a second information processing device capable of communicating with a first information processing device, a display control means for displaying converted design data in which a first element, which is a text element included in the design data, is translated and a second element, which is a non-text element included in the design data, is converted based on control executed by the first information processing device; a program causing the second information processing device to function as a

2. a transmitting unit configured to transmit to the first information processing device instruction information used for converting the design data managed by the first information processing device, 2. The program according to claim 1, wherein the control executed by the first information processing device is executed based on the instruction information.

3. and receiving means for receiving the converted design data in which the first element is translated and the second element is converted based on control executed by the first information processing device based on the instruction information, 3. The program according to claim 2, wherein the display control means displays the converted design data received by the receiving means.

4. 2. The program according to claim 1, wherein the first element is an element obtained by converting a text portion included in the design data into text.

5. 2. The program according to claim 1, wherein the second element is an image, an illustration, or a graphic element.

6. 4. The program according to claim 3, wherein the instruction information includes a language used to translate the first element and convert the second element, or a locale representing a country or region.

7. The program described in claim 6, characterized in that the receiving means receives the converted design data in which the first element is translated into the language transmitted by the transmitting means based on control executed by the first information processing device, and the second element is converted to become an element corresponding to the language.

8. the receiving means translates the first element by inputting the first element and a prompt for translating the first element into the language into a large-scale language model based on control executed by the first information processing device; The program according to claim 6, characterized in that the converted design data in which the second element is converted is received by inputting the second element and a prompt that converts the second element based on the language into a generative model based on control executed by the first information processing device.

9. The program described in claim 6, characterized in that the receiving means receives the converted design data in which the first element is translated in the language determined based on the locale sent by the sending means based on control executed by the first information processing device, and the second element is converted to become an element corresponding to the locale.

10. the receiving means translates the first element by inputting the first element and a prompt for translating the first element into the language into a large-scale language model based on control executed by the first information processing device; The program according to claim 6, characterized in that the converted design data in which the second element is converted is received by inputting the second element and a prompt that converts the second element based on the locale into a generative model based on control executed by the first information processing device.

11. In the instruction information, the language and the locale are the language and the locale set as setting values ​​of the second information processing device, The program described in claim 6, characterized in that the receiving means receives the converted design data in which the first element is translated in the language transmitted by the transmitting means based on control executed by the first information processing device, and the second element is converted to become an element corresponding to the locale.

12. the instruction information further includes location information of the second information processing device; The program described in claim 6, characterized in that the receiving means, based on control executed by the first information processing device, receives the converted design data in which the first element is translated into the language determined based on the locale identified from the location information of the second information processing device transmitted by the transmitting means, and the second element is converted to become an element corresponding to the locale.

13. The program according to claim 6, characterized in that the receiving means receives the converted design data in which elements that are inappropriate to display in the locale have been deleted from the second elements by inputting the second elements and negative prompts corresponding to elements that are inappropriate to display in the locale into a generative model based on control executed by the first information processing device.

14. The first element is included in plural, and includes a first target element that is a target to be translated and a first non-target element that is not a target to be translated, The second element is included in plural, and includes a second target element that is a target of conversion and a second non-target element that is not a target of conversion, the instruction information includes the first target element and the second target element; 7. The program according to claim 6.

15. a second information processing device capable of communicating with the first information processing device, a display control means for displaying converted design data in which a first element, which is a text element included in the design data, is translated and a second element, which is a non-text element included in the design data, is converted based on control executed by the first information processing device; A second information processing device comprising:

16. A control method for a second information processing device capable of communicating with a first information processing device, comprising: a step of displaying converted design data in which first elements, which are text elements included in the design data, are translated and second elements, which are non-text elements included in the design data, are converted based on control executed by the first information processing device; A control method for a second information processing device, comprising:

17. An information processing system for performing communication between a first information processing device and a second information processing device, The first information processing device a conversion control means for causing a conversion means to translate a first element, which is a text element included in the design data, and for causing the conversion means to convert a second element, which is a non-text element included in the design data, based on instruction information used for converting the design data managed by the first information processing device; a transmitting means for transmitting the converted design data converted under the control of the conversion control means to the second information processing device; and The second information processing device receiving means for receiving the converted design data from the first information processing device; a display control means for displaying the converted design data; An information processing system comprising:

Images