Data processing apparatus, data processing method, and data processing program

Abstract

To provide a data processing apparatus, a data processing method, and a data processing program capable of reducing the burden on users required for metaverse operation.SOLUTION: A data processing apparatus comprises: an acquisition unit for acquiring input data indicating a user input; a processing unit for performing specification processing using a generative model that generates a three-dimensional image and a script according to the input data; and a control unit for controlling the display content of a display unit based on the result of the specification processing.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology disclosed herein relates to a data processing device, a data processing method, and a data processing program. [Background technology]

[0002] Patent Document 1 discloses a technique that can increase a distributor user's willingness to distribute, a viewer user's willingness to view, and / or a user's willingness to interact with others. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2023-098894 A Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, in operating a metaverse as described in Patent Document 1, it is necessary to generate worlds and avatars, and customize the worlds and avatars to suit events planned in the metaverse. Here, the generation of three-dimensional models of worlds and avatars may require advanced technology, and it has been difficult for general users to generate them in terms of technology and labor. In addition, it is possible to outsource the generation of three-dimensional models, but this may be difficult in terms of cost. Thus, in the conventional technology, there is room for improvement in the load of metaverse operation.

[0005] Therefore, an object of the present disclosure is to reduce the burden on users required for metaverse operation. [Means for solving the problem]

[0006] A data processing device of a first aspect includes an acquisition unit that acquires input data indicating user input, a processing unit that performs a specific processing using a generation model that generates a three-dimensional image and a script according to the input data, and a control unit that controls the display content of a display unit based on a result of the specific processing, wherein the processing unit inputs a prompt indicating an instruction to generate a virtual space obtained from the input data into the generation model as a first prompt, thereby generating a specific virtual space corresponding to the first prompt, inputs a prompt indicating an instruction to generate an avatar capable of operating in the specific virtual space obtained from the input data into the generation model as a second prompt, thereby generating a specific avatar corresponding to the second prompt, inputs a prompt indicating an instruction to operate the specific avatar obtained from the input data into the generation model as a third prompt, thereby generating a specific script for causing the specific avatar to perform an operation corresponding to the third prompt in the specific virtual space, and the control unit displays the specific virtual space and the specific avatar generated by the processing unit on the display unit, and executes the specific script generated by the processing unit to operate the specific avatar displayed on the display unit.

[0007] A data processing device of a second aspect is a data processing device of the first aspect, and is provided with a first collection unit that collects user information regarding the specific virtual space of an administrative user who manages the specific virtual space, and the processing unit inputs a prompt indicating an instruction to customize the specific virtual space based on the user information collected by the first collection unit into the generation model as a fourth prompt, and generates the specific virtual space to which customization corresponding to the fourth prompt has been applied.

[0008] A data processing device of a third aspect is a data processing device of the first or second aspect, wherein the specific virtual space is accessible to the avatars of multiple users, the acquisition unit acquires an avatar name which is a display name of the avatar existing in the specific virtual space, and when identity verification has been completed for a special avatar whose avatar name acquired by the acquisition unit matches a predetermined name, the control unit attaches information indicating that identity verification has been completed to the special avatar and displays it on the display unit.

[0009] A data processing device of a fourth aspect is a data processing device of any of the first to third aspects, wherein the specific virtual space is accessible to the avatars of each of a plurality of users, and the data processing device is provided with a second collection unit that collects the content of a conversation between a first avatar and a second avatar, the avatars existing in the specific virtual space, the processing unit inputs a prompt indicating an instruction to generate an answer to a question obtained from a utterance of the first avatar collected by the second collection unit into the generation model as a fifth prompt, thereby generating a proposed answer corresponding to the fifth prompt, and the control unit displays the proposed answer generated by the processing unit on the display unit as a utterance of the second avatar.

[0010] A data processing device of a fifth aspect is a data processing device of any of the first to fourth aspects, wherein the specific virtual space is accessible to the avatars of a plurality of users, and the data processing device is provided with a third collection unit that collects the behavior of the avatars present in the specific virtual space as avatar information, and when a violating avatar, which is an avatar that has performed a predetermined prohibited behavior, is extracted based on the avatar information collected by the third collection unit, the control unit performs control such that in the specific virtual space, the violating avatar is only visible to the violating user, who is the user of the violating avatar, and the violating avatar is not visible to users other than the violating user.

[0011] A data processing device of a sixth aspect is a data processing device of the fifth aspect, wherein the control unit performs control such that, when the violating avatar makes a statement to a good avatar which is an avatar other than the violating avatar, the statement of the violating avatar is only visible to the violating user in the specific virtual space, and the statement of the violating avatar is not visible to the user of the good avatar.

[0012] A seventh aspect of the data processing method is a computer-executed process that acquires input data indicating a user input, performs a specific process using a generation model that generates a three-dimensional image and a script according to the input data, controls the display content of a display unit based on a result of the specific process, inputs a prompt indicating an instruction to generate a virtual space obtained from the input data into the generation model as a first prompt, generates a specific virtual space corresponding to the first prompt, inputs a prompt indicating an instruction to generate an avatar capable of operating in the specific virtual space obtained from the input data into the generation model as a second prompt, generates a specific avatar corresponding to the second prompt, inputs a prompt indicating an instruction to operate the specific avatar obtained from the input data into the generation model as a third prompt, generates a specific script for causing the specific avatar to perform an operation corresponding to the third prompt in the specific virtual space, displays the generated specific virtual space and the specific avatar on the display unit, and executes the generated specific script to operate the specific avatar displayed on the display unit.

[0013] A data processing program of an eighth aspect causes a computer to execute the following processes: acquire input data indicating a user input; perform a specific process using a generation model that generates a three-dimensional image and a script according to the input data; control the display content of a display unit based on a result of the specific process; input a prompt indicating an instruction to generate a virtual space obtained from the input data into the generation model as a first prompt, thereby generating a specific virtual space corresponding to the first prompt; input a prompt indicating an instruction to generate an avatar capable of operating in the specific virtual space obtained from the input data into the generation model as a second prompt, thereby generating a specific avatar corresponding to the second prompt; input a prompt indicating an instruction to operate the specific avatar obtained from the input data into the generation model as a third prompt, thereby generating a specific script for causing the specific avatar to perform an operation corresponding to the third prompt in the specific virtual space; display the generated specific virtual space and the specific avatar on the display unit; and execute the generated specific script to operate the specific avatar displayed on the display unit. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a conceptual diagram illustrating an example of a configuration of a data processing system. [Diagram 2] FIG. 2 is a conceptual diagram showing an example of main functions of a data processing device and a smart device. [Diagram 3] 1 is a first block diagram showing an example of a functional configuration of a specific processing unit. FIG. [Figure 4] 13 is an example of an operational flow regarding an operation of a specific processing unit performing a first specific processing. [Diagram 5] 13 is an example of an operational flow regarding an operation of the identification processing unit performing a second identification process. [Figure 6] 13 is an example of an operational flow regarding an operation of the specific processing unit performing a third specific processing. [Figure 7] 13 is a first example of a metaverse app displayed on a display. [Figure 8]13 is a second example of a metaverse app displayed on a display. [Figure 9] 13 is a third example of a metaverse app displayed on a display. [Figure 10] 4 is a fourth example of a metaverse app displayed on a display. [Figure 11] 5 is a fifth example of a metaverse app displayed on a display. [Figure 12] 13 is a second block diagram showing an example of a functional configuration of a specific processing unit. FIG. [Figure 13] 6 is a sixth example of a metaverse app displayed on a display. [Figure 14] 7 is a seventh example of a metaverse app displayed on a display. [Figure 15] 8 is an eighth example of a metaverse app displayed on a display. [Figure 16] FIG. 13 is a third block diagram showing an example of a functional configuration of a specific processing unit. [Figure 17] 9 is a ninth example of a metaverse app displayed on a display. [Figure 18] 10 is a tenth example of a metaverse app displayed on a display. [Figure 19] 11 is an eleventh example of a metaverse app displayed on a display. [Figure 20] 1 shows a first emotion map onto which multiple emotions are mapped. [Figure 21] 13 shows a second emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, an example of an embodiment of a data processing device, a data processing method, and a data processing program according to the technique of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0018] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0022] (First embodiment) First, a first embodiment of a data processing system 10 according to the embodiment will be described. FIG. 1 shows an example of the configuration of a data processing system 10 according to the embodiment.

[0023] As shown in Fig. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server. An example of the smart device 14 is a smartphone. In this embodiment, the data processing device 12 is an example of a "data processing device" according to the technology of the present disclosure.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits input data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the input data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression (e.g., voice and / or text) that can be perceived by a human user. The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor. The camera 42 stores position data 45. The display 40A is an example of a "display unit" according to the technology of the present disclosure.

[0028] The communication I / F 44 is connected to a network 54. The communication I / F 44 and the communication I / F 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "data processing program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 60. The data generation model 58 and the emotion identification model 60 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 62. The reception output program 62 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 62 from the storage 50, and executes the read reception output program 62 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 62 executed on the RAM 48.

[0033] Here, an application that allows users to experience the metaverse (hereinafter, "metaverse app") is installed in the smart device 14. A user of the smart device 14 can have various experiences in the metaverse by executing the metaverse app. The various experiences include, for example, creating worlds and avatars as NFT (Non-Fungible Token) items, interacting with other users in the world, and buying and selling worlds and avatars.

[0034] In addition, the user who manages the data processing device 12 is a platform provider that provides a metaverse platform that can be experienced with a metaverse app. For example, the platform provider generates a world and an avatar on the platform according to input data through the data processing device 12, and monitors the users and avatars who use the platform.

[0035] Next, a description will be given of the processing of the specific processing unit 290 when the data processing device 12 performs specific processing for generating a three-dimensional image and a script according to input data indicating a user input. Fig. 3 is a first block diagram showing an example of the functional configuration of the specific processing unit 290.

[0036] As shown in FIG. 3, the specific processing unit 290 includes an acquisition unit 290A, a processing unit 290B, and a control unit 290C.

[0037] The acquiring unit 290A acquires input data indicating a user input accepted by the smart device 14. Specifically, the acquiring unit 290A acquires input data based on a text input, a voice input, or the like of the user accepted by the smart device 14.

[0038] The processing unit 290B performs a specific process using the data generation model 58. The data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of the data generation model 58 include generative AI such as ChatGPT and Visual ChatGPT, which is ChatGPT equipped with an image generation function. In this embodiment, the data generation model 58 is Visual ChatGPT. ChatGPT is, for example, an AI used for Internet searches.<URL: https: / / openai.com / blog / chatgpt> and internet searches<URL: https: / / aismiley.co.jp / ai_news / chatgpt-tsukattemita / > Since the above-mentioned method is publicly known as disclosed in, for example, Internet search, Visual ChatGPT is not described in detail here.<URL: https: / / gigazine.net / news / 20230314-chatgpt-visual-generating / > and<URL: https: / / dev.classmethod.jp / articles / tried-visual-chatgpt / > Since this is publicly known as disclosed in, for example, US Pat. No. 6,399,343, detailed description thereof will be omitted.

[0039] The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The data generation model 58 is an example of a "generation model" according to the technology of the present disclosure.

[0040] In addition, the above-mentioned identification process is composed of a first identification process that generates a world as a three-dimensional image according to input data, a second identification process that generates an avatar as the three-dimensional image, and a third identification process that generates a script for causing the avatar to perform an action in the world according to the input data.

[0041] Then, the processing unit 290B inputs a prompt indicating an instruction to generate a world obtained from the input data as a first prompt into the data generation model 58, and generates a specific world corresponding to the first prompt. The specific world is an original world of the user of the smart device 14 in which buildings and streetscapes corresponding to the first prompt are reproduced. The user has authority over the specific world, such as the ability to publish the specific world on a metaverse app, sell the specific world as an NFT item on the metaverse app, and customize the specific world by adding new objects to the specific world. The world is an example of a "virtual space" according to the technology of the present disclosure, and the specific world is an example of a "specific virtual space" according to the technology of the present disclosure.

[0042] Further, the processing unit 290B inputs a prompt indicating an instruction to generate an avatar operable in a specific world obtained from the input data as a second prompt into the data generation model 58 to generate a specific avatar corresponding to the second prompt. The specific avatar is an original avatar of the user of the smart device 14, in which the appearance and physique corresponding to the second prompt are reproduced. The user has authority over the specific avatar, such as the ability to make the specific avatar appear in a specific world, sell the specific avatar as an NFT item in a metaverse app, and customize the specific avatar, such as changing its appearance. The avatar may be a human, animal, robot, machine, or fictional creature that appears in the metaverse, and the type of the avatar is not particularly limited.

[0043] In addition, the processing unit 290B inputs a prompt indicating an instruction to operate the specific avatar obtained from the input data as a third prompt into the data generation model 58, and generates a specific script for causing the specific avatar to execute an operation corresponding to the third prompt in a specific world. The specific script is an original script of the user of the smart device 14 for causing the specific avatar to execute an operation corresponding to the third prompt. The user has authority over the specific script, such as the ability to sell the specific script as an NFT item in a metaverse app or customize the specific script by changing it.

[0044] The control unit 290C controls the display contents of the display 40A based on the result of the specific processing. Specifically, the control unit 290C applies the specific world and the specific avatar generated in the specific processing to the platform, and makes the specific world and the specific avatar appear in the metaverse. As a result, the specific world and the specific avatar are displayed on the display 40A. In addition, the control unit 290C executes a specific script corresponding to the specific avatar generated in the specific processing. As a result, the specific avatar displayed on the display 40A executes an operation based on the specific script.

[0045] Fig. 4 shows an example of an operation flow relating to the operation of the specific processing unit 290 performing the first specific processing for generating a world according to input data. The operation flow shown in Fig. 4 is performed by the specific processing unit 290 reading the specific processing program 56 from the storage 32, expanding it in the RAM 30, and executing it. The operation flow shown in Fig. 4 is automatically executed repeatedly, for example, at regular intervals.

[0046] 4, the identification processing unit 290 acquires input data indicating a user input accepted by the smart device 14. Then, the identification processing unit 290 proceeds to step S11.

[0047] In step S11, the identification processing unit 290 determines whether or not the input data acquired in step S10 satisfies the first trigger condition. If the identification processing unit 290 determines that the first trigger condition is satisfied (step S11: YES), the process proceeds to step S12. On the other hand, if the identification processing unit 290 determines that the first trigger condition is not satisfied (step S11: NO), the process ends the operation flow shown in Fig. 4. As an example, the identification processing unit 290 determines that the first trigger condition is satisfied when the input data indicates an instruction to generate a world.

[0048] In step S12, the identification processing unit 290 inputs a prompt indicating an instruction to generate a world obtained from the input data acquired in step S10 to the data generation model 58 as a first prompt, and acquires a result of the first identification process based on the output of the data generation model 58. For example, the identification processing unit 290 inputs text "Please generate a world represented by this text. A streetscape with trees" as the first prompt to the data generation model 58. Here, the identification processing unit 290 acquires a specific world as a result of the first identification process. Then, the identification processing unit 290 proceeds to step S13.

[0049] In step S13, the specific processing unit 290 controls the display contents of the display 40A based on the result of the first specific processing. Specifically, the specific processing unit 290 applies the specific world acquired in step S12 to the platform, and makes the specific world appear on the metaverse. As a result, the specific world is displayed on the display 40A. Then, the specific processing unit 290 ends the operation flow shown in FIG. 4.

[0050] Fig. 5 shows an example of an operation flow relating to the operation of the specific processing unit 290 performing the second specific processing to generate an avatar according to input data. The operation flow shown in Fig. 5 is performed by the specific processing unit 290 reading the specific processing program 56 from the storage 32, expanding it in the RAM 30, and executing it. The operation flow shown in Fig. 5 is automatically executed repeatedly, for example, at regular intervals.

[0051] 5, the specific processing unit 290 acquires input data indicating a user input accepted by the smart device 14. Then, the specific processing unit 290 proceeds to step S21.

[0052] In step S21, the specific processing unit 290 determines whether or not the input data acquired in step S20 satisfies the second trigger condition. If the specific processing unit 290 determines that the second trigger condition is satisfied (step S21: YES), the process proceeds to step S22. On the other hand, if the specific processing unit 290 determines that the second trigger condition is not satisfied (step S21: NO), the process ends the operation flow shown in Fig. 5. As an example, the specific processing unit 290 determines that the second trigger condition is satisfied when the input data indicates an instruction to generate an avatar.

[0053] In step S22, the identification processing unit 290 inputs a prompt indicating an instruction to generate an avatar obtained from the input data acquired in step S20 to the data generation model 58 as a second prompt, and acquires a result of the second identification process based on the output of the data generation model 58. For example, the identification processing unit 290 inputs text "Please generate an avatar represented by this text. A man wearing glasses" as the second prompt to the data generation model 58. Here, the identification processing unit 290 acquires a specific avatar as a result of the second identification process. Then, the identification processing unit 290 proceeds to step S23.

[0054] In step S23, the identification processing unit 290 controls the display contents of the display 40A based on the result of the second identification processing. Specifically, the identification processing unit 290 applies the specific avatar acquired in step S22 to the platform, and makes the specific avatar appear on the metaverse. As a result, the specific avatar is displayed on the display 40A. Then, the identification processing unit 290 ends the operation flow shown in FIG. 5.

[0055] Fig. 6 shows an example of an operation flow diagram of the third specific processing performed by the specific processing unit 290 to generate a script according to input data. The operation flow shown in Fig. 6 is performed by the specific processing unit 290 reading out the specific processing program 56 from the storage 32, expanding it in the RAM 30, and executing it. The operation flow shown in Fig. 6 is automatically and repeatedly executed, for example, at regular intervals.

[0056] 6, the specific processing unit 290 acquires input data indicating a user input accepted by the smart device 14. Then, the specific processing unit 290 proceeds to step S31.

[0057] In step S31, the specific processing unit 290 determines whether or not the input data acquired in step S30 satisfies a third trigger condition. If the specific processing unit 290 determines that the third trigger condition is satisfied (step S31: YES), the process proceeds to step S32. On the other hand, if the specific processing unit 290 determines that the third trigger condition is not satisfied (step S31: NO), the process ends the operation flow shown in Fig. 6. As an example, the specific processing unit 290 determines that the third trigger condition is satisfied when the input data indicates an instruction to move a specific avatar displayed on the display 40A.

[0058] In step S32, the identification processing unit 290 inputs a prompt indicating an instruction to move the specific avatar obtained from the input data obtained in step S30 as a third prompt to the data generation model 58, and obtains a result of the third identification process based on the output of the data generation model 58. For example, the identification processing unit 290 inputs text "This text represents the movement of the avatar. Please operate the avatar. Cross the road" as the third prompt to the data generation model 58. Here, the identification processing unit 290 obtains a specific script as a result of the third identification process. Then, the identification processing unit 290 proceeds to step S33.

[0059] In step S33, the identification processing unit 290 controls the display contents of the display 40A based on the result of the third identification process. Specifically, the identification processing unit 290 executes the identification script corresponding to the identification avatar acquired in step S32. As a result, the identification avatar displayed on the display 40A executes an operation based on the identification script. Then, the identification processing unit 290 ends the operation flow shown in FIG.

[0060] Next, the results of the specific processing displayed on the display 40A will be described with reference to FIGS. FIG. 7 is a first display example of the metaverse app displayed on display 40A.

[0061] The display example shown in FIG. 7 includes a text input section 70 that accepts text input by the user of the smart device 14, and an information display section 72 that displays information about the world, avatars, and the like.

[0062] In FIG. 7, the text "Please generate a world represented by this text. A streetscape with trees." is input to the text input unit 70. In this case, the specific processing unit 290 acquires the input data of the text input unit 70 and inputs the input data to the data generation model 58 as a first prompt. Then, the specific processing unit 290 applies the specific world output from the data generation model 58 to the platform, and makes the specific world appear on the metaverse. As a result, the specific world in which a streetscape with trees is reproduced is displayed on the information display unit 72 shown in FIG. 7.

[0063] 8 is a second display example of the metaverse app displayed on display 40 A. As an example, FIG 8 shows a case where the user has input additional text into text input section 70 shown in FIG 7.

[0064] In FIG. 8, the text "Please generate a world represented by this text. Streetscape with trees. Car." is input to the text input unit 70. In this case, the specific processing unit 290 acquires the input data of the text input unit 70 and inputs the input data to the data generation model 58 as a first prompt. Then, the specific processing unit 290 applies the specific world output from the data generation model 58 to the platform, and makes the specific world appear on the metaverse. As a result, the information display unit 72 shown in FIG. 8 displays the specific world in which the car C has been added to the streetscape shown in FIG. 7.

[0065] Fig. 9 is a third display example of the metaverse app displayed on display 40A. As an example, Fig. 9 shows a generation screen for generating an avatar. The display example shown in Fig. 9 includes a text input section 70 and an information display section 72, similar to Fig. 7.

[0066] In FIG. 9, the text "Please generate an avatar represented by this text. A man wearing glasses" is input to the text input unit 70. In this case, the identification processing unit 290 acquires the input data of the text input unit 70 and inputs the input data to the data generation model 58 as a second prompt. Then, the identification processing unit 290 applies the specific avatar output from the data generation model 58 to the platform, causing the specific avatar to appear on the metaverse. As a result, the information display unit 72 shown in FIG. 9 displays a specific avatar A that reproduces a man wearing glasses.

[0067] Fig. 10 is a fourth display example of the metaverse app displayed on the display 40A. As an example, Fig. 10 shows a case where the user of the smart device 14 has made the specific avatar A generated in Fig. 9 appear in the specific world shown in Fig. 7. In Fig. 10, the specific avatar A is located on the sidewalk on the right side of the figure in the specific world displayed on the information display unit 72.

[0068] 10, the text "This text represents the avatar's action. Please operate the avatar. Cross the road" is input to the text input unit 70. In this case, the identification processing unit 290 acquires the input data of the text input unit 70 and inputs the input data to the data generation model 58 as a third prompt. Then, the identification processing unit 290 executes the specific script corresponding to the specific avatar A output from the data generation model 58.

[0069] Fig. 11 is a fifth display example of the metaverse app displayed on display 40A. As an example, Fig. 11 shows a state after a specific avatar A shown in Fig. 10 executes an action based on a specific script. As shown in Figs. 10 and 11, based on the execution of a specific script, the specific avatar A moves from the walkway on the right side of the figure to the walkway on the left side of the figure in the specific world displayed on information display unit 72.

[0070] As described above, in the data processing device 12, the processing unit 290B of the specific processing unit 290 inputs a prompt indicating an instruction to generate a world obtained from the input data as a first prompt to the data generation model 58, and generates a specific world corresponding to the first prompt. The processing unit 290B also inputs a prompt indicating an instruction to generate an avatar that can operate in the specific world obtained from the input data as a second prompt to the data generation model 58, and generates a specific avatar corresponding to the second prompt. The processing unit 290B also inputs a prompt indicating an instruction to operate the specific avatar obtained from the input data as a third prompt to the data generation model 58, and generates a specific script for making the specific avatar execute an operation corresponding to the third prompt in the specific world. Then, the control unit 290C of the specific processing unit 290 causes the specific world and specific avatar generated by the processing unit 290B to be displayed on the display 40A, and executes the specific script generated by the processing unit 290B to operate the specific avatar displayed on the display 40A.

[0071] With the above configuration, the data processing device 12 can reduce the burden on the user required for metaverse operation. In detail, the data processing device 12 can generate a specific world, a specific avatar, and a specific script based on the user's text input as user input, so that the hurdle to use the metaverse can be lowered for users who have not used the metaverse before due to production costs, production man-hours, etc. Furthermore, in the data processing device 12, a specific world, etc. is generated based on the output of the data generation model 58, so that various specific worlds, etc. can be easily generated by inputting various texts.

[0072] Second Embodiment Next, a second embodiment of the data processing system 10 according to the present invention will be described while omitting or simplifying parts that overlap with the above embodiment.

[0073] FIG. 12 is a second block diagram showing an example of the functional configuration of the specific processing unit 290. As shown in FIG. As shown in FIG. 12, the specific processing unit 290 includes an acquisition unit 290A, a processing unit 290B, a control unit 290C, and a collection unit 290D.

[0074] The collection unit 290D collects user information on the specific world of an administrative user who manages the specific world. The administrative user is a user who has the authority to publish the specific world, sell the specific world, customize the specific world, and the like. For example, the administrative user may be a user of the smart device 14 who generated the specific world, or a user of the smart device 14 who purchased the specific world from the user. The user information may be, for example, the input history of the text input by the administrative user to the text input unit 70 and the schedule of the administrative user input to the schedule function of the metaverse app. The collection unit 290D is an example of a "first collection unit" according to the technology of the present disclosure.

[0075] In the second embodiment, the processing unit 290B inputs a prompt indicating an instruction to customize a specific world based on the user information collected by the collection unit 290D as a fourth prompt to the data generation model 58, and generates a specific world to which the customization corresponding to the fourth prompt is applied. An example of application of customization to a specific world by the processing unit 290B will be described below with reference to FIG.

[0076] 13 is a sixth display example of the metaverse app displayed on display 40A. As an example, FIG. 13 shows a specific world to which customization corresponding to the fourth prompt is applied. The display example shown in FIG. 13 includes a text input section 70 and an information display section 72, similar to FIG. 7.

[0077] In FIG. 13, the text "Please generate a world to which the customization represented by this text is applied. School" is input to the text input unit 70. The text is not based on the input of the administrative user, but is automatically input by the processing unit 290B based on the user information. For example, assume that "host a social event at school" is input as the administrative user's plan for three days later in the schedule function of the metaverse app. In this case, the collection unit 290D collects the text information "administrator user will hold a social event at school" as user information. The processing unit 290B extracts, for example, "school" from the text information collected by the collection unit 290D, and inputs the above text indicating an instruction to customize a specific world to the text input unit 70.

[0078] Next, the processing unit 290B acquires the input data of the text input unit 70 and inputs the input data as a fourth prompt to the data generation model 58. Then, the processing unit 290B applies the specific world output from the data generation model 58 to the platform, and makes the specific world appear on the metaverse. As a result, the information display unit 72 shown in FIG. 13 displays the specific world showing the school to which the customization corresponding to the fourth prompt has been applied.

[0079] As described above, in the data processing device 12, the collection unit 290D of the specific processing unit 290 collects user information related to the specific world of the administrative user who manages the specific world. Then, the processing unit 290B of the specific processing unit 290 inputs a prompt indicating an instruction to customize the specific world based on the user information collected by the collection unit 290D as a fourth prompt to the data generation model 58, and generates a specific world to which the customization corresponding to the fourth prompt is applied. As a result, according to the data processing device 12, the specific world is automatically customized based on the user information, so that the hurdle to use the metaverse can be lowered for users who have not used the metaverse before due to the service operation load.

[0080] (Third embodiment) Next, a third embodiment of the data processing system 10 according to the embodiment will be described while omitting or simplifying the overlapping parts with the above embodiment. Here, the specific world in the third embodiment is accessible to the avatars of each of the multiple users. Specifically, the specific world is published on the metaverse app, and multiple users who use the metaverse app can enter and exit using their own avatars. Each user can use their own avatar in the specific world to converse with other avatars through a chat function, or buy and sell NFT items such as the specific world and the specific avatar.

[0081] Here, an example of the functional configuration of the specific processing unit 290 in the third embodiment is similar to the configuration shown in FIG. 3 described in the first embodiment.

[0082] In the third embodiment, the acquisition unit 290A acquires an avatar name, which is a display name of an avatar existing in a specific world. Here, the metaverse app is designed to register the avatar name of an avatar when each user sets his / her avatar. The avatar name registered by each user is stored in the storage 32 after being associated with the identification information of the user and the avatar.

[0083] In the third embodiment, when identity verification has been completed for a special avatar whose avatar name acquired by the acquisition unit 290A matches a predetermined name, the control unit 290C causes information indicating that identity verification has been completed (hereinafter, "confirmed information 82B (see FIG. 14)") to be attached to the special avatar and displayed on the display 40A. The predetermined names are names that are contrary to public order and morals, names of famous people, etc. In the third embodiment, the predetermined names are "famous people's names".

[0084] Here, the platformer registers names that it considers to be "famous person names" as name information in the storage 32 from time to time. When the avatar name matches the name registered in the name information at the time of avatar setting, the metaverse app is designed to request the user to submit identity verification information for identity verification. When the platformer determines that the user is the same person as the person whose name is registered in the name information based on the submitted identity verification information, it adds certification information that proves the authenticity to the avatar name of the user stored in the storage 32. At this time, when the control unit 290C displays a special avatar on the display 40A, if the certification information has been added to the avatar name of the special avatar, it determines that identity verification has been completed, and attaches the confirmed information 82B to the special avatar and displays it on the display 40A.

[0085] Next, a display example of a special avatar on display 40A will be described with reference to Fig. 14. Fig. 14 is a seventh display example of a metaverse app displayed on display 40A. As an example, Fig. 14 shows a specific world in which multiple special avatars are displayed on information display section 72.

[0086] 14, avatar 80 and avatar 82 are displayed as multiple special avatars. In the following, it is assumed that avatar 80 is a special avatar for which identity verification has not been completed, and avatar 82 is a special avatar for which identity verification has been completed.

[0087] As shown in FIG. 14, avatar name information 80A and avatar name information 82A indicating the avatar names are displayed above the heads of avatar 80 and avatar 82. The avatar name indicated by avatar name information 80A is "xyz" which matches the name registered in the name information stored in storage 32, and the avatar name indicated by avatar name information 82A is "abc" which matches the name registered in the name information. At this time, confirmation information 82B indicated by a star mark (☆) is displayed at the end of the avatar name of avatar 82, which is a special avatar for which identity verification has been completed. In contrast, nothing is displayed at the end of the avatar name of avatar 80, which is a special avatar for which identity verification has not been completed.

[0088] As described above, in the data processing device 12, the acquisition unit 290A of the specific processing unit 290 acquires the avatar name of an avatar existing in a specific world from the storage 32. Then, when identity verification has been completed for a special avatar whose avatar name acquired by the acquisition unit 290A matches the name of a celebrity, the control unit 290C of the specific processing unit 290 attaches the confirmed information 82B to the special avatar and displays it on the display 40A. As a result, according to the data processing device 12, it is possible to determine whether identity verification has been completed for a special avatar that matches the name of a celebrity based on the presence or absence of the confirmed information 82B, thereby reducing damage caused by impersonation of a celebrity in a specific world.

[0089] (Fourth embodiment) Next, a fourth embodiment of the data processing system 10 according to the present invention will be described while omitting or simplifying the overlapping parts with the above-mentioned embodiments. Here, the specific world in the fourth embodiment is accessible to the avatars of each of the multiple users, as in the third embodiment. In addition, an example of the functional configuration of the specific processing unit 290 in the fourth embodiment is similar to the configuration shown in FIG. 12 described in the second embodiment.

[0090] In the fourth embodiment, the collection unit 290D collects the contents of a conversation between a first avatar and a second avatar, which are avatars existing in a specific world. The first avatar is an avatar of a user other than the administrative user who manages the specific world. The second avatar is an avatar of the administrative user. In the fourth embodiment, as an example, the first avatar is an avatar of a user who is considering purchasing an NFT item such as a specific world and a specific avatar managed by the administrative user. In addition, the second avatar is a specific avatar generated by the administrative user who serves customers regarding the sale of the NFT item. In the following, the first avatar may be referred to as a "customer avatar" and the second avatar as a "store clerk avatar". The collection unit 290D is an example of a "second collection unit" according to the technology disclosed herein.

[0091] In the fourth embodiment, the processing unit 290B inputs a prompt indicating an instruction to generate an answer to a question obtained from a statement of a customer avatar collected by the collection unit 290D as a fifth prompt into the data generation model 58, and generates a proposed answer corresponding to the fifth prompt.

[0092] In the fourth embodiment, the control unit 290C causes the display 40A to display the proposed answer generated by the processing unit 290B as a statement by the store clerk avatar.

[0093] Next, a specific example of the content of a conversation between a customer avatar and a store clerk avatar will be described with reference to Fig. 15. Fig. 15 is an eighth display example of the metaverse app displayed on display 40A. As an example, Fig. 15 shows a specific world in which a customer avatar and a store clerk avatar are displayed in information display section 72.

[0094] 15, an avatar 80 is displayed as a customer avatar, and an avatar 84 is displayed as a store clerk avatar. In addition, avatar name information 80A and avatar name information 84A indicating the names of the avatars are displayed above the heads of the avatars 80 and 84, respectively.

[0095] As shown in Fig. 15, above the avatar name information 80A, utterance information 80B indicating the contents of the avatar 80's utterance is displayed. As an example, in Fig. 15, "Do you have any avatars wearing dresses?" is displayed as the utterance information 80B. The utterance information 80B indicates the contents entered by the user of the avatar 80 using the chat function of the metaverse app.

[0096] Moreover, above the avatar name information 84A, statement information 84B indicating the content of the statement made by the avatar 84 is displayed. As an example, in FIG. 15, "Yes. First of all, . . . " is displayed as the statement information 84B. The statement information 84B indicates the content of the proposed answer that is automatically displayed by the control unit 290C, not based on input from the user of the avatar 84, i.e., the administrative user. The flow of how the proposed answer is displayed as the statement information 84B will be described below.

[0097] First, the collection unit 290D collects the statement of the avatar 80, "Do you have any avatars wearing dresses?" as text information. The processing unit 290B extracts, for example, "dress" from the text information collected by the collection unit 290D, and generates a prompt indicating an instruction to generate an answer to the question obtained from the statement of the avatar 80. For example, the processing unit 290B generates the text "Please generate an answer to the question expressed by this text. You are a clerk who sells NFT items. Do you have any avatars wearing dresses?" as the prompt.

[0098] The processing unit 290B inputs the generated prompt as a fifth prompt to the data generation model 58. Then, the control unit 290C applies the proposed answer to the question output from the data generation model 58 to the platform and displays the proposed answer on the metaverse. As a result, the proposed answer corresponding to the fifth prompt is displayed as a comment from the avatar 84 in the comment information 84B shown in FIG.

[0099] As described above, in the data processing device 12, the collection unit 290D of the specific processing unit 290 collects the contents of the conversation between the customer avatar and the store clerk avatar present in the specific world. The processing unit 290B of the specific processing unit 290 inputs a prompt indicating an instruction to generate an answer to a question obtained from the utterances of the customer avatars collected by the collection unit 290D as a fifth prompt into the data generation model 58, and generates a proposed answer corresponding to the fifth prompt. Then, the control unit 290C of the specific processing unit 290 causes the proposed answer generated by the processing unit 290B to be displayed on the display 40A as the utterances of the store clerk avatar. In this way, according to the data processing device 12, an answer to a question from a customer avatar can be displayed in a specific world without being manually input.

[0100] Fifth embodiment Next, a fifth embodiment of the data processing system 10 according to the present invention will be described while omitting or simplifying the overlapping parts with the above-mentioned embodiments. Here, the specific world in the fifth embodiment is a world in which the avatars of each of a plurality of users can enter and exit, similarly to the third and fourth embodiments.

[0101] FIG. 16 is a third block diagram showing an example of the functional configuration of the specific processing unit 290. As shown in FIG. As shown in FIG. 16, the specific processing unit 290 includes an acquisition unit 290A, a processing unit 290B, a control unit 290C, a collection unit 290D, and an extraction unit 290E.

[0102] In the fifth embodiment, the collection unit 290D collects the actions of an avatar present in a specific world as avatar information. The actions of the avatar include the body posture and movement of the avatar, the position of the avatar, and the content of the avatar's statements. Therefore, by checking the avatar information, the platformer can grasp what movements an avatar made, where it moved, and what it said in a specific world. The collection unit 290D is an example of a "third collection unit" according to the technology of the present disclosure.

[0103] The extraction unit 290E extracts a violating avatar, which is an avatar that has performed a predetermined prohibited behavior, based on the avatar information collected by the collection unit 290D. The predetermined prohibited behaviors include, for example, obscene behavior, slander, insults, discriminatory remarks, threats, intimidating remarks, noise, molestation, stalking, theft, vandalism, trespassing, impersonation, and fraudulent solicitation.

[0104] In the fifth embodiment, when the extracting unit 290E extracts a violating avatar, the control unit 290C performs control so that the violating avatar can be recognized only by the violating user who is the user of the violating avatar in the specific world, and the violating avatar cannot be recognized by users other than the violating user. For example, as the control, the control unit 290C performs the control so that the violating avatar is not displayed in the specific world on the metaverse app executed by users other than the violating user, and the violating avatar is displayed only in the specific world on the metaverse app executed by the violating user.

[0105] Furthermore, when a violating avatar makes a statement to a good avatar that is an avatar other than the violating avatar, the control unit 290C performs control so that the statement of the violating avatar can be understood only by the violating user in the specific world, and the statement of the violating avatar cannot be understood by the user of the good avatar. For example, as this control, the control unit 290C performs control so that the statement of the violating avatar is not displayed in the specific world on the metaverse app executed by the user of the good avatar, and the statement of the violating avatar is displayed only in the specific world on the metaverse app executed by the violating user.

[0106] Next, a specific example of the above control by the control unit 290C will be described with reference to FIG. 17 to FIG. Fig. 17 is a ninth display example of a metaverse app displayed on display 40A. As an example, Fig. 17 shows a specific world in a metaverse app executed by a violating user, displayed on information display unit 72.

[0107] 17 displays, as a plurality of avatars, an avatar 80, an avatar 82, and an avatar 84. In the following, it is assumed that the avatar 80 is a violating avatar, and the avatar 82 and avatar 84 are good avatars.

[0108] Control unit 290C displays both the good avatar and the violating avatar in a specific world on the metaverse app executed by the violating user. As a result, information display unit 72 shown in FIG. 17 displays avatar 80, which is a violating avatar, and avatars 82 and 84, which are good avatars.

[0109] Fig. 18 is a tenth display example of the metaverse app displayed on display 40A. As an example, Fig. 18 shows a specific world on the metaverse app executed by a user of avatar 84, which is a good avatar, displayed on information display unit 72. Here, the specific worlds shown in Figs. 17 and 18 are the same.

[0110] Control unit 290C does not display the violating avatar in a specific world on the metaverse app executed by a user other than the violating user. As a result, unlike FIG. 17, only avatar 82 and avatar 84, which are good avatars, are displayed as multiple avatars in information display unit 72 shown in FIG.

[0111] Fig. 19 is an eleventh display example of a metaverse app displayed on display 40A. As an example, Fig. 19 shows a specific world in a metaverse app executed by a violating user, displayed on information display unit 72.

[0112] 19 displays, as a plurality of avatars, an avatar 80, an avatar 82, and an avatar 84. In the following, it is assumed that the avatar 80 is a violating avatar, and the avatar 82 and avatar 84 are good avatars.

[0113] Control unit 290C displays the statement of the violating avatar in a specific world on the metaverse app executed by the violating user. As a result, in information display unit 72 shown in FIG. 19, statement information 80B indicating the statement content of avatar 80 is displayed above avatar name information 80A of avatar 80, which is a violating avatar. As an example, in FIG. 19, "Aren't you interested in a deal that's sure to make you money?" is displayed as statement information 80B. The statement information 80B indicates the content entered by the violating user using the chat function of the metaverse app.

[0114] Here, the control unit 290C does not display the statement of the violating avatar in a specific world on the metaverse app executed by the user of the good avatar. Therefore, even if the violating avatar makes the above statement, the display content of the display 40A of the smart device 14 owned by the user of the good avatar will be as shown in FIG. 18, and the statement will not be displayed.

[0115] As described above, in the data processing device 12, the collection unit 290D of the identification processing unit 290 collects the actions of avatars present in a specific world as avatar information. Then, when the extraction unit 290E extracts a violating avatar, the control unit 290C of the identification processing unit 290 performs control so that the violating avatar can be known only from the violating user in the specific world, and cannot be known from users other than the violating user.

[0116] Here, it is possible for the platform to take strong sanctions, such as revoking the account of a violating user when a violating avatar is identified in a specific world. However, taking such strong sanctions is not necessarily an effective measure, as it is expected that the violating user will become enraged and take more serious prohibited actions in other metaverses.

[0117] Therefore, in the specific world according to the fifth embodiment, the prohibited actions of the violating avatar are left unattended in a state where the violating avatar cannot be recognized by users other than the violating user based on the above configuration. As a result, according to the data processing device 12 according to the fifth embodiment, even if the violating avatar performs a prohibited action, users other than the violating user will not feel uncomfortable. Furthermore, according to the data processing device 12, the violating user can be made to recognize that he or she is free to perform trolling actions at first glance.

[0118] In addition, in the data processing device 12, when a violating avatar makes a statement to a good avatar, the control unit 290C of the specific processing unit 290 performs control so that the statement of the violating avatar can be known only from the violating user in the specific world, and cannot be known from the user of the good avatar. In the specific world according to the fifth embodiment, based on the configuration, the statement of the violating avatar is left in a state in which the statement of the violating avatar cannot be known from the user of the good avatar. As a result, according to the data processing device 12 according to the fifth embodiment, even if the violating avatar makes a statement, the user of the good avatar will not feel uncomfortable. Furthermore, according to the data processing device 12, the violating user can be made to recognize that he or she is free to make statements at first glance.

[0119] (others) In the above embodiment, the data generation model 58 is Visual ChatGPT, but is not limited thereto. For example, the data generation model 58 may be a combination of ChatGPT and an image generation AI that inputs text data output by ChatGPT and generates image data corresponding to the text data.

[0120] In the above embodiment, the action of the specific avatar A based on the execution of the specific script is shown as crossing a road (see Figs. 10 and 11), but is not limited thereto. The action of the specific avatar A based on the execution of the specific script may be any action that the avatar can perform in the metaverse app, such as changing posture, jumping, running, and dancing. In addition, the action of the specific avatar A based on the execution of the specific script is not limited to performing a single action among the above, and may be a combination of multiple actions.

[0121] In the above embodiment, the control for making the violating avatar undetectable is to not display the violating avatar in a specific world on a metaverse app executed by a user other than the violating user, but is not limited to this. For example, the control may be to make the displayed violating avatar transparent in a specific world on a metaverse app executed by a user other than the violating user.

[0122] In the above embodiment, the control for making it impossible to grasp the remarks of the violating avatar is to not display the remarks of the violating avatar in a specific world on the metaverse app executed by a user of a good avatar, but is not limited to this. For example, the control may be to make the remarks of the violating avatar displayed in a specific world on the metaverse app executed by a user of a good avatar transparent.

[0123] The processing unit 290B of the identification processing unit 290 may estimate the user's emotion using the emotion identification model 60, and perform identification processing using the user's emotion and the data generation model 58.

[0124] The emotion identification model 60 may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 60 may determine the emotion of the user according to an emotion map (see FIG. 20), which is a specific mapping.

[0125] FIG. 20 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0126] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0127] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0128] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0129] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0130] The emotion identification model 60 inputs user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 21. FIG. 21 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0131] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0132] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0133] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0134] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0135] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing the specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0136] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0137] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0138] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0139] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[0140] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0141] 10 Data Processing System 12 Data Processing Device 14 Smart Devices

Claims

1. an acquisition unit for acquiring input data indicative of a user input; A processing unit that performs specific processing using a generative model that generates a three-dimensional image and a script according to the input data; A control unit that controls a display content of a display unit based on a result of the identification process, The processing unit includes: inputting a prompt indicating an instruction to generate a virtual space obtained from the input data as a first prompt into the generative model, and generating a specific virtual space corresponding to the first prompt; inputting a prompt indicating an instruction to generate an avatar operable in the specific virtual space obtained from the input data as a second prompt into the generative model, thereby generating a specific avatar corresponding to the second prompt; inputting a prompt indicating an instruction to move the specific avatar obtained from the input data as a third prompt into the generative model, and generating a specific script for causing the specific avatar to execute an action corresponding to the third prompt in the specific virtual space; The control unit is displaying the specific virtual space and the specific avatar generated by the processing unit on the display unit, and executing the specific script generated by the processing unit to operate the specific avatar displayed on the display unit; Data processing device.

2. a first collection unit that collects user information related to the specific virtual space of an administrator user who manages the specific virtual space; The processing unit inputs a prompt indicating an instruction to customize the specific virtual space based on the user information collected by the first collection unit as a fourth prompt to the generation model, and generates the specific virtual space to which customization corresponding to the fourth prompt is applied.

2. A data processing apparatus according to claim 1.

3. the specific virtual space is accessible to the avatars of a plurality of users; The acquisition unit acquires an avatar name which is a display name of the avatar existing in the specific virtual space, When identity verification has been completed for a special avatar whose avatar name acquired by the acquisition unit matches a predetermined name, the control unit causes the display unit to display information indicating that identity verification has been completed attached to the special avatar.

2. A data processing apparatus according to claim 1.

4. the specific virtual space is accessible to the avatars of a plurality of users; a second collection unit that collects conversation content between a first avatar and a second avatar, the first avatar and the second avatar being the avatars existing in the specific virtual space; the processing unit inputs a prompt indicating an instruction to generate an answer to a question obtained from the utterance of the first avatar collected by the second collection unit as a fifth prompt into the generative model, thereby generating a proposed answer corresponding to the fifth prompt; The control unit causes the display unit to display the proposed answer generated by the processing unit as a statement of the second avatar.

2. A data processing apparatus according to claim 1.

5. the specific virtual space is accessible to the avatars of a plurality of users; a third collection unit that collects behavior of the avatar present in the specific virtual space as avatar information; When a violating avatar that has performed a predetermined prohibited action is extracted based on the avatar information collected by the third collection unit, the control unit performs control such that, in the specific virtual space, the violating avatar can be recognized only by a violating user that is a user of the violating avatar, and the violating avatar cannot be recognized by users other than the violating user.

2. A data processing apparatus according to claim 1.

6. When the violating avatar makes a statement to a good avatar that is the avatar other than the violating avatar, the control unit performs control such that the statement of the violating avatar can be understood only from the violating user in the specific virtual space, and the statement of the violating avatar cannot be understood by a user of the good avatar.

6. A data processing apparatus according to claim 5.

7. Obtaining input data indicative of user input; performing a specific process using a generative model that generates a three-dimensional image and a script according to the input data; Controlling the display content of a display unit based on a result of the identification process; inputting a prompt indicating an instruction to generate a virtual space obtained from the input data as a first prompt into the generative model, and generating a specific virtual space corresponding to the first prompt; inputting a prompt indicating an instruction to generate an avatar operable in the specific virtual space obtained from the input data as a second prompt into the generative model, thereby generating a specific avatar corresponding to the second prompt; inputting a prompt indicating an instruction to move the specific avatar obtained from the input data as a third prompt into the generative model, and generating a specific script for causing the specific avatar to execute an action corresponding to the third prompt in the specific virtual space; displaying the generated specific virtual space and the specific avatar on the display unit, and executing the generated specific script to operate the specific avatar displayed on the display unit; A data processing method in which processing is performed by a computer.

8. Obtaining input data indicative of user input; performing a specific process using a generative model that generates a three-dimensional image and a script according to the input data; Controlling the display content of a display unit based on a result of the identification process; inputting a prompt indicating an instruction to generate a virtual space obtained from the input data as a first prompt into the generative model, and generating a specific virtual space corresponding to the first prompt; inputting a prompt indicating an instruction to generate an avatar operable in the specific virtual space obtained from the input data as a second prompt into the generative model, thereby generating a specific avatar corresponding to the second prompt; inputting a prompt indicating an instruction to move the specific avatar obtained from the input data as a third prompt into the generative model, and generating a specific script for causing the specific avatar to execute an action corresponding to the third prompt in the specific virtual space; displaying the generated specific virtual space and the specific avatar on the display unit, and executing the generated specific script to operate the specific avatar displayed on the display unit; A data processing program for causing a computer to execute processing.

Images