Portrait voice right protection system, protection method and protection program

Abstract

To provide a portrait voice right management system capable of proving that data of human voice is unique by associating and managing the human voice with personal information on the owner of the voice and of efficiently and securely managing and utilizing voice data by guaranteeing the voice data to a "portrait voice right" corresponding to a portrait right.SOLUTION: In a portrait voice right management system, a guarantee system includes a voice data generation part for acquiring a sound signal to generate voice data D1, a relevant data acquisition part for acquiring personal information D2 including personal information, a portrait voice right management part for associating voice data with the personal information to generate portrait voice right data, and a plurality of nodes 90a to 90f for encrypting and storing at least part of the portrait voice right data. The nodes 90a to 90f aggregate at least part of the portrait voice right data at prescribed timing to form a block chain and share the block chain by a plurality of nodes to store the block chain as a distributed ledger.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to a system, method and program for protecting voice data, including characteristics unique to each individual person. [Background technology]

[0002] Voice data recorded from a human voice contains the so-called "voiceprint," which is a unique characteristic of each individual, and is composed of many elements such as frequency, pitch, tone, and intensity. Similar to fingerprints and retinal scans, voice data is used as one of the biometrics for identifying individual people (see, for example, Patent Document 1).

[0003] This voice data is highly personal because it can identify individual people with a high degree of certainty, and since its unauthorized acquisition and use could lead to a violation of privacy, it requires advanced security measures for its storage and transfer, as well as appropriate management and use. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2007-052496 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, when voice data is not properly managed as described above, it is often distributed anonymously and its ownership and origin are unclear, which creates a situation where unauthorized use and infringement of rights are likely to occur. In addition, because voice data is digital data, it can be easily copied or tampered with, which may result in unauthorized copying and misuse. Furthermore, voice data, like the voiceprints mentioned above, contains information that can identify individuals, and if this data is misused, it may lead to privacy violations or incidents or accidents.

[0006] Therefore, the present invention aims to solve the above problems by providing a monitoring system, monitoring method, and monitoring program that can efficiently and safely manage and utilize voice data by linking a person's voice with the personal information of the voice owner and managing it using a blockchain, thereby proving that the voice data is unique and even guaranteeing the "right to likeness," which is equivalent to the right to one's image. [Means for solving the problem]

[0007] In order to solve the above problems, the present invention provides a system for managing voice data as a voice record, comprising: a voice data generating unit that acquires an acoustic signal and generates voice data; a related data acquisition unit for acquiring related data including personal information; a voice data generating unit for generating voice data by associating the voice data with the related data; a data storage means for encrypting and storing at least a part of the portrait right data; The present invention is characterized by comprising:

[0008] The present invention also provides a method for managing voice data as a voice record, comprising the steps of: (1) a voice data generating step in which a voice data generating unit acquires an acoustic signal to generate voice data, a related data acquiring unit acquires related data including personal information, and a voice data generating step in which the voice data is associated with the related data to generate voice data; (2) a data storage step in which the data storage means encrypts and stores at least a part of the portrait right data; The present invention is characterized by comprising:

[0009] In the above invention, the data storage means includes a guarantee system linking unit that links with a guarantee system constructed on a communication network and encrypts and stores at least a part of the image right data; The security system includes a plurality of nodes that encrypt and store at least a portion of the image recognition data; The node aggregates at least a portion of the voting rights data at a predetermined timing, forms a block, links the block to an existing block, forms a blockchain, and shares the blockchain among a plurality of the nodes and stores it as a distributed ledger. It is preferable.

[0010] In the above invention, the voice data generation unit has a function of adding time information including a date and time when the acquired acoustic signal was acquired to the associated data, It is preferable that the associated data acquisition unit has a function of analyzing the correlation between the date and time when the added acoustic signal was acquired and information relating to age contained in the personal information, and including the analysis result in the associated data.

[0011] In the above invention, a voice feature extraction unit for extracting any one of frequency, duration, amplitude, intensity, and frequency components contained in the voice data as a feature parameter; a conversion unit that converts the voice quality or language of the voice data based on the feature parameters; a conversion parameter recording unit that records, as conversion parameters, characteristic parameters of the voice data after the conversion unit converts the voice quality or language; It is preferable that the device further comprises:

[0012] In addition, in the above invention, a voice feature extraction unit for extracting any one of frequency, duration, amplitude, intensity, and frequency components contained in the voice data as a feature parameter; a conversion unit that converts the voice quality or language of the voice data based on the feature parameters; a conversion parameter recording unit that records, as conversion parameters, characteristic parameters of the voice data after the conversion unit converts the voice quality or language; It is preferable that the device further comprises:

[0013] The above-mentioned system and method according to the present invention can be realized by executing the program of the present invention written in a specific language on a computer. That is, the program of the present invention can be installed in an IC chip or memory device of a general-purpose computer such as a portable terminal device, a smartphone, a wearable terminal, a mobile PC or other information processing terminal, a personal computer or a server computer, and executed on a CPU to construct a system having the above-mentioned functions and implement the method according to the present invention.

[0014] The program of the present invention can be distributed, for example, through a communication line, and can be transferred as a package application that runs on a stand-alone computer by recording it on a computer-readable recording medium. Specifically, the recording medium can be a magnetic recording medium such as a flexible disk or cassette tape, an optical disk such as a CD-ROM or DVD-ROM, or a RAM card. The computer-readable recording medium on which the program is recorded makes it possible to easily implement the above-mentioned system and method using a general-purpose computer or a dedicated computer, and also makes it easy to store, transport, and install the program. Effect of the Invention

[0015] As described above, these inventions allow for efficient and safe management and use of voice data by linking a person's voice with the personal information of the voice's owner and guaranteeing the rights. More specifically, the present invention allows for voice data to be protected as a "voice right," and this voice right allows for clear ownership of the voice data, making it possible to prevent unauthorized use and infringement of rights.

[0016] In addition, in the present invention, voice data is linked to the personal information of its owner via blockchain technology, which provides strong privacy protection and establishes the right to use one's voice, making it easier to take legal action against unauthorized use of voice data.

[0017] Furthermore, since the present invention makes it possible to properly protect voice data as a voice right, a new business model for safe commercialization becomes possible; clear ownership rights can be established for voice data; and the owner of the voice data can be appropriately compensated for its use, allowing voice data to be applied more freely and safely for creative purposes; and since voice data is authenticated and its reliability is ensured, it is expected that the use of voice data in a variety of industries will be promoted. [Brief description of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an overall configuration of a copyright management system according to an embodiment; [Diagram 2] A block diagram showing the internal configuration of a voice rights management server according to an embodiment. [Diagram 3] FIG. 2A is a block diagram showing the internal configuration of a smartphone according to an embodiment, and FIG. 2B is a block diagram showing the configurations of a voice data generating unit and a related data acquiring unit. [Figure 4] 1 is a block diagram showing an internal configuration of a guarantee system according to an embodiment. [Diagram 5] 10 is a sequence diagram showing an operation of the voice right management service according to the embodiment; FIG. [Figure 6]A sequence diagram showing the operation when regenerating voice data from obtaining voice data to obtaining related data in the embodiment. [Figure 7] FIG. 1 is a flow chart showing the procedure for NFT transfer in the image rights management system according to an embodiment. [Figure 8] 2 is an explanatory diagram illustrating a relationship between a public key and a private key in the image rights management system according to the embodiment; [Figure 9] An explanatory diagram illustrating a definition of a transaction related to the issuance, transfer, and cancellation of an NFT in an embodiment. [Figure 10] FIG. 1 is an explanatory diagram of a block chain of a copyright management system according to an embodiment. [Figure 11] FIG. 1 is an explanatory diagram of a block chain of a copyright management system according to an embodiment. [Figure 12] FIG. 1 is an explanatory diagram of a block chain of a copyright management system according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, with reference to the accompanying drawings, an embodiment of a method, a management system, and a management program for managing a voice right according to the present invention will be described in detail. In this embodiment, a case where a method for generating environmental value digital art according to the present invention is used in an energy trading service provided by a voice right management system is illustrated. Note that the embodiment shown below is an example of an apparatus for embodying the technical idea of ​​the present invention, and the technical idea of ​​the present invention does not specify the material, shape, structure, arrangement, etc. of each component part as described below. The technical idea of ​​the present invention may be modified in various ways within the scope of the claims.

[0020] (Overall configuration of the photography rights management system) In this embodiment, a voice rights management service is provided by a voice rights management system. FIG. 1 is a conceptual diagram showing a transaction form in the voice rights management service according to this embodiment. As shown in the figure, in this embodiment, a voice rights management service for managing and using voice rights is provided through a voice rights management system 1 constructed on a communication network 2. Specifically, in this service, the voice of a user U1 is recorded as voice data D1 through a smartphone 5 used by the user, and the voice data is managed and operated by a voice rights management server 3.

[0021] Here, voice rights refer to a person's voice and the personal information D2 of the voice owner that are linked and managed by blockchain, and are guaranteed as rights. This voice right allows clear ownership to be set for the voice data, making it possible to prevent unauthorized use and infringement of rights. This voice data record can be used to set voice rights for each age group (for example, from infancy to old age) for each user U1 at that time, and the unique value of the voice of each age group can be protected as a right and used.

[0022] More specifically, the voice rights management system 1 according to this embodiment is configured by connecting a voice rights management server 3 that manages voice data D1 as a voice right and a smartphone 5, which is an information processing terminal used by each user, to each other via a communication network 2. In this embodiment, the smartphone 5 is described as an example of an information processing terminal. Also, a guarantee system 4 that provides a blockchain interface service to guarantee the voice rights management is installed on the communication network 2, and the guarantee system 4 can be used to connect to a distributed ledger system such as a blockchain.

[0023] The communication network 2 is an IP network such as the Internet using the communication protocol TCP / IP, and is a distributed communication network constructed by interconnecting various communication lines (public lines such as telephone lines, ISDN lines, ADSL lines, and optical lines, dedicated lines, fourth generation (4G) communication methods such as LTE, and fifth generation (5G) and later communication methods, as well as wireless communication networks such as Wifi (registered trademark) and Bluetooth (registered trademark). This IP network also includes LANs such as intranets (corporate networks) and home networks using 10BASE-T, 100BASE-TX, etc.

[0024] The voice rights management server 3 is a server device that manages and utilizes voice rights, and in this embodiment, has a function of generating, storing, and managing voice data D1 of a user U1 as voice rights data D3 through a smartphone 5, and can be realized by a single server device or a group of multiple server devices. The voice rights management server 3 is configured to virtually construct multiple functional modules on a CPU, and each functional module cooperates to execute processing. The voice rights management server 3 can also send and receive data through the communication network 2 using a communication function, and can present websites and provide services through browser software using a web server function.

[0025] On the other hand, the smartphone 5 is a portable information processing terminal device that uses wireless communication, and the smartphone 5 communicates wirelessly with relay points such as the wireless base station 22, and can receive communication services such as telephone calls and data communications while moving. Examples of this communication method include the 3G (3rd. Generation) method, the LTE (Long Term Evolution) method, the 4G method, the FDMA method, the TDMA method, the CDMA method, the W-CDMA method, and the PHS (Personal Handyphone System) method. The smartphone 5 is also equipped with various functions such as a digital camera function, an application software execution function, and a position information acquisition function using the GPS (Global Positioning System), and also includes mobile computers such as tablet PCs.

[0026] The location information acquisition function of the smartphone 5 is a function for acquiring and recording location information indicating the location of the smartphone itself. Examples of this location information acquisition function include, as shown in FIG. 1, a method of detecting the location of the smartphone itself using a signal from a satellite 21, such as GPS, and a method of detecting the location using radio wave strength from a mobile phone wireless base station 22 or a Wi-Fi communication access point.

[0027] The smartphone 5 includes a display unit 53a such as a liquid crystal display as a display unit that displays information, and an operation device such as an operation button for a user to perform an input operation, and the operation device includes a touch panel 52a as an input unit that is arranged superimposed on the liquid crystal display and acquires an operation signal by a touch operation that specifies a coordinate position on the liquid crystal display. Specifically, the touch panel 52a is an input device that inputs an operation signal by pressure or electrostatic detection by a touch operation using a user's fingertip or pen, and is configured by superimposing a liquid crystal display that displays graphics and a touch sensor that receives an operation signal corresponding to the coordinate position of the graphic displayed on the liquid crystal display.

[0028] (Internal structure of each device) Next, the internal structure of each device constituting the voice rights management system 1 described above will be described. Fig. 2 is a block diagram showing the internal configuration of the voice rights management server 3 according to this embodiment, and Fig. 3 is a block diagram showing the internal configuration of the smartphone 5 according to this embodiment. Note that the term "module" used in the description refers to a functional unit configured by hardware such as a device or equipment, software having the functions thereof, or a combination of these, for achieving a predetermined operation.

[0029] (1) Voice Rights Management Server 3 First, there will be described an internal configuration of the voice rights management server 3. The voice rights management server 3 is a server device disposed on the communication network 2, and is capable of transmitting and receiving data to and from each smartphone 5 via the communication network 2.

[0030] Specifically, the voice rights management server 3 includes a communication interface 31 that communicates data via the communication network 2, an authentication unit 33 that authenticates the authority of users and user terminals, a location information management unit 32 that collects and manages location information of each user terminal, a voice rights data control unit 36 ​​that controls the generation, storage, and management of voice rights data, an information distribution unit 34 that distributes information to each user, and various database groups 35a to 35b.

[0031] The database group includes a portrait right database 35a as a data storage means for storing the generated portrait right, a user database 35b for storing information about users, and an accumulated information database 35c for storing external information such as current affairs information, weather information, and economic information collected from the Internet. Each of these databases may be a single database, or may be divided into multiple databases and a relational database in which each data is linked to the other by setting a relationship between them.

[0032] The ivory right database 35a is a data storage means for encrypting and storing at least a part of the ivory right data. The data stored in the ivory right database 35a includes voice data uploaded by each user, time information and location information at the time the voice data was acquired, and personal information data, and these pieces of information are recorded as ivory right. The ivory right database 35a is equipped with advanced security functions and plays an important role in safely storing the user's voice data and related information. Each data item in the database is managed by a unique identifier, and multi-layered security measures are implemented to prevent unauthorized access and data tampering.

[0033] The information stored in the user database 35b includes authentication information that links an identifier (user ID, terminal ID) that identifies the user or the information processing terminal used by the user with a password, etc., and also includes personal information of the user linked to the user ID, the model of the terminal device, etc. The user database 35b also stores authentication history (access history) for each user or each user terminal, and information (including related information) regarding the portrait right for each user through a relationship with the portrait right database 35a.

[0034] The user database 35b can also store image information of official identification documents such as driver's licenses, passports, and My Number cards for the purpose of detailed authentication and identity verification of users. The user database 35b also stores image data and text data extracted from these identification documents, and information such as the face photo, name, address, issue date, and expiration date of the driver's license, information such as the face photo, name, nationality, issue date, and expiration date of the passport, and information such as the face photo, name, address, issue date, expiration date, and My Number card are also stored as user information. Since these personal information are highly confidential, they are encrypted and stored using strong encryption technology, and such image information and text data of the identification documents are used to ensure safety and meet legal requirements for the use of the service, such as user identity verification, age verification, and identity verification during transactions. For example, when buying, selling, or transferring portrait rights, these information are referenced to verify the identities of both parties in the transaction and prevent fraud and dishonesty.

[0035] The information stored in the accumulated information database 35c includes external information such as current news information, weather information, traffic information, stock price and exchange rate information, real estate price information, and sports and entertainment information. In detail, the accumulated information database 35c aggregates information for capturing the social background and environment when the voice data was generated, which makes it possible to reproduce and understand not only the voice data but also the social environment and cultural background at the time when the voice data was acquired. Specifically, current news information reflects important events and topics of the time, weather information indicates the climate and weather conditions at the time, traffic information indicates the means of transportation and traffic congestion, stock price and exchange rate information indicates economic trends, real estate price information indicates the local economic situation and demographics, and sports and entertainment information indicates the entertainment and interests of people at the time.

[0036] The authentication unit 33 is a module that establishes a communication session with each smartphone 5 through the communication interface 31 and performs authentication processing for each established communication session. In this authentication processing, authentication information is acquired from the smartphone 5 of the user who is the accessing user, and the user is identified by referring to the user database 35b, and their authority is authenticated. The authentication result by the authentication unit 33 (user ID, authentication time, session ID, etc.) is sent to the authentication right data control unit 36 ​​and is also accumulated in the user database 35b as an authentication history.

[0037] The location information management unit 32 is a module that acquires location information that is acquired on the user's smartphone 5 and sent to the voice rights management server 3. The location information management unit 32 links the identifiers (user ID, terminal ID, etc.) of the user and user terminal device identified by the authentication process by the authentication unit 33 with the location information and stores it as usage history in the user database 35b.

[0038] The portrait right data control unit 36 ​​is a module that executes and manages the generation (issuance), update, transfer and deletion of portrait right data, and updates the block chain to execute the generation, update, transfer or deletion of each portrait right data in cooperation with the guarantee system cooperation unit 36c. Specifically, the portrait right data control unit 36 ​​includes a voice data control unit 36a, a related data control unit 36b, and a guarantee system cooperation unit 36c.

[0039] The voice data control unit 36a is a module having a function of a voice data generation unit that acquires an acoustic signal of the user's voice and generates voice data. The related data control unit 36b is a module having a function of a related data acquisition unit that acquires related data including personal information. The personal information data control unit 36 ​​associates the voice data generated by the voice data control unit 36a with the related data acquired by the related data control unit 36b to generate personal information data.

[0040] The guarantee system cooperation unit 36c is a module that requests the guarantee system 4 on the network to perform processes necessary for the management of the ivory right, such as information related to the ivory right, security management, and storage of transfer records, and cooperates with the guarantee system 4 to proceed with the process. The ivory right data control unit 36 ​​manages the generation, transition, and transfer of the ivory right by coordinating with the guarantee system 4 through the guarantee system cooperation unit 36c. For example, each ownership is transferred by adding a public address related to the acquirer of the ivory right and changing the owner of each type of right certified by the blockchain.

[0041] The guarantee system 4, which is a blockchain interface service, includes a plurality of nodes that encrypt and store at least a portion of the voice recorder data. These nodes link the voice recorder data and related information included in the voice recorder data, aggregate at least a portion of the information at a predetermined timing, and form blocks. These blocks are linked to existing blocks to form a blockchain, which is shared by the plurality of nodes and stored as a distributed ledger.

[0042] The information distribution unit 34 is a module that, in response to access from a user, presents a website related to the voice rights management service, provides each user with a unique My Page, etc., and enables each user to individually view information regarding the voice rights they own. It also has a function of presenting an NFT trading platform for registering and trading voice rights as NFTs, and distributes the generated graphics and user interface of the NFT trading platform to each user via the communication interface 31.

[0043] (2) Smartphone 5 Next, a description will be given of the internal configuration of the smartphone 5. As shown in Fig. 3(a) and (b), the smartphone 5 is roughly composed of a communication interface 51, an input interface 52, an output interface 53, an application execution unit 54, and a memory 55.

[0044] The communication interface 51 is a communication device for making calls or data communications, and has a function of non-contact communication by radio or the like, and contact (wired) communication by a cable, adapter means, or the like. The input interface 52 is a device for inputting user operations, such as a mouse, keyboard, operation buttons, or touch panel 52a. The output interface 53 is a device for outputting video and audio, such as a display or speaker. In particular, the output interface 53 includes a display unit 53a such as a liquid crystal display, and this display unit is superimposed on the touch panel 52a, which is the input interface.

[0045] The memory 55 is a storage device that stores an OS (Operating System), firmware, programs for various applications, other data, etc., and in addition to a user ID that identifies a user, this memory 55 also stores application data downloaded from the voice rights management server 3, as well as various data processed by the application execution unit 54. In particular, in this embodiment, the memory 55 stores rights information, etc. obtained from the voice rights management server 3.

[0046] The application execution unit 54 is a module for executing applications such as a general OS, game applications, browser software, etc., and is usually realized by a CPU, etc. In this application execution unit 54, a voice rights management unit 543, a display data generation unit 546, a display control unit 545, and a position information acquisition unit 544 are virtually constructed by executing a program of the voice rights management application according to the present invention.

[0047] The portrait right management unit 543 is a module having a function of performing operations such as generating, acquiring, changing, and deleting portrait right data on the smartphone 5. In particular, the portrait right management unit 543 has a function as a portrait right data generation unit that generates portrait right data D3 by associating personal information D2 with voice data D1. The portrait right management unit 543 stores the generated portrait right data in the memory 55 in association with metadata such as a user ID, time, and location information. In this embodiment, the portrait right management unit 543 includes a voice data generation unit 541, an associated data acquisition unit 542, and a synchronization processing unit 540.

[0048] The voice data generation unit 541 is a module that acquires an acoustic signal and generates voice data, and receives the user's voice as an input of an acoustic signal, and records and saves it as digital data. When the user performs an operation such as "record voice" on the application, this part is activated, and the voice data is acquired using the built-in microphone or an external microphone, and the data is encoded in an appropriate format and saved. In addition, a timestamp is added to the recorded voice data, and information on when the data was acquired is also saved.

[0049] In this embodiment, the voice data generation unit 541 includes a language conversion unit 541a and a voice quality conversion unit 541b. The language conversion unit 541a is a module that converts the language of the voice data based on the feature parameters extracted by the voice feature extraction unit 542a. Specifically, the language conversion unit 541a recognizes phrases and words in the voice data uttered by the user through voice recognition processing, converts non-standard formats such as abbreviations, shorthands, and slang for the corresponding text into standard formats, and translates the text into the target language. In this translation processing, translation is performed by referring to dictionary data or using a neural network-based model, and, if necessary, converts the text into a non-standard format or style such as a colloquial style or a dialect with a specific accent. The text data whose language has been converted in this way is regenerated as voice data by the voice synthesis engine of the voice data generation unit 541.

[0050] On the other hand, the voice conversion unit 541b is a module that converts the voice quality of the voice data based on the feature parameters extracted by the voice feature extraction unit 542a. The voice conversion by the voice conversion unit 541b uses signal processing technology to change the characteristics of the input voice and regenerate a voice different from the original voice. Specifically, the voice conversion process by the voice conversion unit 541b involves changing the pitch (frequency) of the voice to change the pitch, or changing the speed (tempo) of the voice to shorten or extend the duration of the voice. At this time, it is also possible to use pitch shift technology to change only the pitch without changing the speed of the voice, or to change only the tempo without changing the pitch.

[0051] In addition, the voice conversion unit 541b can convert the voice into a voice of a different age or gender by changing the formant, which is an element that determines the characteristics of the voice, and can also add echo or reverb to add reverberation or delay to the voice, modulate the frequency or amplitude of the voice signal with modulation effects or filtering, or remove only a specific frequency band to create special voice effects such as a robot's voice or an alien's voice. By combining these processes, the voice conversion unit 541b can generate various voice effects, and detailed adjustments and customization are possible through user settings, making it possible to create realistic conversions and special voice effects.

[0052] In addition, when generating voice data in the voice data generating unit 541, the unit also has a function of singing a song by inheriting the feature parameters of the voice data acquired from the user. Specifically, the voice data generating unit 541 acquires the voices of various sentences and songs by recording or the like in order to cover various phrases, words, pitches, and pronunciations, and performs preprocessing such as noise removal and normalization on the voice data for the acquired voices, and extracts feature parameters such as Mel Frequency Cepstral Coefficients (MFCC), pitch, pronunciation speed, and sound intensity through the voice feature extracting unit 542a. Then, based on the extracted feature parameters, a voice synthesis model is constructed, and the voice of the target person is synthesized based on text such as lyrics and melody information.

[0053] On the other hand, the related data acquisition unit 542 is a module that acquires related data, including personal information, to be stored together with the voice data. This includes, for example, location information at the time of recording the voice data, information on the user's device, the recording environment (e.g., indoors, outdoors, presence or absence of noise, etc.), and other metadata. These related data can provide more detailed context and background information when analyzing and using the voice data. In this embodiment, the related data acquisition unit 542 has a voice feature extraction unit 542a, a conversion parameter recording unit 542b, and a processing information control unit 542c.

[0054] The voice feature extraction unit 542a is a module that extracts any of the frequency, duration, amplitude, intensity, or frequency components contained in the voice data as feature parameters. Specifically, the voice feature extraction unit 542a extracts the fundamental frequency indicating the pitch of an individual's voice, the resonance frequency (formant frequency) indicating the characteristics of an individual's vowels, the Mel-frequency Cepstrum Coefficients (MFCCs) for capturing specific characteristics and patterns of an individual's voice, the vibration pattern of the vocal cords, the spectral envelope which is the characteristics of the frequency components of the voice, the intensity or power (energy) of an individual's voice, the spectral shape, and the like.

[0055] The related data acquisition unit 542 is a module that records, in the related data, various parameters used when the voice quality or language is converted by the voice data generation unit 541. In particular, the conversion parameter recording unit 542b of the conversion parameter recording unit 542b provides a function of recording, as conversion parameters, characteristic parameters of the voice data after processing such as generation, conversion, and synthesis by the voice data generation unit 541.

[0056] The processing information control unit 542c is a module that controls various information required for processing such as voice generation, conversion, synthesis, and processing performed by the voice data generation unit 541 as processing information. The processing information includes conversion parameters (pitch, tempo, formant, effect, etc.) required for voice synthesis or voice conversion, as well as preset information, text data, character settings, music data, etc. Here, the text data includes text such as sentences and lyrics, as well as information required for language, pronunciation, accent, emphasis, speed, pronunciation in a specific language or dialect, and adjustment of the emphasis degree of words and phrases, which are written in text form. The character settings include information required to adjust the tone and intonation of the voice in accordance with the attributes and characteristics of the character, such as the gender, age, personality, emotion, and nationality of the person (person, animal, creature, etc.) of the synthetic voice. Furthermore, the music data includes information required for changing the pitch and speed of the voice in accordance with the melody and rhythm of the music, as well as for adjusting the emphasis and volume of a specific part.

[0057] Furthermore, this processing information can be preset information, that is, a specific voice profile that is either set in advance by the user or provided by the software. By appropriately combining this information, a wide variety of voice conversions and voice synthesis are possible, and by adjusting these conversion parameters according to the user's requests and purposes, a more natural and realistic voice can be generated.

[0058] Such processing information may be provided by, for example, a graphical user interface (GUI) provided by the GUI control unit 545a, with visual elements such as sliders, drop-down menus, and check boxes, allowing the user to intuitively adjust parameters, and information or commands may be directly input as text data into a text box. Also, the above-mentioned various settings may be prepared as presets, such as "old man" or "angry cat," and the user may select them through operation. Also, these profiles, audio data, and music data may be directly uploaded to the related data acquisition unit 542, or the application may be linked to an external service or database to automatically acquire and input information.

[0059] The synchronization processing unit 540 is a module that synchronizes the voice rights management processing on the smartphone 5 side with the voice rights management processing on the voice rights management server 3 side. Specifically, the voice rights management server 3 side predicts possible event processing based on the positions of other users' characters and objects, generates the event processing conditions on the voice rights management server 3 side, transmits the conditions to the smartphone 5 side, receives the event processing conditions by the synchronization processing unit 540, and executes the actual event processing and the graphic processing therefor by the GUI control unit 545a on the smartphone 5 side based on the event processing conditions received from the voice rights management server 3. The results of the event processing executed by the GUI control unit 545a on the smartphone 5 side (such as the outcome of a battle or mini-game, or points) are notified to the GUI control unit 545a on the voice rights management server 3 side through the synchronization processing unit 540, and are reflected in the subsequent game progress processing.

[0060] The display data generating unit 546 is a module that generates display data to be displayed on the display unit 53a. The display data is data generated by combining graphic data, image data, character data, video data, audio data, and other data, and generates a screen such as a list display of portrait rights or a detailed display of specific portrait rights data. The display process of the display data generated by the display data generating unit 546 is controlled by the display control unit 545.

[0061] The display control unit 545 is a module that controls the specific screen display based on the data generated by the display data generation unit 546. In this embodiment, the display control unit 545 has a GUI control unit 545a, realizes interaction with the user via the touch panel 52a and the display unit 53a, and performs appropriate screen transitions and information updates in response to user operations.

[0062] The GUI control unit 545a is a module in charge of displaying and controlling a graphical user interface (GUI) on the display of the smartphone 5. This GUI control unit 545a displays graphical elements such as various buttons, menus, icons, and windows on the display so that the user can intuitively operate the application, and performs appropriate reactions and actions in response to inputs such as touch operations from the user. The GUI control unit 545a operates in cooperation with the display data generation unit 546.

[0063] Based on the data and information generated by the display data generation unit 546, the GUI control unit 545a presents it to the user in a visual and easy-to-operate format. For example, when displaying a list of voice rights, the GUI control unit 545a controls design elements such as the design, layout, and color usage of the list. Furthermore, the GUI control unit 545a also plays a role in calling up appropriate application functions in response to user operations. For example, when the user taps the "Record new voice data" button, the GUI control unit 545a receives the operation and sends a command to the voice rights management unit 543 to start the voice data recording process.

[0064] The location information acquisition unit 544 is a module having a function of acquiring current location information by using the built-in GPS sensor of the smartphone 5 and communication information with a wireless base station. Specifically, the location information acquisition unit 544 acquires information by a Global Positioning System (GPS) using an artificial satellite, base station positioning by triangulation based on the radio wave strength from the base station and base station information, Wifi positioning using a database that combines the Wifi SSID (Service Set ID) and radio wave conditions with longitude and latitude, etc.

[0065] In this embodiment, the location information acquisition unit 544 also includes a time information acquisition unit 544a. The location information and time information acquired by the location information acquisition unit 544 are linked as metadata, which is related data, when the voice data is generated, and are stored as information on when and where the voice data was acquired. This function makes it possible to add a temporal and geographical context to the voice data, thereby enriching the story and background behind the voice data.

[0066] (3) Guarantee System 4 As described above, in this embodiment, a guarantee system 4 is provided that guarantees the copyright using blockchain technology. Specifically, as shown in FIG. 4, the guarantee system 4 includes a communication interface 43, an authentication unit 42, a rights management execution unit 44, an NFT transaction history database 41a, a key information database 41b, and an account database 41c.

[0067] The communication interface 43 is a module that transmits and receives data to and from other communication devices through the communication network 2, and in this embodiment, is connected to each voice rights management server 3. The authentication unit 42 is a computer or software having that function that verifies the legitimacy of the accessor, and executes authentication processing based on a user ID that identifies each user. In this embodiment, the authentication unit 42 obtains a user-specific public address, public key, user ID, password, etc. from the accessor's smartphone 5 through the communication network 2, and verifies the accessor's right and whether the accessor is the person in question by collating the information with the key information database 41b.

[0068] The rights management execution unit 44 is a module that handles the digital rights through NFT (Non-Fungible Token) technology. In this embodiment, the digital rights are also one of the so-called digital assets, and the uniqueness and rarity of the digital rights are verified and guaranteed through the rights management execution unit 44 based on the blockchain technology. The rights management execution unit 44 employs this NFT technology to verify and guarantee the uniqueness and rarity of the digital rights data.

[0069] In this embodiment, the voice rights data generated by the user is issued as an NFT, and the related information and attributes are stored on the blockchain. Through this process, the originality and ownership of the voice rights data are reliably protected, and unauthorized copying and distribution by third parties can be prevented. In addition, the voice rights data as an NFT is traded on the market, and users can buy and sell their voice rights as NFTs, creating new economic value.

[0070] Specifically, in this embodiment, the portrait right in the NFT is linked to the public account of the legitimate owner, and one can prove that one is the legitimate owner by presenting the public account relationship. Furthermore, only the legitimate owner can carry out transfer procedures such as assignment of the portrait right.

[0071] The rights management execution unit 44 has a function of coordinating with the guarantee system cooperation unit 36c on the voice rights management server 3 side, and this guarantee system cooperation function is requested by devices of other service institutions, for example the guarantee system cooperation unit 36c of the voice rights management server 3, to perform processes related to rights management, such as security management related to voice rights management using NFT, transaction records, and storage of service history, and proceeds with rights management processes through NFT by communicating with each of these devices. Specifically, the rights management execution unit 44 includes a rights information provision unit 44a, a public address management unit 44b, a validity verification unit 44c, and a data update unit 44d.

[0072] The right information providing unit 44a is a module that provides detailed right information when a user issues a cryptocurrency as an NFT via the Internet, etc. This provided information includes information such as the price of the cryptocurrency, the date of creation of the cryptocurrency (the user's age, generation, etc.), the current owner, and transfer benefits such as how many times it has been traded.

[0073] The public address management unit 44b is a module that manages users' public accounts (public addresses), and has a private key issuing function that issues a public address generated from a public key in a public key cryptosystem to identify a specific user, and a private key that can be paired with the public key to identify the public key and is used for electronic signatures for image rights management via the public address, and these issued public addresses and their associated key information are stored in the key information database 41b. When a user trades NFTs, this public address becomes a key that proves the legitimacy of the transaction, and the public address functions as the user's identifier on the blockchain and is used to uniquely identify the NFTs and transaction history owned by the user.

[0074] The legitimacy verification unit 44c is a module that verifies whether an operation is legitimate when a transfer or transaction of a imitation right is performed. Specifically, the legitimacy verification unit 44c verifies whether the transaction is performed by a legitimate owner by comparing the public address managed by the public address management unit 44b with the transaction information. This prevents fraudulent transactions and unauthorized access by third parties. For example, the legitimacy verification unit 44c verifies that the imitation right related to the NFT transaction belongs to the current owner through a legitimate transaction and has not been tampered with, and confirms the legitimacy of the imitation right using the public key of the current owner linked to the public account. In addition, all transactions related to the imitation right are stored in the NFT transaction history database 41a, and the legitimacy can be confirmed by comparing the NFT transaction history database 41a based on the public key.

[0075] The data update unit 44d is a module that updates information about the ivory right. When a user wants to change the information about the ivory right or when a user acquires a new ivory right, this part is activated. The data update unit adds new information to the blockchain and performs procedures to maintain consistency with previous information. For example, the data update unit 44d acquires information about the ivory right through the NFT, adds a public address for the new NFT owner, and changes the token owner certified in the distributed ledger, thereby transferring the ownership of the ivory right (NFT). The data update by the data update unit 44d is protected by high-level security, and double transfer and tampering of transaction history are firmly prevented.

[0076] (Method of managing portrait rights) The method for managing a portrait right of the present invention can be implemented by operating the portrait right management system 1 described above. Fig. 5 is a sequence diagram showing the operation of the portrait right management system. Note that the processing procedure described below is merely an example, and each process may be modified as much as possible. Furthermore, steps may be omitted, replaced, or added to the processing procedure described below as appropriate depending on the embodiment.

[0077] First, the voice rights management server 3 is accessed from the smartphone 5, which is a user terminal, and authentication processing begins (S201). Specifically, user U1 operates the voice rights management application on the smartphone 5 and attempts to access the voice rights management server 3. If the connection to the voice rights management server 3 is successful, the user is prompted to enter a user ID and password as login information, and if the user enters this information correctly, the server compares it with the authentication information stored in the user database 35b. If this authentication information matches, the user is successfully authenticated and the user is identified (S101).

[0078] On the user terminal side, if the authentication process is successful, voice data is acquired (S202) and related data is acquired (S203), and image data is generated (S204). In more detail, if the authentication process is successful on the smartphone 5 side, the voice data generating unit 541 records and stores an acoustic signal as digital data. Here, when the user U1 executes an operation such as "record voice" on the application, the acoustic signal of the voice is acquired as voice data via the built-in microphone of the smartphone 5 or an external microphone. The acquired voice data is encoded in an appropriate format, and at the same time, a timestamp is added, so that information on when the data was generated is also stored.

[0079] Next, the related data acquisition unit 542 collects various personal information D2 to be stored together with the voice data D1. This related data includes the current location information and time information of the place where the voice was recorded, the user's device information, details of the recording environment (e.g., whether the recording was made indoors or outdoors, whether there was background noise, etc.), and other metadata. This information is accumulated for the purpose of providing background and context under which the voice data was recorded when the voice data is analyzed or used in the future.

[0080] In this embodiment, when acquiring voice data (step S202) and related data (step S203), the voice quality, language, and other parameters of the original voice data may be converted as necessary to regenerate the voice data, and the related voice data may be generated based on the new voice data. Figure 6 shows the procedure for regenerating the voice data.

[0081] In step S202, an acoustic signal is acquired as voice data, and the voice feature extraction unit 542a extracts any one of the frequency, duration, amplitude, intensity, or frequency components contained in the voice data as feature parameters. These feature parameters are passed to the voice quality conversion unit 541b and the language conversion unit 541a.

[0082] In step S202a, the voice quality conversion unit 541b converts the voice quality of the voice data based on the characteristic parameters extracted by the voice feature extraction unit 542a. At this time, the voice quality conversion unit 541b acquires various information required for processing such as voice generation, conversion, synthesis, and processing as processing information through the processing information control unit 542c. This processing information includes conversion parameters (pitch, tempo, formant, effect, etc.) required for voice synthesis or voice conversion, as well as preset information, text data, character settings, music data, etc. For example, if the original voice data is the voice of a woman in her twenties, the age and gender can be changed by raising or lowering the frequency to raise or lower the pitch of the voice and at the same time adjusting some formants, making it possible to change the voice into the voice of an elderly person or the voice of a teenage idol. In addition, filters and effects can be applied to reproduce specific accents and pronunciation characteristics, such as nationalities and dialects, and the pitch of the input voice can be significantly raised or lowered to reproduce the cries of small animals such as birds and mice, and the voices of wild animals such as lions and bears, and the voices of aliens and creatures can be reproduced by applying effects such as echo, reverb, filtering, and modulation. The voice quality conversion unit 541b can convert to various voices by combining these processes, and can also use AI and machine learning to achieve more realistic voice conversion.

[0083] In step S202b, the language conversion unit 541a converts the language of the voice data based on the feature parameters extracted by the voice feature extraction unit 542a. Specifically, the language conversion unit 541a recognizes phrases and words in the voice data spoken by the user through a voice recognition process, converts non-standard formats such as abbreviations, shorthands, and slang for the corresponding text into a standard format, and translates the text into the target language. In this translation process, the translation is performed by referring to dictionary data or using a neural network-based model, and, if necessary, converts the text into a non-standard format or style, such as a colloquial style or a dialect with a specific accent.

[0084] In this way, by converting the voice quality and language, it is possible to generate a variety of voice data. For example, as shown in Fig. 6, the voice data of a woman in her twenties can be converted into a conversation voice in Italian, which she is not supposed to be able to speak, into a teenage idol singing in English, into the voice of a character or creature such as a cat or an alien speaking the language of various countries, or into a conversation voice in the future by increasing the age.

[0085] The text data with converted voice quality and language is then regenerated as voice data by the voice synthesis engine of the voice data generation unit 541 (S202c). At the same time, the conversion parameter recording unit 542b records, via the related data acquisition unit 542, various conversion parameters and processing data used when the voice quality or language was converted by the voice data generation unit 541, including them in the related data (S203). At this time, the conversion parameter recording unit 542b also records, as conversion parameters, characteristic parameters of the voice data after processing such as generation, conversion, synthesis, etc. by the voice data generation unit 541 (S202d).

[0086] Next, once all of this information has been collected, voice rights data is generated (S204). This generated data is packaged by combining the voice and related data, and is treated as the user's one and only voice rights. The generated voice rights data is then uploaded to the voice rights management server 3 (S205), and is received by the voice rights management server 3 (S102). Specifically, once a series of processes on the smartphone 5 side is completed and voice rights data D3 is generated, uploading of this voice rights data D3 from the smartphone 5 to the voice rights management server 3 via the communication network 2 begins (S205).

[0087] Upon receiving this upload, the voice rights management server 3 performs a process to receive the transmitted voice rights data (S102). Once the voice rights data has been correctly received by the voice rights management server 3, the validity of the data is then verified (S103). In verifying the validity, the authenticity of the newly received data is confirmed by checking whether the received voice rights data is reliable and whether there are any elements that may be suspected of tampering or fraud, and by comparing it with existing information stored on the voice rights management server 3 and data uploaded by other users.

[0088] After the validity of the voice rights data is successfully verified, the user performs a right setting operation (S107), and the voice rights management server 3 executes a right setting process (S108). Specifically, the voice rights data is registered in the voice rights database 35a and stored in the block chain through the guarantee system 4 (S105), and the setting of the voice rights is completed (S106). In setting the rights here, the user's intention and setting contents are set based on the user's operation, and rules and restrictions regarding the management and use of data are established. That is, the set voice rights data is registered in the voice rights database 35a. In addition, in this embodiment, in addition to registering in the database, in order to increase the reliability and immutability of the data, it is also stored in a system using block chain technology via the guarantee system 4. The block chain makes it extremely difficult to tamper with the data, thereby ensuring the safety and reliability of the voice rights data. In this way, the setting of the right regarding the user's voice, that is, the voice rights, is completed (S106).

[0089] After the setting of this portrait right is completed, a right creation notification is sent to the smartphone 5 to notify that the right has been created. On receiving this notification (S208), the smartphone 5 executes a right creation completion process and updates the application data on the user terminal.

[0090] (Operation of the warranty system) Here, the mechanism of the distributed ledger system adopted in the above-mentioned guarantee system will be described in detail. In this embodiment, the guarantee system provides a blockchain interface service, which includes a plurality of nodes that store at least a part or all of the data generated by the voice rights management server 3, and these nodes aggregate and block the stored data at a predetermined timing, form a blockchain using the blocks, and share the blockchain among the plurality of nodes and store it as a distributed ledger.

[0091] Specifically, as shown in Fig. 9, when issuing, transferring, or canceling an NFT related to a national anthem, the national anthem management system 1 according to this embodiment issues a key pair of a public key PKa and a private key SKa based on a public key cryptosystem through a guarantee system 4, and generates a public address PAa from the public key PKa corresponding to the issued NFT. This public address PAa is used as an address indicating the transferee (purchaser) and transferor (rights holder) in a national anthem trading contract, while the private key SKa is used for the electronic signature of a transaction in which the public address PAa is the transfer source.

[0092] The copyright management according to this embodiment is carried out between two nodes (here, between a right holder and a purchaser) on a P2P (Peer-to-Peer) network 90, and the transaction information is broadcast to and shared among the nodes 90a to 90f in the P2P network 90. ​​As a result, a transaction history database (so-called blockchain) based on a distributed ledger system is formed on the P2P network 90, and transaction histories of various tokens and copyright management are stored.

[0093] In this embodiment, transactions using this distributed ledger system execute, approve, and manage the voice rights transaction contract when issuing NFTs or rewriting their owners through the voice rights management server 3. The voice rights management server 3, which acts as an intermediary in voice rights transactions, generates an intermediary public address PA3 unique to the voice rights management server 3 (unique to each transaction transaction) as an NFT platform in order to mediate transactions between the rights holder and the purchaser, and relays the transfer of tokens, with the tokens being the subject of the transaction being temporarily deposited in the voice rights management server 3.

[0094] The parties to the transaction (rights holder and purchaser) then use the voice rights management system 1 to transfer the NFT from the current rights holder to an intermediary public address PA3 specific to this transaction set in the voice rights management server 3, thereby initially receiving the NFT, and then transferring it to the new purchaser via the intermediary public address PA3, thereby establishing a sales contract on the NFT platform between the rights holder and the purchaser.

[0095] Here, the NFT transfer process will be described in detail. FIG. 7 is a flow diagram illustrating a processing procedure at the time of transfer of the energy trading system according to this embodiment, and FIG. 8 illustrates the relationship between the public key and the private key according to this embodiment. In this embodiment, the NFT transfer process and the account process related to the NFT transfer utilize the mechanism of the distributed ledger system according to this embodiment. Here, an example will be described in which a right holder sells an NFT to a purchaser who becomes a new right holder through an NFT trading platform. As shown in FIG. 8, this NFT transaction includes step S301 of issuing a public address and a private key, step S302 of registering related service history information, and step S303 of executing a right transfer process.

[0096] First, in step S301, in the guarantee system 4, the public address management unit 44b issues a pair of an intermediary public address PA3 unique to the NFT trading platform and a private key SK3 corresponding to this intermediary public address PA3. Specifically, as illustrated in Fig. 8, the guarantee system 4 uses a random number generator or the like to generate a private key SK3 associated with a public address unique to the token trading platform by public key cryptography. The random number generator may be built into the public address management unit 44b as a program, for example.

[0097] Next, in the assurance system 4, a public key PK3 is generated from the private key SK3 based on an algorithm for electronic signature such as Elliptic Curve Digital Signature Algorithm (ESDSA). The generated public key PK3 and the private key SK3 form a key pair in a public key cryptosystem, and due to the nature of this public key cryptosystem, it is possible to generate the public key PK3 from the private key SK3, but it is impossible to generate the private key SK3 from the public key PK3 in terms of the amount of calculation required. In other words, the private key SK3 cannot be identified from the public key PK3, but the public key PK3 can be identified from the private key SK3. Note that the type of algorithm for electronic signature used is not limited to Elliptic Curve DSA, and may be appropriately selected depending on the embodiment.

[0098] Next, the assurance system 4 generates an intermediary public address PA3 from the public key PK3 by applying a one-way hash function such as SHA-256 or RIPEMD-160 to the public key PK3. For example, the assurance system 4 can generate the intermediary public address PA3 by applying SHA-256 twice to the public key PK3. That is, this intermediary public address PA3 is a hash value of the public key used to sign the above-mentioned transaction, and is used to identify the transfer destination and transfer source of the NFT.

[0099] In the next step S302, transaction history data associated with the NFT, such as the history of services provided to the NFT trading platform by the rights holder who is the transfer source, is recorded in the node by linking it to the intermediary public address PA3 generated in step S301. Specifically, the power of attorney data acquired by the guarantee system 4 is linked to the intermediary public address PA3 and made public. This power of attorney data can be freely viewed by anyone who has obtained the public key PK3 related to the intermediary public address PA3. As a result, anyone can verify whether there is any fraud or tampering in the history of the power generation method and power generation location from which the electricity originates, or the transaction history.

[0100] After that, in step S303, the guarantee system 4 performs a transaction of transfer of rights to the intermediary public address PA3 generated in step S301. Then, when the transfer is completed, the guarantee system 4 ends the processing related to this operation example. Here, for example, an application executed on the smartphone 5 is used for the exchange of NFTs related to this embodiment. Therefore, an application that executes the mechanism of NFT transactions is also installed in the public address management unit 44b of the guarantee system 4, and the intermediary public address managed by the platform is controlled by this application.

[0101] While the portrait right as an NFT belongs to the right holder, the NFT's unique public address PAa is associated with a paired private key SKa on the right holder's smartphone 5, and when the transfer procedure is carried out on the NFT trading platform, the purchaser uses an information terminal such as a smartphone to temporarily transfer the NFT from the source public address PAa (transfer source) to the intermediary public address PA3 (transfer destination) generated by the NFT platform in step S301.

[0102] On the other hand, a buyer who wishes to purchase a new NFT can use their own information processing terminal device such as a smartphone 5 to obtain the public key PK3 linked to the NFT, and the buyer can view the voice data regarding the NFT, transaction progress information, and related external information linked to the intermediary public address PA3 of the NFT trading platform.

[0103] Specifically, an application for NFT transactions is also installed on the smartphone 5 of the purchaser, and this application manages the public address PAb held by the purchaser. The public address PAb is associated with the private key SKb of the purchaser, which allows the purchaser to further transfer the NFT from his / her public address PAb to another person. In other words, each private key SKb allows the purchaser to freely use the NFT stored in the public address PAb and its transaction history. Here, the purchaser uses the application on the smartphone 5 to receive the NFT transferred from the intermediary public address PA3 specific to the NFT platform to the public address PAb.

[0104] As a result, the purchaser, who is the transferee, can receive the NFT at his / her public address PAb, and can view the service history and use the service linked to this intermediary public address PA3. The public address may be issued by the voice rights management server 3, or by software on each transaction user terminal, or by a server of an independent service management institution or financial institution. Here, the detailed mechanism of NFT transactions will be specifically explained. Figures 10 to 12 show examples of part of the token transaction history (blockchain).

[0105] The transaction history of the issuance, transfer, and cancellation of an NFT is defined as a chain of a series of digital signatures. When the owner of this NFT transfers the transaction history to the next owner, the owner adds to the token's transaction history the hash value of the previous transaction and the hash value of the public key of the next owner, digitally signed with his / her private key. Note that one-way hash functions such as SHA-256 and RIPEMD-160 are used to calculate these hash values.

[0106] In Figure 10, as a specific example of a transaction, an NFT is transferred from owner Z to owner A, then from owner A to owner B, and then from owner B to owner C. In this case, when transferring a token from owner A to owner B, owner A adds to the token a hash value of the transfer transaction from owner Z to owner A and a hash value of the public key of the next owner, owner B, digitally signed with owner A's private key.

[0107] Subsequent token owners, including owner B, can determine whether the transaction has been tampered with by comparing the value obtained by decrypting this digital signature with owner A's public key with the hash value of the transfer transaction from owner Z to owner A and the hash value of owner B's public key. Similarly, when transferring tokens from owner B to owner C, owner B adds to the token the hash value of the transfer transaction from owner A to owner B and the hash value of the public key of owner C, the next owner, digitally signed with owner B's private key. This makes it possible to determine whether the transfer transaction from owner B to owner C has been tampered with.

[0108] An NFT can be defined as a chain of such a series of electronic signatures. Here, the hash value of the public key is a public address. In other words, the tokens stored in this public address can only be transferred by those who can perform an electronic signature of the token with this public address as the transfer source, that is, by those who have a private key corresponding to this public address. Therefore, the private key is generally kept secret so as not to be leaked to anyone other than the owner. Note that the token and data related thereto, such as transaction history, are stored in a public address linked to the current owner. In addition, since it is not possible to verify that any of the past owners of the token have used the token multiple times (transferred multiple times) using only this electronic signature, the token trading mechanism according to this embodiment uses a mechanism called a blockchain exemplified in FIG. 11 and FIG. 12 to prevent this multiple use.

[0109] 11 and 12, each block recorded in a token or the like stores multiple transactions, a nonce, and the hash value of the immediately preceding block. A nonce is a disposable random value used in cryptographic communication, and the node (miner) 90a to 90f that first discovers this value acts as an approver and adds the block in which the nonce was discovered to the end of the blockchain, thereby updating the blockchain.

[0110] As a result, a consistent transaction history is recorded in the blockchain, and by sharing this blockchain among all of the nodes 90a to 90f participating in the P2P network 90, a consistent transaction history can be shared throughout the P2P network 90. ​​In other words, this blockchain serves as part or all of the NFT transaction history database 41a and the key information database 41b in the above-mentioned guarantee system 4. In this embodiment, NFT transactions are carried out using this mechanism in image rights management based on public key cryptography.

[0111] (Action and effect) As described above, according to this embodiment, a person's voice and the personal information of the owner of that voice are linked by a blockchain and managed as an NFT, and the person's voice can be guaranteed as a right, and each individual's voice data can be managed and utilized efficiently and safely. In more detail, according to this embodiment, the voice data can be protected as a "voice right," and in the form of this voice right, clear ownership can be set for the individual's voice data, making it possible to prevent unauthorized use and infringement of rights.

[0112] In this embodiment, the original voice data is converted as necessary in terms of voice quality, language, etc. to regenerate new voice data that is different from the original, and the imitation rights based on this new voice data can be utilized in various services to increase the added value of the services. For example, by using the reproduced voice data, the imitation rights can be utilized in various fields such as education, entertainment, games, welfare, and travel, for example, a virtual travel experience in which a user simulates a travel experience in a foreign country by speaking a foreign language with their own voice, promotion of a song in a foreign language version, a dialogue simulation in a different space in which a user enjoys a dialogue in a virtual space as an animal character or an alien, a dialogue with a past or future self is simulated by making the user's voice younger or older, providing an opportunity to think about life choices and goals, and language learning assistance.

[0113] In addition, in this embodiment, voice data and the personal information of the owner are linked via blockchain technology, so privacy is strongly protected, and the voice right is established, making it easy to take legal action against unauthorized use of voice data. Furthermore, since the voice data can be properly protected as a voice right according to this embodiment, a new business model for safe commercialization becomes possible, clear ownership of the voice data can be set, and the owner of the voice data can be appropriately remunerated for its use, so that the voice data can be applied more freely and safely to creative uses, and since the voice data is authenticated and its reliability is ensured, it is expected that the voice data will be utilized in various industries.

[0114] The present invention is not limited to the above-described embodiments, and the components can be modified without departing from the spirit of the invention. Various inventions can be created by appropriately combining the components disclosed in the above-described embodiments. For example, some components may be deleted from all the components shown in the embodiments. [Explanation of symbols]

[0115] D1…Voice data D2…Personal information D3…Imperial rights data PAa…Public address PAb…Public address PKa...public key SKa...private key SKb…Private key U1...User 1. Voice Rights Management System 2. Communication network 3. Voice rights management server 4. Warranty system 5. Smartphone 21…satellite 22...Radio base station 31...Communication interface 32…Location information management department 33…Authentication Department 34…Information Distribution Department 35a…Imperial rights database 35b...User database 35c…Accumulation information database 36…Privacy Data Control Unit 36a...Voice data control section 36b... Related data control section 36c…Assurance System Integration Department 41a…NFT transaction history database 41b…Key information database 41c…Account database 42…Authentication department 43...Communication interface 44…Rights Management Department 44a…Rights Information Section 44b…Public Address Management Department 44c…Validity Verification Department 44d…Data update section 51...Communication interface 52...Input interface 52a…Touch panel 53...Output interface 53a...Display section 54…Application execution unit 55…Memory 90…P2P network 90a~90f…node 540...Synchronization processing unit 541…Voice data generation unit 541a…Language conversion section 541b…Voice conversion section 542…Related data acquisition section 542a…Voice feature extraction unit 542b…Conversion parameter recording section 542c... Processing information control unit 543…Imperial Rights Management Department 544...Location information acquisition unit 544a...Time information acquisition unit 545...Display control unit 545a...GUI control unit 546...Display data generation unit

Claims

1. A system for managing voice data as a voice right, comprising: a voice data generating unit that acquires an acoustic signal and generates voice data; a related data acquisition unit for acquiring related data including personal information; a voice data generating unit for generating voice data by associating the voice data with the related data; a data storage means for encrypting and storing at least a part of the portrait right data; A voice rights management system comprising:

2. The data storage means includes a guarantee system linking unit that links with a guarantee system constructed on a communication network to encrypt and store at least a part of the image right data; The security system includes a plurality of nodes that encrypt and store at least a portion of the image recognition data; The node aggregates at least a portion of the voting rights data at a predetermined timing, forms a block, links the block to an existing block, forms a blockchain, and shares the blockchain among a plurality of the nodes and stores it as a distributed ledger. The voice right management system according to claim 1 .

3. the voice data generating unit has a function of adding time information including a date and time of acquisition of the acquired acoustic signal to the associated data; The related data acquisition unit has a function of analyzing a correlation between a date and time when the added acoustic signal was acquired and information related to age included in the personal information, and including a result of the analysis in the related data. The voice right management system according to claim 1 .

4. a voice feature extraction unit for extracting any one of frequency, duration, amplitude, intensity, and frequency components contained in the voice data as a feature parameter; a conversion unit that converts the voice quality or language of the voice data based on the feature parameters; a conversion parameter recording unit that records, as conversion parameters, characteristic parameters of the voice data after the conversion unit converts the voice quality or language; The system of claim 1 , further comprising:

5. A method for managing voice data as a voice right using the voice right management system according to claim 1, comprising: a voice data generating step in which a voice data generating unit obtains an acoustic signal to generate voice data, a related data obtaining unit obtains related data including personal information, and a voice data generating step in which the voice data and the related data are related to each other to generate voice data; a data storage step in which a data storage means encrypts and stores at least a part of the image right data; 13. A method for managing voice rights, comprising:

6. the data storage step includes a guarantee system linking step of encrypting and storing at least a part of the image right data in cooperation with a guarantee system constructed on a communication network; The security system includes a plurality of nodes that encrypt and store at least a portion of the image recognition data; In the data storage step, the node aggregates at least a portion of the voting rights data at a predetermined timing to form a block, links the block to an existing block to form a blockchain, and shares the blockchain among a plurality of the nodes to store it as a distributed ledger.

6. The method for managing voice rights according to claim 5.

7. In the voice data generating step the voice data generating unit adds time information including a date and time of acquisition of the acquired acoustic signal to the associated data; The related data acquisition unit analyzes a correlation between a date and time when the added acoustic signal was acquired and information related to age included in the personal information, and includes a result of the analysis in the related data.

6. The method for managing voice rights according to claim 5.

8. In the image data generating step, a voice feature extraction step in which a voice feature extraction unit extracts any one of frequency, duration, amplitude, intensity, or frequency components contained in the voice data as a feature parameter; a conversion step of converting a voice quality or a language of the voice data based on the feature parameters by a conversion unit; a conversion parameter recording unit that records, as conversion parameters, characteristic parameters of the voice data after the conversion unit converts the voice quality or language; The method of claim 5, further comprising:

9. A program for managing voice data as a voice record, comprising: a voice data generating unit that acquires an acoustic signal and generates voice data; a related data acquisition unit for acquiring related data including personal information; a voice data generating unit for generating voice data by associating the voice data with the related data; A data storage means for encrypting and storing at least a part of the image rights data. The voice rights management program is characterized in that it functions as a voice rights management program.

10. The data storage means includes a guarantee system linking unit that links with a guarantee system constructed on a communication network to encrypt and store at least a part of the image right data; The security system includes a plurality of nodes that encrypt and store at least a portion of the image recognition data; The node aggregates at least a portion of the voting rights data at a predetermined timing, forms a block, links the block to an existing block, forms a blockchain, and shares the blockchain among a plurality of the nodes and stores it as a distributed ledger.

10. The copyright management program according to claim 9.

11. the voice data generating unit has a function of adding time information including a date and time of acquisition of the acquired acoustic signal to the associated data; The related data acquisition unit has a function of analyzing a correlation between a date and time when the added acoustic signal was acquired and information related to age included in the personal information, and including a result of the analysis in the related data.

10. The copyright management program according to claim 9.

12. The information processing device, a voice feature extraction unit for extracting any one of frequency, duration, amplitude, intensity, and frequency components contained in the voice data as a feature parameter; a conversion unit that converts the voice quality or language of the voice data based on the feature parameters; a conversion parameter recording section for recording, as conversion parameters, characteristic parameters of the voice data after the conversion section converts the voice quality or language; 10. The computer-readable medium according to claim 9, further functioning as:

Images