System and method for providing contract support service

Abstract

This contract support service provision server communicates with a plurality of contractor terminals, an interplanetary file system (IPFS) and a blockchain network and manages member information about the plurality of contractor terminals. The contract support service provision server acquires audio / video content including an audio / video file, in which contract-related audio / video data is stored, and audio / video metadata in response to a contract support request from one contractor terminal from among the plurality of contractor terminals, generates contract statement information corresponding to the contract-related audio / video data, generates provisional contract content including the generated contract statement information and the audio / video content, generates first link information providing a connection that enables access to the generated provisional contract content and second link information connecting a web page such that same can receive electronic signatures of the plurality of contractor terminals, and transmits the first link information and the second link information to the plurality of contractor terminals included in the contract support request. In addition, when a plurality of electronic signatures are input for each contractor from the plurality of contractor terminals that have checked the provisional contract content, the contract support service provision server generates an audio / video watermark for contract content including the provisional contract content and the plurality of electronic signatures, generates storage contract content by adding the generated audio / video watermark to the provisional contract content, encrypts the storage contract content by using an encryption key, and decentralizes and stores the encrypted storage contract content by using the IPFS and the blockchain network.

Description

System and method for providing contract support services

[0001] The present invention relates to a system and method for providing a contract support service, and more particularly, to a system and method for providing a voice and video contract support service utilizing distributed file storage and blockchain technology, voice and video encryption, electronic signatures, and digital signatures.

[0002] When it comes to contracts, written contracts are common, and there was a social custom that these had to be made using paper and pencil, a computer, or in the presence of an expert. In the case of impromptu contracts on the spot, it was common to record them on a cell phone or make them verbally in the presence of a third party witness.

[0003] In everyday life, oral contracts are more frequently entered into than written ones. However, due to the difficulty of proving oral contracts, written contracts have been more commonly used officially. Even companies currently operating electronic contracts using electronic signatures exclusively handle written contracts. However, written contracts present challenges such as the time, cost, and procedural inconvenience of documenting them. Furthermore, while most contracts have been conducted on the spot, tools like recorders and mobile phones have been used, such as the aforementioned possibility of forgery and falsification, they present vulnerabilities in proving contracts.

[0004] The technical problem to be solved by the present invention is to provide a system and method for providing a voice and video contract support service that can increase the transparency and integrity of voice and video contracts performed on the spot by blocking the possibility of forgery and alteration of contract contents and increasing the security of contract contents by utilizing distributed file storage technology (IPFS), blockchain technology, voice and video encryption, electronic signatures, and digital signatures.

[0005] According to one aspect of the present invention, a contract support service providing server in which a plurality of contract terminals, a distributed file system (IPFS), and a blockchain network are connected through a network comprises: a communication unit communicating with a plurality of contract terminals, a distributed file system (IPFS), and a blockchain network; a service management unit managing member information for a plurality of contract terminals; a contract content preprocessing unit controlling the communication unit to obtain, in response to a contract support request from one of the plurality of contract terminals, a voice / video file storing contract-related voice / video data and a voice / video content including metadata for voice / video, generate contract document information corresponding to the contract-related voice / video data, generate provisional contract content including the generated contract document information and the voice / video content, generate first link information connecting to enable access to the generated provisional contract content and second link information connecting to a web page so as to receive electronic signatures of a plurality of contract terminals, and transmit the first link information and the second link information to a plurality of contract terminals included in the contract support request; and a contract content preprocessing unit controlling the communication unit to generate a plurality of electronic signatures for each contracting party from the plurality of contract terminals that have confirmed the provisional contract content. When input, a contract content post-processing unit is included that controls a communication unit to generate a voice / video watermark for contract content including a provisional contract content and multiple electronic signatures, add the generated voice / video watermark to the provisional contract content to generate a storage contract content, encrypt the storage contract content using an encryption key, and store the encrypted storage contract content in a distributed manner using a distributed file system (IPFS) and a blockchain network, and the audio / video metadata includes location information and storage time information of the contract terminal where the audio / video file is stored.

[0006] The contract content preprocessing unit can control the communication unit to transmit a user input screen that inquires whether multiple contracting parties will use one contracting terminal at the same location to record and record audio / video files or whether multiple contracting parties will use Internet telephony to record and record audio / video files, to the contracting terminal that sent the contract support request.

[0007] When receiving a user input from a contract terminal that has transmitted a contract support request that multiple contracting parties perform recording and video recording using a single contract terminal at the same location, the contract content preprocessing unit receives contract-related voice and video data from the single contract terminal and creates a voice / video file, and when receiving a user input from a contract terminal that has transmitted a contract support request that multiple contracting parties perform recording and video recording using an Internet phone, the contract content preprocessing unit requests an Internet phone channel from a VoIP server connected through a network, transmits recording / recording link information that can access the Internet phone channel to the emails of each of the multiple contracting parties, and receives contract-related voice / video data recorded and video recorded through the Internet phone channel and creates a voice / video file.

[0008] The contract content preprocessing unit can create contract document information using STT (Speech to Text) from the voice contained in the voice / video file.

[0009] The service management department may control the communication department to request confirmation of audio / video files included in the contract content via the email addresses of multiple contracting parties included in the member information, request an electronic signature based on the confirmation input for the audio / video files, send an authentication number as a text message to the contracting party's terminal, and receive the transmitted authentication number from the contracting party's terminal using the same screen as the screen where the electronic signature is input, thereby additionally confirming that the contracting party is a contracting party.

[0010] The contract content post-processing unit receives electronic signature metadata including location information and storage time information where the electronic signature was made along with multiple electronic signatures for each contracting party from multiple contracting terminals, and can generate contract content for storage by further including the electronic signature metadata.

[0011] The contract content post-processing unit generates an encryption key by applying the Bcrypt algorithm to the contract party name and email domain name, encrypts the contract content for storage using the generated encryption key and the AES algorithm, controls the communication unit to upload the encrypted contract content for storage to the distributed file system (IPFS), and when the IPFS hash value is received from the distributed file system (IPFS) through the communication unit, controls the communication unit to store the received IPFS hash value in the blockchain network, and obtains a transaction ID (TxID) for obtaining the IPFS hash value from the blockchain network through the communication unit and a data ID (DataID) which is information identifying the IPFS hash value.

[0012] According to another aspect of the present invention, a method for providing a contract support service by a server for providing a contract support service, in which a plurality of contract terminals, a distributed file system (IPFS), and a local server are connected through a network, comprises the steps of: acquiring, in response to a contract support request from one of the plurality of contract terminals, a voice / video file storing contract-related voice and video data and a voice / video content including metadata for the voice / video, generating contract written information corresponding to the contract-related voice and video data, and generating a provisional contract content including the generated written information and the voice and video content, generating first link information that connects to access the generated provisional contract content and second link information that connects to a web page so that electronic signatures of the plurality of contract terminals can be input, and transmitting the first link information and the second link information to the plurality of contract terminals included in the contract support request, and when a plurality of electronic signatures are input for each contracting party from the plurality of contract terminals that received the provisional contract content, generating a voice / video watermark for the contract content including the provisional contract content and the plurality of electronic signatures, and transmitting the generated voice / video watermark to the plurality of contract terminals. It includes a step of creating storage contract content in addition to the contract content, a step of encrypting the storage contract content using an encryption key, and a step of storing the encrypted storage contract content in a distributed manner using a distributed file system (IPFS) and a blockchain network, and the audio / video metadata includes location information and storage time information of the contract terminal where recording and video recording were performed.

[0013] According to the present invention, a system and method for providing a contract support service that can increase the transparency of oral contracts by blocking the possibility of falsification of contract contents and increasing the security of contract contents by utilizing the distributed file storage technology (IPFS) and blockchain can be provided, thereby reducing the cost and time of contracts occurring in various aspects of life and increasing the convenience of contracts.

[0014] In addition, according to the present invention, the contents of the contract, i.e., data uploaded to the blockchain such as voice, video, electronic signature, and personal information, can be stored and verified forever regardless of whether the company is operating, thereby ensuring the security and stability of storage of the contract contents and increasing the reliability of the contract evidence.

[0015] FIG. 1 is a diagram showing the configuration of a contract support service provision system according to one embodiment of the present invention.

[0016] FIG. 2 is a diagram showing the configuration of a contract support service providing server of the contract support service providing system of FIG. 1.

[0017] Figure 3 is a drawing showing an example of a contract content creation process of the contract support service provision system of Figure 1.

[0018] Fig. 4 is a drawing showing another example of the contract content creation process of the contract support service provision system of Fig. 1.

[0019] FIG. 5 is a diagram showing a contract content storage process of the contract support service provision system of FIG. 1 according to one embodiment of the present invention.

[0020] FIG. 6 is a diagram showing a contract content confirmation process of the contract support service provision system of FIG. 1 according to one embodiment of the present invention.

[0021] FIG. 7 is a drawing showing an example of a user interface screen provided to a contract terminal during a contract content creation process of the contract support service provision system of FIG. 1 according to one embodiment of the present invention.

[0022] FIG. 8 is a drawing showing an example of a user interface screen provided to a contract terminal during a contract content storage process of the contract support service provision system of FIG. 1 according to one embodiment of the present invention.

[0023] Figure 9 is a flowchart showing a method for providing a contract support service by the contract support service providing server of Figure 1.

[0024] Hereinafter, an embodiment of the present invention will be described in detail with reference to the attached drawings. In describing the present invention, detailed descriptions of known functions or configurations will be omitted if they are deemed to unnecessarily obscure the gist of the present invention. Furthermore, the terms described below are defined based on their functions in the present invention and may vary depending on the intentions or practices of the user or operator. Therefore, their definitions should be based on the overall content of this specification.

[0025] FIG. 1 is a diagram showing the configuration of a contract support service provision system according to one embodiment of the present invention.

[0026] A contract support service provision system (100) according to one embodiment of the present invention integrates the advantages of written and oral contracts to facilitate and immediately perform the process of contracts frequently made in daily life without preparation, thereby eliminating the time for documentation, the cost of hiring experts, and procedural inconvenience, while maximizing legal probative power and evidentiary power, thereby supporting contracts to change the paradigm of existing contract methods.

[0027] A contract support service provision system (100) according to one embodiment of the present invention includes a plurality of contract terminals (110-1, 110-2, 110-3), a contract support service provision server (120), a local database (130), a distributed file system (IPFS) (140), a blockchain network (150), and a VoIP server (160), and the plurality of contract terminals (110-1, 110-2, 110-3), the contract support service provision server (120), the local database (130), the distributed file system (IPFS) (140), the blockchain network (150), and the VoIP server (160) are connected to enable communication via a network (10). Here, the network (10) may be various wired or wireless communication networks such as the Internet, a mobile communication network WAN (Wireless LAN), a LAN, and Bluetooth.

[0028] The multiple contract terminals (110-1, 110-2, 110-3) are terminals on which applications can be installed, and may include all types of wireless communication devices that ensure portability and mobility, such as 3G, 4G, 5G terminals, smart phones, smart pads, tablet PCs, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), W-CDMA (W-Code Division Multiple Access), and Wibro (Wireless Broadband Internet). Although three contract terminals (110-1, 110-2, 110-3) are illustrated in Fig. 1, the number of contract terminals may be as many as the number of contracting parties participating in the contract. For the convenience of explanation, the following description will focus on the first contract terminal (110-1).

[0029] The app installed on the first contract terminal (110-1) can be downloaded and installed by the administrator from a public app store such as Google Play. Furthermore, the first contract terminal (110-1) can also directly access the contract support service server (120) to download and install the app.

[0030] The first contract terminal (110-1) can access the contract support service providing server (120) and execute various applications such as an application for contract support that can be downloaded, a decryption program for verifying contract contents, etc. The first contract terminal (110-1) includes a microphone (not shown) for receiving voice input and a camera (not shown) for receiving video input. In the contract support service process provided by the contract support service providing server (120), when the microphone (not shown) is activated for recording, or the microphone (not shown) and camera (not shown) are activated for video recording, contract voice / video data according to the user's speech and contract actions are collected through the microphone (not shown) and camera (not shown) and transmitted to the contract support service providing server (120). The first contract terminal (110-1) can include a touch panel or a touch screen and can receive information according to user input. For example, the first contract terminal (110-1) can receive input of contract signature, contract party name, email address, contact information, etc. In this specification, ' / ' is used to indicate terms concisely, and means that the word before and after ' / ' are connected in parallel with 'and'.

[0031] In addition, the first contractor terminal (110-1) can measure real-time location information (e.g., GPS information) and international standard time information of the contractor terminal, and can be configured to perform a metadata storage function that can add real-time location record information and international standard time information to digital data generated from the first contractor terminal (110-1). The metadata may include audio / video metadata including location information and storage time information at the time of recording and filming of the contractor terminal related to audio and video, and electronic signature metadata including location information and storage time information at the time of electronic signature of the contracting party. The time information of the audio / video data may be called an audio / video timestamp, and the time information at which an electronic signature was made may be called an electronic signature timestamp. Here, the timestamp may be timestamp information that complies with RFC 3161.

[0032] The contract support service provision server (120) uses a distributed file system (IPFS) (140) and a blockchain network (150) to create a verbal contract, thereby firstly eliminating the possibility of forgery or alteration of the verbal contract and improving the legal evidentiary capacity, and secondly, by using a voice watermark technique, it inserts a voice watermark, which is a high-frequency signal in an inaudible frequency range unique to the contractor, into an arbitrary time interval for each contract, thereby enabling detection of forgery or alteration through recent AI-based deepfake and deep voice.

[0033] The local database (130) is a data storage that can be accessed by the contract support service providing server (120) and may be located within the contract support service providing server (120).

[0034] The InterPlanetary File System (IPFS) (140) is a file system for storing data in a distributed manner across a network and sharing it over the Internet. In the IPFS (140), each file is composed of multiple blocks, and each block has a unique name expressed as a hash. Each node stores only the files it is interested in, and indexing information allows it to determine who is storing which files. To find a network file, simply look up the file name and track the node that holds it.

[0035] A user who wishes to store files in a distributed manner using the distributed file system (IPFS) (140) uploads the files through a processing process provided by the distributed file system (IPFS) (140). According to the present invention, the contract support service provision server (120) uploads storage contract content to the distributed file system (IPFS) (140) and receives an IPFS hash value corresponding to the identification information (content ID) of the storage contract content from the distributed file system (IPFS) (140).

[0036] The blockchain network (150) can store and manage the IPFS hash value obtained from the processing process of the Distributed File System (IPFS) (140) on the blockchain. Blockchain refers to a data distribution processing technology that decentralizes and stores all data subject to management by all users participating in the network. This blockchain is a technology to prevent hacking such as falsification of transaction details. It uses a method to prevent data falsification by sending transaction details to all users participating in the transaction and comparing them with each transaction. The core concept of blockchain is decentralization, which is free from central control and management. This environment enables the realization of smart contracts, which can be conveniently concluded and modified without an intermediary.

[0037] In blockchain technology, a transaction typically refers to an act of updating data on a blockchain. In this system, a new IPFS hash value is recorded in the blockchain's smart contract during this process, while generating a transaction ID (TxID) and a data ID (DataID). At this time, the IPFS hash value becomes a completely different value even if the data changes slightly, and this can be used to look up and identify the blockchain network (150) record for the transaction. The transaction ID (TxID) and data ID (DataID) identify the IPFS hash value recorded on the blockchain network (150) and refer to information stored in the local DB of the blockchain network (150) and used in the self-management system.

[0038] According to another embodiment of the present invention, the contract support service server (120) adds electronic signature functionality, one of the elements of a written contract, enabling the final result to function as a complete contract, including voice and video contracts (documentation via STT), electronic signatures, and electronic signatures. In the case of electronic signatures, the recognition fee is automatically recommended based on the contract type selected during the oral contract, thereby enhancing user convenience. In this way, the elements of a written contract, such as an electronic signature and a contractor's confirmation process, can be expected to have the same effect as a written contract.

[0039] This method primarily blocks the possibility of falsification of contractual content by utilizing the Interoperable File System (IPFS) (140) and blockchain network (150), and the contents of the contract, i.e. data uploaded to the blockchain such as audio, video, signature, and personal information, can be permanently stored and verified regardless of the operation of the company providing the contract support service. Next, to alleviate the anxiety of contract holders who doubt the establishment and evidentiary ability of a contract based on audio and video contracts without a written contract, elements of a written contract can be added to maximize the convenience and formality of a written contract. Furthermore, considering the importance of the contract process, in order to maximize security, security measures can be added to detect and block manipulation and falsification, such as deepfakes for videos and deepvoice for audio, resulting from recent advancements in AI technology.

[0040] FIG. 2 is a drawing showing the configuration of a contract support service providing server (120) of the contract support service providing system (100) of FIG. 1.

[0041] The contract support service provision server (120) may include a service management unit (210), a contract content preprocessing unit (220), a contract content postprocessing unit (230), and a communication unit (240). Here, for convenience of explanation, the description will focus on the case where the contracting parties are users of the first contracting terminal (110-1) and the second contracting terminal (110-2), and the description will be made for the case where a contract support request is received from the first contracting terminal (110-1).

[0042] The service management unit (210) manages member information for multiple contract terminals. Member information may include the contract signature, contract party name, email address, or contact information of the contracting parties using the service. The service management unit (210) may be configured to control all components within the contract support service provision server (120).

[0043] The contract content preprocessing unit (220) may, in response to a contract support request from one of a plurality of contract terminals (110-1, 110-2), acquire a voice / video file storing contract-related voice / video data and a voice / video content including metadata for voice / video, generate contract document information corresponding to the contract-related voice / video data, and generate a preliminary contract content including the generated contract document information and voice / video content. The voice / video metadata may include location information and storage time information at the time of recording and recording of each of the plurality of contract terminals (110-1, 110-2) where the voice / video file is stored.

[0044] When multiple electronic signatures are input for each contracting party from multiple contracting terminals (110-1, 110-2) that have received the provisional contract content, the contract content post-processing unit (230) generates an audio / video watermark for the contract content including the provisional contract content and multiple electronic signatures, adds the generated audio / video watermark to the provisional contract content to generate contract content for storage, and encrypts the contract content for storage using an encryption key. To this end, the contract content post-processing unit (230) may include a watermarking processing unit (232) that generates an audio / video watermark. The watermarking processing unit (232) may utilize some of the voice data of the contracting parties to add a high-frequency signal of a unique inaudible frequency range to a section selected for the contracting parties, and may add an inconspicuous watermark image to a specific part of the electronic signature of the contracting parties. In addition, the watermarking processing unit (232) performs a personal terminal verification procedure, such as entering a number transmitted as a text message to a personal mobile terminal during the final process of signing, thereby fundamentally blocking the creation of fake contracts based on deep fake and deep voice.

[0045] The contract content post-processing unit (230) can control the communication unit (240) to store encrypted storage contract content in a distributed manner using the distributed file system (IPFS) (140) and the blockchain network (150). Specifically, under the control of the contract content post-processing unit (230), the communication unit (240) can upload the encrypted storage contract content to the distributed file system (IPFS) (140), receive the IPFS hash value from the distributed file system (IPFS) (140), store the IPFS hash value in the blockchain network (150), and obtain a transaction ID (TxID) and a data ID (DataID) for managing the IPFS hash value from the blockchain network (150).

[0046] Meanwhile, the contract content preprocessing unit (220) can control the communication unit (240) to transmit a user input screen that inquires whether multiple contracting parties (110-1, 110-2) will record and record using one contracting terminal at the same location to obtain voice / video files, or whether multiple contracting parties will record and record using Internet telephony, to the first contracting party terminal (110-1) that sent the contract support request.

[0047] When receiving a user input from the first contracting party terminal (110-1) that transmitted the contract support request that multiple contracting parties use one contracting party terminal at the same location to perform recording and video recording, the contract content preprocessing unit (220) may receive contract-related voice and video data from the first contracting party terminal (110-1) and generate a voice / video file. In addition, when receiving a user input from the first contracting party terminal (110-1) that transmitted the contract support request that multiple contracting parties use Internet telephony to perform recording and video recording, the contract content preprocessing unit (220) may request an Internet telephony channel from a VoIP server (160) connected through a network, transmit recording / recording link information that can access the Internet telephony channel to the emails of each of the multiple contracting parties, and receive voice / video data of the recorded and video recorded contract through the Internet telephony channel to generate a voice / video file.

[0048] The contract content preprocessing unit (220) can generate contract document information using STT (Speech to Text) from audio contained in audio / video files. To this end, the contract content preprocessing unit (220) may be configured to include an STT execution unit (222) that executes an STT algorithm, and the STT execution unit (222) may be configured to enhance performance through a separate artificial intelligence service.

[0049] The service management unit (210) can control the communication unit (240) to request confirmation of the audio / video files included in the tentative contract contents via the email addresses of multiple contracting parties included in the member information, and to transmit an electronic signature request file requesting an electronic signature based on the confirmation input for the audio / video files. In the final step, the signature input step, an authentication number input field via a text message sent to a personal mobile phone number can be added to authenticate the device of the contracting party terminal (110-1, 110-2). To this end, the service management unit (210) can additionally confirm that the contracting party is a contracting party by sending the authentication number via a text message to the contracting party terminal and having the contracting party terminal (110-1, 110-2) input the transmitted authentication number using the same screen as the screen where the electronic signature is input.

[0050] The contract content post-processing unit (230) can generate an encryption key by applying the Bcrypt algorithm to the contracting party's name and email domain name. The information used in the encryption key can be selectively modified, and for example, the first letter of the contracting party's name and the email address domain can be combined to generate the encryption key. Here, the Bcrypt algorithm is an algorithm designed for password hashing. The Advanced Encryption Standard (AES) is a Rijndael block cipher algorithm selected in a contest hosted by the National Institute of Standards and Technology (NIST) in the United States. The encryption block size is 128 bits, and encryption key lengths of 128, 192, and 256 bits are supported. It was standardized as FIPS 197, AES in 2001.

[0051] The contract content post-processing unit (230) can encrypt the storage contract content using the generated encryption key and AES algorithm. The generated encryption key can be configured so that the contents cannot be viewed by others even if viewed, and can only be used to confirm whether the storage contract content is original. The generated encryption key and the TxID and DataID generated by the communication unit (240) above are transmitted to the contract parties, including the first contract party terminal (110-1), as a contract confirmation email sent upon completion of the contract support service. In order to request the storage contract content from a judicial agency or to personally view it, the contract party's name and email domain name must be entered to access the encryption key, TxID, and DataID, and decrypt the contract confirmation content by combining them with the IPFS hash key at the first contract party terminal (110-1).

[0052] In addition, the contract content post-processing unit (230) can safely store electronic signatures using the IPFS (Internet Protocol File System) and blockchain, and can generate contract content for storage by further including electronic signature metadata that indicates the location and time information of the electronic signature image being signed. By inserting electronic signature metadata in this way, it is possible to detect forged signature images that do not have electronic signature metadata or are different from the original. Additionally, a difficult-to-detect watermark unique to the contractor can be inserted into the signature image, for example, by weakly adding it to the edge of the image, so that an original contract can be distinguished from a forged signature.

[0053] Hereinafter, for convenience of explanation, it is assumed that the contract terminal requesting the contract support service in the contract support service provision system (10) of FIG. 1 is the first contract terminal (110-1), and that the user of the first contract terminal (110-1) and the user of the second contract terminal (110-2) use the contract support service to enter into a contract.

[0054] As described above, the contract support service provision system (100) can provide a user input screen to the first contracting party terminal (110-1) that transmitted the contract support request, which queries whether multiple contracting parties will use one contracting party terminal at the same location to record and record in order to obtain voice and video files, or whether multiple contracting parties will use Internet telephony to record and record. Hereinafter, FIG. 3 illustrates a process in which multiple contracting parties use one contracting party terminal at the same location to record and record, and FIG. 4 illustrates a process in which multiple contracting parties use Internet telephony to record and record.

[0055] FIG. 3 is a drawing showing an example of a contract content creation process of the contract support service provision system (10) of FIG. 1.

[0056] A user of the first contract terminal (110-1) can access the contract support service provision server (120) and register as a member to use the contract support service (310). During this process, the contract support service provision server (120) may collect the user name, email address, etc. of the first contract terminal (110-1).

[0057] When the first contract terminal (110-1) transmits a message indicating a contract support service request to the contract support service server (120) (312), the contract support service providing server (120) can provide the first contract terminal (110-1) with a user screen that provides the type of contract and the option to purchase an electronic certificate. The contract support service providing server (120) can also provide a service that automatically recommends an electronic certificate according to the type, and the electronic certificate can be configured to be confirmed in the final contract result (contract).

[0058] The contract support service provision server (120) can collect the contract party name and email information and transmit a user screen for performing the contract recording / recording service to the first contract terminal (110-1) to provide a recording and recording service (314).

[0059] Recording of the contract content can be performed primarily through the first contract terminal (110-1) (340). Additionally, recording of the contract content can be performed further (340), and audio / video content generated from the first contract terminal (110-1) is transmitted to the contract support service provision server (120) (342).

[0060] The contract support service providing server (120) processes voice and video content (350). Specifically, the contract support service providing server (120) responds to a contract support request from one of a plurality of contract terminals, obtains voice / video content including voice / video files storing contract-related voice / video data and metadata for voice / video, generates contract document information corresponding to the contract-related voice / video data, and generates provisional contract content including the generated contract document information and voice / video content (350). The contract support service providing server (120) generates first link information that connects to access the generated provisional contract content and second link information that connects to a web page so that electronic signatures of a plurality of contract terminals can be input, and can transmit the first link information and the second link information to a plurality of contract terminals included in the contract support request (352, 354).

[0061] The first electronic signature of the contracting party on the first contracting terminal (110-1) is entered (360), and the first electronic signature is transmitted to the contract support service providing server (120) (362). When the second electronic signature of the contracting party on the second contracting terminal (110-2) is entered (364), the second electronic signature is transmitted to the contract support service providing server (120) (366). In this way, multiple electronic signatures are entered for each contracting party from multiple contracting terminals that have received and confirmed the provisional contract content. At this time, the contract support service provision server (120) may transmit an authentication number to the first contract terminal (110-1) and the second contract terminal (110-2) via text message, add an authentication number input area to the bottom of the electronic signature input screen of the first contract terminal (110-1) and the second contract terminal (110-2), and enable personal authentication by entering the received authentication number into the authentication number input area.

[0062] The contract support service provider server (120) creates a storage contract content using the contract content and the electronic signatures of multiple contracting parties (370). In this way, after recording the contract in a call, the electronic signatures of each contracting party are entered, the storage contract content is created, and finally, the storage contract content is encrypted and stored in the IPFS (Internet Protocol File System) (140) to obtain an IPFS hash value. This hash value is then stored in the blockchain network (150), and the TxID and DataID are stored in the internal database, confirming the contract completion.

[0063] Accordingly, according to the present invention, the contracting parties can complete the contract process by making an electronic signature on their own terminal and then sending it to proceed to the next step, and since there are procedures such as making an electronic signature through their own terminal and entering an authentication number that can be received as a text message, there is an effect of being able to identify the person in the contract, and there is an effect of blocking the possibility of recent deep voices and deep fakes.

[0064] Fig. 4 is a drawing showing another example of the contract content creation process of the contract support service provision system of Fig. 1.

[0065] A user of the first contract terminal (110-1) can access the contract support service provision server (120) and register as a member to use the contract support service (410). During this process, the contract support service provision server (120) may collect the user name, email address, etc. of the first contract terminal (110-1).

[0066] When the first contract terminal (110-1) transmits a message requesting a contract support service, if an option is selected in which multiple contract terminals use the contract support service in different locations, the contract support service providing server (120) requests an Internet phone call (voice and video) to the VoIP server (160) connected via the network (10) (420). The VoIP server (160) transmits recording / recording link information that enables recording and videotaping by accessing the Internet phone channel to the email address of the contracting party of the first contract terminal (110-1) (432), transmits recording / recording link information that enables recording and videotaping by accessing the Internet phone channel to the email address of the contracting party of the second contract terminal (434), and provides an Internet phone channel between the first contract terminal (110-1) and the second contract terminal (110-2), and can proceed with recording the contract content (440). At step 440, recording is performed by default and additional recording and recording can be set to proceed.

[0067] The contract support service providing server (120) processes recorded and video content (450). Specifically, the contract support service providing server (120) responds to a contract support request from one of a plurality of contract terminals, obtains audio / video files storing contract-related audio and video data and audio / video content including audio / video metadata, generates contract document information corresponding to the contract-related audio / video data, and generates provisional contract content including the generated contract document information and audio / video content (450). The contract support service providing server (120) generates first link information that connects to access the generated provisional contract content and second link information that connects to a web page so that electronic signatures of a plurality of contract terminals can be input, and transmits the first link information and the second link information to a plurality of contract terminals (110-1, 110-2) included in the contract support request (452, 454).

[0068] The first electronic signature of the contracting party on the first contracting terminal (110-1) is entered (460), and the first electronic signature is transmitted to the contract support service providing server (120) (462). When the second electronic signature of the contracting party on the second contracting terminal (110-2) is entered (464), the second electronic signature is transmitted to the contract support service providing server (120) (466). The electronic signature input screen is configured to proceed to the next screen only when authentication information via text message is entered, thereby performing authentication on the device of the contracting terminal, so that the contract cannot be concluded if it does not match the mobile phone of the contracting terminal.

[0069] The contract support service providing server (120) generates contract content for storage using the provisional contract content and the electronic signatures of multiple contracting parties (470). At this time, the contract support service providing server (120) can generate an audio / video watermark for the contract content including the provisional contract content and the multiple electronic signatures, and add the generated audio / video watermark to the provisional contract content to generate the contract content for storage.

[0070] In this way, after recording the contract in a call, the electronic signatures of each contracting party are entered, and the contract content is created, the contract can be confirmed as completed.

[0071] FIG. 5 is a diagram showing a contract content storage process of the contract support service provision system of FIG. 1 according to one embodiment of the present invention.

[0072] The contract support service providing server (120) performs processing to store the storage contract content in the IPFS (Internet Protocol File System) (140) (510). To this end, the contract support service providing server (120) encrypts the storage contract content using an encryption key and transmits the encrypted storage contract content to the IPFS (Internet Protocol File System) (140) (512). The storage contract content includes contract-related audio and video data, electronic signatures, and minimal personal information of the contracting parties, and may be compressed in a Zip format.

[0073] Then, the contract support service provision server (120) encrypts the compressed storage contract content using an encryption key (generated with the participant name and email domain name) through the Bcrypt algorithm, and stores the encrypted file in the distributed file system (IPFS) (140). Then, the management process of the distributed file system (IPFS) (140) uploads the storage contract content and generates an IPFS hash value when storage is completed (520). In this way, since the encrypted contract content is stored in the distributed file system (IPFS) (140) and the IPFS hash value for accessing IPFS is stored in the system contract of the blockchain network (150), it is possible to safely store digital documents (522, 524).

[0074] The contract support service provision server (120) records the IPFS hash value in a smart contract created in advance in the blockchain network (150) and receives a data ID (DataID), which is a key for searching the IPFS hash value, and a transaction ID (TxID) (530, 532).

[0075] The contract support service provision server (120) generates a contract confirmation content including a Data ID (DataID) and a Transaction ID (TxID), which are keys for looking up IPFS hash values, an encryption key for decrypting encrypted contract content, and contract document information (540), and transmits a contract confirmation email including the contract confirmation content to the contracting party terminals (110-1, 110-2) (542, 544). The contract confirmation email can confirm contract audio and video data, and may include files that enable access to electronic signatures, such as an encryption key generation program and a decryption program. In order to confirm one's own contract or a contract with a judicial agency, one must have the email for the relevant contract.

[0076] The contract support service provision server (120) can organize the local database (130) to prevent unnecessary storage and potential leakage of personal information (550). The local database (130) can be organized in such a way that only the transaction ID (TxID), data ID (DataID), and information for encryption keys (e.g., the first letter of the participant's name, email domain) are stored for service management of the contract support service provision server (120), and other information transmitted and received during the contract support service provision process is deleted.

[0077] FIG. 6 is a diagram showing a contract content confirmation process of the contract support service provision system of FIG. 1 according to one embodiment of the present invention.

[0078] When a user input signal for opening an attachment of a contract confirmation email by selecting a contract confirmation email from a user of the first contract terminal (110-1) is received (610), the first contract terminal (110-1) provides a message window on the user screen requesting the user to enter a password (e.g., a phone number of the contract party) to open the contract confirmation email, and when a password matching a pre-stored password is entered and searched from the message window, the first contract terminal can perform an operation for confirming the contract voice and video data and contract document information of the contract party.

[0079] To this end, the first contractor terminal (110-1) transmits the data ID (DataID) and transaction ID (TxID) attached to the contract confirmation email to the blockchain network (150) (612), the blockchain (150) returns the IPFS hash value to the distributed file system (IPFS) (140) (614), and the distributed file system (IPFS) (140) searches for the contract content for storage using the IPFS hash value (620) and transmits it to the first contractor terminal (110-1) (622).

[0080] The first contract terminal (110-1) can decrypt the AES-encrypted storage contract content using the encryption key attached to the contract confirmation email and provide the user with contract-related voice data and contract document information (630).

[0081] Contract-related voice data includes audio / video metadata, including location information and storage time information where the contract-related voice and video were recorded and recorded. Signature images included in contract documents also include electronic signature metadata, including location information and storage time information where the electronic signature was made. Furthermore, voice watermarks, which are inaudible random frequency band sounds that can distinguish customers, are stored together with contract-related voice data only during specific time periods, thereby preventing duplication. Additionally, a method of adding watermarks to the edges of signature images containing text can be utilized to prevent forgery.

[0082] In this way, the resulting audio and video data, electronic signature images, metadata including location and time information, and minimal personal information, which are the output of the contract process, can be compressed and encrypted using the Interoperable File System (IPFS) and blockchain, maximizing security. For oral contracts, voice watermarking techniques can be used to embed the contractor's unique inaudible high-frequency signal into each contract's unique time period, enabling detection of forgery and falsification through recent AI-powered deepfakes and deep voice. For written contracts, video watermarking techniques for signed videos and the embedding of location and time information into audio / video metadata can fundamentally prevent forgery and falsification. Furthermore, additional evidentiary value can be secured through procedures such as allowing the contract to be reviewed again, a waiting process for an electronic signature, the option to add electronic signatures, documentation reinforcement through automatic contract generation using speech-to-text (STT) solutions, and a process for entering an authentication number in a text message sent to the contractor's device.

[0083] FIG. 7 is a drawing showing an example of a user interface screen provided to a contract terminal during a contract content creation process of the contract support service provision system of FIG. 1 according to one embodiment of the present invention.

[0084] Screen (710) is an example of a user input screen in which the contract content preprocessing unit (220) of the first contract terminal (110-1) inquires whether multiple contracting parties will use one contracting terminal at the same location to record and record in order to obtain voice and video files, or whether multiple contracting parties will use Internet telephony to record and record.

[0085] Screen (720) is a screen for receiving information on multiple contracting parties for a contract that is the subject of a contract support service, and can be configured to receive the name and email address of each contracting party.

[0086] Screen (730) may be provided while contract recording and recording are in progress, and screen (740) may be provided while contract-related voice and video data are uploaded to the contract support service provision server (120).

[0087] FIG. 8 is a drawing showing an example of a user interface screen provided to a contract terminal during a contract content storage process of the contract support service provision system of FIG. 1 according to one embodiment of the present invention.

[0088] Screen (810) is a screen for receiving an electronic signature from a contracting party, who is a user of the first contracting party terminal (110-1). Screen (820) may be provided while the contract support service providing server (120) receives and processes contract content, creates contract content for storage, and distributes and stores it. Screen (830) may be provided when a contract is completed by the contract support service of the present invention.

[0089] Figure 9 is a flowchart showing a contract support service providing method of the contract support service providing server (120) of Figure 1.

[0090] The contract support service provision server (120) responds to a contract support request from one of multiple contract terminals and acquires audio and video content, including audio / video files storing contract-related audio and video data and audio / video metadata (910). The audio / video metadata includes location information and storage time information of the contract terminal where recording and video recording were performed.

[0091] The contract support service provision server (120) generates contract document information corresponding to contract-related voice and video data, and generates contract content including the generated contract document information and voice and video content (920).

[0092] In addition, the contract support service providing server (120) can generate first link information that connects to access the generated provisional contract content and second link information that connects to a webpage so that the electronic signatures of multiple contracting parties can be input, and transmit the first link information and the second link information to multiple contracting parties included in the contract support request. Accordingly, the contract support service providing server (120) receives multiple electronic signatures for each contracting party from multiple contracting parties that have received the provisional contract content (930), and then generates a voice watermark and a video watermark for the contract content including the provisional contract content and the multiple electronic signatures (940). The contract support service providing server (120) can generate a voice watermark using the recorded and video voices and videos of the contracting parties, and can generate a video watermark for each electronic signature.

[0093] The contract support service provision server (120) adds the generated voice and video watermarks to the contract content to create contract content for storage (950).

[0094] The contract support service provision server (120) encrypts the storage contract content using an encryption key (960) and stores the encrypted storage contract content in a distributed manner using a distributed file system (IPFS) (150) and a blockchain network (160) (970).

[0095] According to the present invention, in order to simplify the process of contracts frequently made in daily life, the advantages of written and oral contracts are integrated, thereby eliminating the time required for documentation, the cost of hiring experts, and procedural inconveniences, while maximizing the legal ability to prove and prove the evidence, thereby changing the paradigm of existing contract methods.

[0096] One aspect of the present invention can be implemented as computer-readable code on a computer-readable recording medium. Codes and code segments implementing the above program can be easily inferred by a computer programmer in the art. A computer-readable recording medium includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, etc. Furthermore, the computer-readable recording medium can be distributed across network-connected computer systems, and stored and executed as computer-readable code in a distributed manner.

[0097] The above description merely exemplifies one embodiment of the present invention. Those skilled in the art will appreciate that modifications and variations can be implemented without departing from the essential characteristics of the present invention. Therefore, the scope of the present invention is not limited to the aforementioned embodiments, but should be interpreted to encompass various embodiments within the scope equivalent to the claims.

Claims

1. A contract support service providing server that connects multiple contract terminals, a distributed file system (IPFS), and a blockchain network through a network. Communication unit that communicates with multiple contractor terminals, distributed file system (IPFS) and blockchain network; Service management department that manages member information for multiple contract terminals; A contract content preprocessing unit that, in response to a contract support request from one of a plurality of contract terminals, acquires a voice / video file storing contract-related voice / video data and a voice / video content including metadata for voice / video, generates contract document information corresponding to the contract-related voice / video data, generates a provisional contract content including the generated contract document information and the voice / video content, generates first link information that connects to access the generated provisional contract content and second link information that connects to a web page so that electronic signatures of a plurality of contract terminals can be input, and controls a communication unit to transmit the first link information and the second link information to the plurality of contract terminals included in the contract support request; and When multiple electronic signatures are input for each contracting party from multiple contracting terminals that have confirmed the provisional contract content, a contract content post-processing unit that controls the communication unit to generate a voice / video watermark for the contract content including the provisional contract content and multiple electronic signatures, add the generated voice / video watermark to the provisional contract content to generate a storage contract content, encrypt the storage contract content using an encryption key, and store the encrypted storage contract content in a distributed manner using a distributed file system (IPFS) and a blockchain network; A contract support service providing server characterized in that the metadata for voice / video includes location information and storage time information of the contract terminal where the voice / video file is stored.

2. In paragraph 1, A contract support service providing server characterized in that the contract content preprocessing unit controls the communication unit to transmit a user input screen inquiring whether multiple contracting parties will use one contracting terminal at the same location to record and record or whether multiple contracting parties will use Internet telephony to record and record in order to obtain voice / video files, to the contracting terminal that sent the contract support request.

3. In paragraph 2, When receiving a user input from a contract terminal that has sent a contract support request that multiple contracting parties are using a single contract terminal at the same location to perform recording and video recording, the contract content preprocessing unit receives contract-related voice and video data from the single contract terminal and creates a voice / video file. A contract support service providing server characterized in that, when receiving a user input from a contract terminal that has sent a contract support request that multiple contracting parties perform recording and videotaping using an Internet phone, the contract content preprocessing unit requests an Internet phone channel from a VoIP server connected through a network, transmits recording / recording link information that can access the Internet phone channel to the emails of each of the multiple contracting parties, and receives contract-related voice / video data recorded and videotaped through the Internet phone channel to create a voice / video file.

4. In paragraph 1, The contract content preprocessing unit is a contract support service providing server that creates contract document information using STT (Speech to Text) on the voice contained in the voice / video file.

5. In paragraph 1, A contract support service providing server characterized in that the service management department controls the communication department to request confirmation of audio / video files included in the contract content via the emails of multiple contracting parties included in the member information, to request an electronic signature based on the confirmation input for the audio / video files, to send an authentication number as a text message to the contracting party's terminal, and to receive the transmitted authentication number from the contracting party's terminal using the same screen as the screen where the electronic signature is input, thereby additionally confirming that the contracting party is a contracting party.

6. In paragraph 5, A contract support service providing server characterized in that the contract content post-processing unit receives electronic signature metadata including location information and storage time information where the electronic signature was made along with multiple electronic signatures for each contracting party from multiple contracting terminals, and creates contract content for storage by further including the electronic signature metadata.

7. In paragraph 1, A contract support service providing server characterized in that the contract content post-processing unit generates an encryption key by applying the Bcrypt algorithm to the contract party name and email domain name, encrypts the contract content for storage using the generated encryption key and the AES algorithm, controls the communication unit to upload the encrypted contract content for storage to a distributed file system (IPFS), and when an IPFS hash value is received from the distributed file system (IPFS) through the communication unit, controls the communication unit to store the received IPFS hash value in a blockchain network, and obtains a transaction ID (TxID) for obtaining the IPFS hash value from the blockchain network through the communication unit and a data ID (DataID) which is information identifying the IPFS hash value.

8. A method for providing contract support services by providing a contract support service server in which multiple contract terminals, a distributed file system (IPFS), and a local server are connected through a network, A step of obtaining voice / video content including voice / video files storing voice and video data related to a contract and metadata for voice / video in response to a request for contract support from one of a plurality of contract terminals; A step of creating contract written information corresponding to contract-related voice and video data, and creating a provisional contract content including the created written information and voice and video content; A step of generating first link information that connects to access the generated contract content and second link information that connects to a webpage so that electronic signatures of multiple contracting party terminals can be entered, and transmitting the first link information and the second link information to multiple contracting party terminals included in the contract support request; When multiple electronic signatures are entered for each contracting party from multiple contracting terminals that have received the provisional contract content, a step of creating a voice / video watermark for the contract content including the provisional contract content and multiple electronic signatures; A step of creating a storage contract content by adding the generated voice / video watermark to the contract content; A step of encrypting the storage contract content using an encryption key; and A step of storing encrypted storage contract content in a distributed manner using a distributed file system (IPFS) and a blockchain network; A method for providing a contract support service, characterized in that the metadata for voice / video includes location information and storage time information of the contract terminal where recording and video recording were performed.

Images