System

Abstract

To provide a system.SOLUTION: A system includes means for providing telephone services using SIP communication, means for treating the contents of telephone calls as intangible information assets, means for building a new service provision infrastructure in a range not violating Telecommunications Business Act, and means for providing advanced services that utilize big data and AI.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] In today's world, telephone conversation content is in a sense an intangible information asset, and there is a need to reconsider how we handle it. However, existing telephone services lack the means to effectively utilize the content of telephone conversations, and it is difficult to build a new service infrastructure without violating the Telecommunications Business Act. [Means for solving the problem]

[0005] The present invention provides a means for providing telephone services using SIP communications, a means for treating telephone call content as intangible information assets, a means for constructing a new service provision infrastructure without violating the Telecommunications Business Act, and a means for providing advanced services using BigData and AI. This makes it possible to effectively utilize telephone call content and construct a new service provision infrastructure. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13]FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15] FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16] FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. [Figure 19] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system according to the embodiment 2 of the second embodiment in the case where an emotion engine is combined. [Figure 20] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when combined with an emotion engine. [Figure 21] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the embodiment 3 of the form example 3 in the case where an emotion engine is combined. [Figure 22] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] As an example of the embodiment of the present invention, there is a system that provides telephone services using SIP communication. This system has a function to handle the contents of calls as intangible information assets. Specifically, the contents of calls are digitized, and the data is analyzed and stored to make effective use of the contents of calls. In addition, the system has a function to build a new service provision infrastructure within the scope that does not violate the Telecommunications Business Act. Specifically, new services are provided to users based on data on the contents of calls. Furthermore, the system has a function to provide advanced services that utilize BigData and AI. Specifically, data on the contents of calls is accumulated as BigData, and the data is analyzed using AI to optimize the provision of services to users.

[0029] "Example 2"

[0030] Another embodiment of the present invention is a telephone service using SIP communication with a function for identifying specific call patterns. This system digitizes the contents of calls and identifies specific call patterns by analyzing the data. For example, it identifies call patterns such as calls where a specific keyword is frequently used or calls concentrated in a specific time period, and provides services to users based on the identified call patterns.

[0031] "Example 3"

[0032] Another embodiment of the present invention is a telephone service using SIP communication that has a function of optimizing the provision of services to users based on the content of the calls. This system optimizes the provision of services to users by accumulating data on the content of the calls as BigData and analyzing the data using AI. For example, the system optimizes the provision of services based on the content of the calls by analyzing calls in which a specific keyword is frequently used and providing the user with services related to the keyword.

[0033] The process flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: Make a telephone call using SIP communication.

[0036] Step 2: The contents of the calls are digitized and treated as an intangible information asset.

[0037] Step 3: Analyze and store the call content data to make effective use of the call content.

[0038] Step 4: Build a new service provision infrastructure within the scope of the Telecommunications Business Act. Step 5: Provide new services to users based on the data on call content.

[0039] Step 6: Accumulate the call content data as BigData and analyze the data using AI.

[0040] Step 7: Optimize the provision of services to users based on the results of AI analysis.

[0041] "Example 2"

[0042] Step 1: Make a telephone call using SIP communication.

[0043] Step 2: Record the call contents and analyze the data.

[0044] Step 3: Identify specific call patterns from the analysis results.

[0045] Step 4: Provide services to the user based on the identified call patterns.

[0046] Step 1: Make a telephone call using SIP communication.

[0047] Step 2: Accumulate the call content data as BigData.

[0048] Step 3: Use AI to analyze the accumulated data.

[0049] Step 4: Optimize the provision of services to users based on the results of AI analysis.

[0050] Example 1

[0051] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0052] Conventional telephone service systems had limited means of effectively utilizing call content, making it difficult to digitize call content for analysis and storage. It was also difficult to build a new service provision infrastructure without violating the Telecommunications Business Act, and services provided to users were not fully optimized. Furthermore, there was a demand for advanced technology to capture call content in real time, store it as voice data, and convert it into text data for analysis.

[0053] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0054] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the calls as intangible information assets, a means for converting the contents of the calls into digital data and analyzing and storing the data, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing new services to users based on the data of the calls, a means for providing advanced services using BigData and AI, a means for capturing the contents of the calls in real time and storing them as voice data, a means for converting the voice data into text data using voice recognition software, a means for storing the converted text data in a database, a means for analyzing the accumulated data with an AI model, and a means for providing services to users based on the analysis results. This makes it possible to effectively utilize the contents of the calls and optimize the provision of services to users.

[0055] "SIP communication" is a communications technology for making voice and video calls using the Session Initiation Protocol.

[0056] "Telephone services" refers to all services for providing voice calls.

[0057] "Intangible information assets" refers to information data that has value but does not have a physical form.

[0058] "Digitization" refers to the conversion of information such as audio and video into digital data.

[0059] "Analysis" refers to the processing of data to extract useful information.

[0060] "Storage" refers to keeping data for an extended period of time.

[0061] The Telecommunications Business Act is a law concerning the operation of telecommunications businesses.

[0062] "Service Provision Infrastructure" means the technical and physical infrastructure for providing the Services.

[0063] "Big Data" refers to large amounts of data and includes the technologies to collect, store, and analyze that data.

[0064] "AI" refers to artificial intelligence and includes technologies such as machine learning and natural language processing.

[0065] "Real-time" refers to processing that occurs immediately, without delay.

[0066] "Capture" refers to obtaining data.

[0067] "Audio data" refers to data that represents audio in digital form.

[0068] "Speech recognition software" means software for converting speech into text.

[0069] "Text data" refers to data that represents character information in digital form.

[0070] "Database" refers to a system for efficiently storing, managing, and retrieving data.

[0071] "AI model" refers to a model for analyzing data using artificial intelligence algorithms.

[0072] "Analysis Results" refers to information obtained through data analysis.

[0073] "Service provision" refers to providing useful functions and information to users.

[0074] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0075] The present invention is a system that provides telephone services using SIP communication and handles call content as intangible information assets. A specific embodiment of this system will be described below.

[0076] 1. Generate the system program

[0077] The server generates a program for providing telephone services using SIP communication. This program has the function of converting the contents of the call into data and analyzing and saving that data.

[0078] 2. Explain the program's processing in natural language

[0079] The server captures the call in real time and stores it as voice data using SIP server software such as Asterisk, which is then converted to text data using voice recognition software (e.g., Google® Cloud Speech-to-Text API).

[0080] The terminal receives the converted text data and stores it in a database (e.g., MySQL (registered trademark)). The stored data is accumulated using BigData technology (e.g., Hadoop).

[0081] The server analyzes the accumulated data with an AI model (e.g., a natural language processing model using TENSORFLOW (registered trademark)) to provide new services to users, including customized suggestions based on the content of the call, call summaries, and sentiment analysis.

[0082] 3. Add examples to your description

[0083] As a concrete example, consider a case where a user calls customer support and asks, "How do I return an item?"

[0084] The server captures the contents of the call and stores them as voice data. It then uses voice recognition software to convert the voice data into text data such as "Please tell me how to return the product." The converted text data is stored in a database and accumulated using BigData technology.

[0085] The AI ​​model analyzes this text data and provides users with detailed guidelines on how to return items. It can also predict frequently asked questions based on past call data and provide FAQs in advance.

[0086] Example of a prompt to be input to a generative AI model:

[0087] "When a user calls customer support to ask how to return an item, digitize the conversation, analyze it, and explain what kind of service you would provide to that user."

[0088] In this way, a system is constructed in which the server, terminals, and users work together to effectively utilize call contents and provide new services.

[0089] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0090] Step 1:

[0091] The server captures calls between users in real time using SIP communication. The input is the call audio between user A and user B, and the output is the captured audio data. Specifically, the server uses SIP server software such as Asterisk to obtain the audio data of the call. For example, when user A says, "Please tell me how to return a product," the audio is captured.

[0092] Step 2:

[0093] The server temporarily stores the captured voice data. The input is the voice data captured in step 1, and the output is the stored voice data. The data is stored in storage within the server (e.g., cloud storage). Specifically, the data is stored as an audio file, and the call date and time and caller information are included in the file name.

[0094] Step 3:

[0095] The server converts the stored voice data into text data using voice recognition software. The input is the voice data stored in step 2, and the output is the converted text data. Specifically, the voice data is converted into text using Google Cloud Speech-to-Text API. For example, the voice saying "Please tell me how to return the product" is converted into text "Please tell me how to return the product."

[0096] Step 4:

[0097] The terminal receives the converted text data and stores it in a database. The input is the text data converted in step 3, and the output is the text data stored in the database. Specifically, it is stored in a MySQL database, and when it is stored, the call date and time and caller information are also stored.

[0098] Step 5:

[0099] The server accumulates and manages the saved text data using BigData technology. The input is the text data saved in step 4, and the output is the accumulated data. Specifically, the data is saved in distributed storage using Hadoop and indexed for efficient access.

[0100] Step 6:

[0101] The server analyzes the accumulated data using an AI model. The input is the data accumulated in step 5, and the output is the analysis results. Specifically, the data is analyzed using a natural language processing model that uses TensorFlow, and sentiment analysis, keyword extraction, and trend analysis are performed.

[0102] Step 7:

[0103] The user receives new services based on the analysis results. The input is the analysis results obtained in step 6, and the output is the service provided to the user. Specifically, this includes customized suggestions based on the content of the call, a summary of the call, and the provision of FAQs. For example, if a user says, "Please tell me how to return an item," the server will provide "detailed guidelines on how to return the item." In addition, based on past call data, the server can predict what questions users will frequently ask and provide FAQs in advance.

[0104] (Application example 1)

[0105] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0106] In the conventional telephone service system, the means to effectively utilize the call contents were limited, and the service provision to users was not sufficiently optimized. In addition, the function to analyze the call contents in real time and recommend the content related to the user was lacking, so an improvement of the user experience was required.

[0107] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0108] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the calls as intangible information assets, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, a means for converting the contents of the calls into digital data in real time and analyzing them using AI, and a means for recommending content related to the user based on the analysis results. This makes it possible to provide advanced services based on the contents of the calls and recommend optimal content to the user.

[0109] "SIP communication" is a communications technology for making voice and video calls using the Session Initiation Protocol.

[0110] "Telephone service" means a system or means for providing voice communications.

[0111] "Call Content" refers to voice data and information exchanged through telephone services.

[0112] "Intangible information assets" are information or data stored in digital form that have value despite not having a physical form.

[0113] The Telecommunications Business Act is a law concerning the operation and regulation of telecommunications businesses.

[0114] "Service Provisioning Infrastructure" means the technical and operational infrastructure for providing the Services.

[0115] "Big Data" refers to large volumes of diverse data sets, and is a technology that creates new knowledge and value by analyzing them.

[0116] "AI" is an abbreviation for artificial intelligence, a technology that allows machines to imitate human intelligence to learn and make inferences.

[0117] "Digitization in real time" refers to instantly converting the contents of a call into digital data.

[0118] "Analysis" is the act of processing data to extract useful information.

[0119] "Content" refers collectively to information and entertainment provided to Users.

[0120] A "recommendation" is the act of presenting the best option based on certain criteria.

[0121] The system for carrying out this invention has the following configuration: The server includes means for providing telephone services using SIP communication, means for treating the contents of the calls as intangible information assets, means for building a new service provision infrastructure within the scope of the Telecommunications Business Act, means for providing advanced services using BigData and AI, means for converting the contents of the calls into digital data in real time and analyzing them using AI, and means for recommending content relevant to the user based on the analysis results.

[0122] Specifically, when a user makes a call using a smartphone, the contents of the call are digitized in real time. This data is sent to a server and analyzed using an AI model. For the analysis, a voice recognition library (e.g., the speech_recognition library) is used to convert the contents of the call into text. The converted text is sent to the API endpoint of the AI ​​model for detailed analysis.

[0123] The server recommends content relevant to the user based on the analysis results. For example, if a user says, "I want to talk about recent movies," the AI ​​model analyzes this text and recommends the latest movie information and reviews. This recommendation is notified to the user's smartphone.

[0124] The hardware used is the smartphone's microphone and speaker, and the server's processing power. The software uses the speech_recognition library and the API of the AI ​​model.

[0125] As a concrete example, if a user says, "I want to talk about recent movies," the AI ​​model analyzes this text and recommends the latest movie information and reviews. The following is an example of a prompt sentence to be input to the generative AI model.

[0126] "If a user says, 'I want to talk about recent movies,' recommend the latest movie information and reviews."

[0127] In this way, advanced services based on the content of the call and optimal content recommendations for the user can be realized.

[0128] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0129] Step 1:

[0130] A user starts a call using a smartphone. The smartphone's microphone captures the voice and digitizes the call content in real time. The input is the user's voice, and the output is digital voice data.

[0131] Step 2:

[0132] The smartphone transmits digital voice data to the server. The server converts the voice data into text using a voice recognition library (e.g., speech_recognition library). The input is the digital voice data and the output is the text data.

[0133] Step 3:

[0134] The server sends the converted text data to the API endpoint of the AI ​​model. The AI ​​model analyzes the text data and extracts the user's intent and interests. The input is the text data, and the output is the analysis result.

[0135] Step 4:

[0136] The server recommends content relevant to the user based on the analysis results. For example, if a user says, "I want to talk about recent movies," the AI ​​model recommends the latest movie information and reviews. The input is the analysis results, and the output is the recommended content.

[0137] Step 5:

[0138] The server notifies the user of the recommended content to the smartphone. The smartphone receives the notification and displays it to the user. The input is the recommended content, and the output is the notification to the user.

[0139] In this way, advanced services based on the contents of the call and optimal content recommendations to the user are realized.

[0140] Example 2

[0141] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0142] In conventional telephone services using SIP communications, it was difficult to efficiently analyze call content and identify specific call patterns. In addition, there was a lack of means to digitize call content in real time and use that data to optimize services for users. This made it difficult to respond quickly and accurately to user needs, resulting in a decline in the quality of service.

[0143] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0144] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using large amounts of data and artificial intelligence, a means for capturing the contents of the call in real time and converting the voice data into text data, a means for storing the converted text data, a means for analyzing the stored text data and extracting patterns such as the frequency of appearance of specific keywords and the time period of the call, and a means for transmitting the analysis results to a user terminal and visually displaying them. This makes it possible to efficiently analyze the contents of the call and identify specific call patterns, thereby optimizing the provision of services to users.

[0145] "SIP communication" is a technology that uses the Session Initiation Protocol to manage multimedia communications such as voice and video.

[0146] "Telephone service" refers to a communication service for providing voice calls.

[0147] "Intangible information assets" are information or data stored in digital form, and are assets that have value despite not having a physical form.

[0148] The Telecommunications Business Act is a law concerning the operation and regulation of telecommunications businesses.

[0149] "Service Provisioning Platform" means the technical and operational infrastructure for providing the Services.

[0150] "Massive data", also known as big data, refers to a very large data set, the amount of data that cannot be handled by conventional data processing techniques.

[0151] "Artificial intelligence" is a technology that enables machines to mimic human intelligence and learn, reason, and solve problems.

[0152] "Real-time" refers to data processing and communication occurring immediately and without delay.

[0153] "Audio data" refers to data in which audio is recorded in digital format.

[0154] "Text data" refers to data in which character information is recorded in digital format.

[0155] A "database" is a system for efficiently storing, searching, and managing data.

[0156] "Natural language processing" is a technology that enables computers to understand and generate human language.

[0157] A "keyword" is a word or phrase that is important when searching for or analyzing specific information.

[0158] A "user terminal" is a device such as a computer or smartphone that is directly operated by a user.

[0159] "Visual display" refers to presenting data or information to a user in an easily viewable form, such as in a graph or chart.

[0160] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0161] This invention is a system for telephone services using SIP communication that digitizes call contents in real time, analyzes the data to identify specific call patterns, and optimizes services for users. A specific embodiment of this system is described below.

[0162] Hardware and Software Used

[0163] The server uses SIP server software (e.g., Asterisk or FreeSWITCH) to capture calls made through SIP communication in real time, and uses a speech recognition API (e.g., Google Cloud Speech-to-Text API) to convert the captured voice data into text data, which is then stored in a relational database (e.g., MySQL or PostgreSQL).

[0164] The server uses Python natural language processing libraries (e.g., NLTK and spaCy) to analyze the stored text data, extracting patterns such as the frequency of occurrence of certain keywords or the time of day of calls.

[0165] The terminal uses a JavaScript (registered trademark) library (e.g., Chart.js or D3.js) to receive the analysis results sent from the server and visually display them to the user.

[0166] Examples

[0167] Consider the case where the user is an administrator of a customer support center. The user wants to identify calls that frequently use certain keywords (e.g., "complaint" or "returned goods"). The server automatically extracts calls that contain these keywords and sends the analysis results to the terminal. The terminal displays a graph to the user showing which time periods these calls are concentrated.

[0168] Example of a prompt to be input to a generative AI model:

[0169] "Analyze call data from your customer support center and identify calls that frequently use certain keywords (e.g., 'claims' or 'returns'). Also, create a graph to show which times of day these calls are concentrated."

[0170] In this way, users can easily keep track of calling patterns and take appropriate measures.

[0171] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0172] Step 1:

[0173] The server captures calls made through SIP communication in real time. Specifically, it uses SIP server software (e.g., Asterisk or FreeSWITCH) to obtain the voice data of calls. The input is the call data from SIP communication, and the output is the captured voice data.

[0174] Step 2:

[0175] The server converts the captured voice data into text data. This conversion is done using a voice recognition API (e.g., Google Cloud Speech-to-Text API). The server sends the voice data to the API and receives the returned text data. The input is the captured voice data, and the output is the converted text data.

[0176] Step 3:

[0177] The server stores the converted text data in a database. Specifically, a relational database (e.g., MySQL or PostgreSQL) is used. The text data is stored together with the call timestamp and caller information. The input is the converted text data, and the output is the text data stored in the database.

[0178] Step 4:

[0179] The server analyzes the stored text data. It uses Python natural language processing libraries (e.g., NLTK and spaCy) to extract patterns such as the frequency of occurrence of specific keywords or the time of day of calls. The input is the text data stored in the database, and the output is the analysis results.

[0180] Step 5:

[0181] The server sends the analysis results to the user terminal. Specifically, it uses a RESTful API to send data in JSON format. The input is the analysis results, and the output is the data sent to the user terminal.

[0182] Step 6:

[0183] The terminal visually displays the received analysis results to the user. It uses JavaScript libraries (e.g. Chart.js and D3.js) to display graphs and lists of calls where specific keywords are frequently used or where calls are concentrated in a specific time period. The input is the analysis results sent from the server, and the output is the visually displayed data.

[0184] (Application example 2)

[0185] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0186] With conventional telephone services, it was difficult to analyze call content in real time and identify specific keywords or patterns. In addition, there was a lack of functionality to detect dangerous call patterns and send warnings to users, so sufficient measures could not be taken against risks such as fraud and blackmail. Furthermore, there was a lack of functionality to analyze call history and provide reports on dangerous call patterns, making it difficult to ensure the safety of users.

[0187] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0188] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, a means for converting the contents of the call into data in real time and identifying specific keywords or patterns, a means for sending a warning to the user based on the identified keywords or patterns, and a means for analyzing the call history and providing a report of dangerous call patterns. This makes it possible to detect specific keywords or patterns in real time during a call and immediately send a warning to the user. In addition, by analyzing the call history and providing a report of dangerous call patterns, the safety of the user can be ensured.

[0189] "SIP communication" is a communication method that uses the Session Initiation Protocol to make voice and video calls.

[0190] "Telephone service" refers to a communication service for providing voice calls.

[0191] An "intangible information asset" is information stored as digital data; it is an asset that has value despite not having a physical form.

[0192] The Telecommunications Business Act is a law concerning the operation of telecommunications businesses, and sets out regulations concerning the provision of telecommunications services.

[0193] "Service provision infrastructure" means the technical and operational infrastructure for providing a service.

[0194] "Big Data" refers to large amounts of data, and is a concept that includes technologies and methods for collecting, storing, and analyzing that data.

[0195] "AI" is an abbreviation for artificial intelligence, a technology that allows machines to imitate human intelligence to learn and make inferences.

[0196] An "advanced service" is a service that has more advanced functions and performance than conventional services.

[0197] "Real-time" refers to immediate processing or response without delay.

[0198] "Digitization" refers to converting information into digital data.

[0199] A "keyword" is a word that has a particular meaning or importance.

[0200] A "pattern" is a collection of data that has a particular regularity or tendency.

[0201] A "warning" is a notification that warns of a possible danger or problem.

[0202] A "call history" is a record of past calls.

[0203] A "report" is a report that summarizes analysis results and information.

[0204] A system for carrying out this invention is configured as follows: The server includes means for providing telephone services using SIP communication, means for treating call contents as intangible information assets, means for building a new service provision infrastructure within the scope of the Telecommunications Business Act, means for providing advanced services using BigData and AI, means for converting call contents into data in real time and identifying specific keywords or patterns, means for sending a warning to the user based on the identified keywords or patterns, and means for analyzing call history and providing a report on risky call patterns.

[0205] Program processing explanation

[0206] Hardware used

[0207] Smartphone (microphone, speaker)

[0208] Software used

[0209] Python

[0210] SpeechRecognition Library

[0211] Google Speech Recognition API

[0212] Data processing and calculation

[0213] The server first captures the call audio through the smartphone's microphone. The captured audio data is converted to text data using the SpeechRecognition library. The converted text data is then analyzed to identify specific keywords and patterns. This analysis includes matching keywords using regular expressions.

[0214] Based on the identified keywords and patterns, the server sends a warning to the user in real time via the smartphone's notification function, and the call history is stored in a database for later analysis. The analysis results are then provided to the user as a report of risky calling patterns.

[0215] Examples

[0216] For example, if the keyword "scam" is detected during a user's call, the server will immediately display a warning and record the call content and the detected keyword. This warning will be sent to the user through the smartphone's notification function. The call history will also be stored in a database for later analysis. The analysis results will be provided to the user as a report of risky calling patterns.

[0217] Examples of prompt statements

[0218] "Write a Python program that will alert the user and record the call if certain keywords (e.g. scam, blackmail) are detected during the call."

[0219] In this way, it is possible to detect certain keywords or patterns during calls in real time and send immediate warnings to users, and also analyze call history to provide reports on risky calling patterns to ensure user safety.

[0220] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0221] Step 1:

[0222] The server captures the call audio through the smartphone's microphone. The input is the call audio data, and the output is an audio data file. This audio data is stored for analysis in subsequent processing steps.

[0223] Step 2:

[0224] The server converts the acquired voice data into text data using the SpeechRecognition library. The input is a voice data file, and the output is text data. This conversion process converts the voice data into an analyzable text format.

[0225] Step 3:

[0226] The server analyzes the converted text data to identify specific keywords or patterns. The input is the text data, and the output is a list of identified keywords or patterns. This analysis includes matching keywords using regular expressions.

[0227] Step 4:

[0228] The server sends alerts to users based on the identified keywords and patterns. The input is a list of identified keywords and patterns, and the output is an alert notification to the user. This notification is done in real time through the smartphone notification feature.

[0229] Step 5:

[0230] The server stores the call history in a database. The input is the text data of the call and a list of identified keywords or patterns, and the output is the call history stored in the database. This storage process accumulates data for later analysis of the call history.

[0231] Step 6:

[0232] The server periodically analyzes the stored call history and provides a report on risky calling patterns. The input is the call history stored in the database, and the output is a report of the analysis results. This report is provided to the user and is used to understand risky calling patterns.

[0233] Example 3

[0234] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0235] With conventional telephone services, it was difficult to efficiently analyze the contents of phone calls and optimize the services provided to users. In addition, there was a lack of appropriate means to treat the contents of phone calls as intangible information assets, and there was a need to build a new service provision infrastructure that did not violate the Telecommunications Business Act. Furthermore, no specific method had been established for providing advanced services that utilize large amounts of data and artificial intelligence.

[0236] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0237] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using large amounts of data and artificial intelligence, a means for capturing the contents of the call in real time and converting the voice data into text data, a means for storing and analyzing the converted text data, and a means for optimizing the provision of services to users based on the analysis results. This makes it possible to efficiently analyze the contents of the call and optimize the provision of services to users.

[0238] "SIP communication" is a communications technology for making voice and video calls using the Session Initiation Protocol.

[0239] "Telephone service" means a system or means for providing voice communications.

[0240] "Intangible information assets" are information or data stored in digital form, and are assets that have value despite not having a physical form.

[0241] The Telecommunications Business Act is a law concerning the operation and regulation of telecommunications businesses.

[0242] "Service Provisioning Platform" means the technical and operational infrastructure for providing the Services.

[0243] "Massive data" refers to very large data sets, usually referred to as big data.

[0244] "Artificial intelligence" is a technology that allows computers to imitate human intelligence to learn and make inferences.

[0245] "Real-time" refers to immediate processing or response without delay.

[0246] "Audio data" refers to data in which audio is recorded in digital format.

[0247] "Text data" refers to data in which character information is recorded in digital format.

[0248] "Storage" refers to storing data and making it available for later use.

[0249] "Analysis" is the process of examining data in detail to find meaning and patterns.

[0250] "Optimization" refers to finding the most efficient way to achieve a particular goal.

[0251] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0252] This invention is a system that provides telephone services using SIP communication, treats the contents of calls as intangible information assets, and builds a new service provision infrastructure within the scope of the Telecommunications Business Law. Furthermore, it provides advanced services by utilizing large amounts of data and artificial intelligence, captures the contents of calls in real time, converts the voice data into text data, and accumulates and analyzes the converted text data, optimizing the provision of services to users.

[0253] Hardware and Software Used

[0254] The server uses the following hardware and software:

[0255] SIP communication module: A communication technology for making voice calls using SIP communication.

[0256] Speech recognition API: An API for converting voice data into text data. Specifically, we use the Google Cloud Speech-to-Text API.

[0257] Data storage system: A distributed file system for efficiently storing and managing large amounts of text data. Specifically, it uses a Hadoop cluster.

[0258] Generative AI model: An AI model for analyzing text data and extracting frequently used keywords and phrases. Specifically, we use GPT-4 (registered trademark) from OpenAI (registered trademark).

[0259] Program processing explanation

[0260] The server captures the conversation between the user and the device in real time. When the conversation starts, the server receives the voice data in streaming format and converts it into text data using the Google Cloud Speech-to-Text API. The converted text data is stored in a Hadoop cluster.

[0261] Next, the server analyzes the accumulated text data using a generative AI model (GPT-4). The AI ​​model extracts frequently used keywords and phrases from the text data and identifies the user's needs and problems. Based on the analysis results, the server optimizes the service provision to the user. For example, if a certain keyword is frequently used, it automatically suggests services related to that keyword.

[0262] Examples

[0263] As a concrete example, consider the case where a user calls customer support and says, "The product was broken and I want to return it. What should I do?" The contents of this call are captured in real time and converted into text data using the Google Cloud Speech-to-Text API. The data is then stored in a Hadoop cluster and analyzed by a generative AI model (GPT-4).

[0264] As a result of the analysis, it is found that keywords such as "broken" and "return" are frequently used. Based on this information, the server can provide the user with information on the return procedure and a service to exchange the broken product.

[0265] Examples of prompt statements

[0266] Example prompt:

[0267] Please enter the user's call details in the following format:

[0268] "The item was broken and I want to return it. What should I do?"

[0269] By inputting this prompt sentence into the generative AI model, the AI ​​proposes an appropriate service provision method. For example, it instructs the server to take specific actions such as "providing detailed information on the return procedure" or "arranging for a replacement product." The flow of the specific process in the third embodiment will be explained with reference to FIG. 15.

[0270] Program processing flow

[0271] Step 1: Capture the call

[0272] The server captures the contents of the call between the user and the terminal in real time. When the call starts, the server receives the voice data in streaming format through SIP communication. The input is the user's voice data, and the output is the voice data sent to the server.

[0273] Specific behavior:

[0274] A user calls customer support.

[0275] The server captures the contents of the call using SIP communication.

[0276] Step 2: Convert audio to text

[0277] The server converts the received voice data into text data using the Google Cloud Speech-to-Text API. The input is the voice data sent to the server, and the output is the text data.

[0278] Specific behavior:

[0279] The server sends the audio data to the Google Cloud Speech-to-Text API.

[0280] The API converts the audio data into text data.

[0281] The server receives the converted text data.

[0282] Step 3: Accumulating text data

[0283] The server accumulates the converted text data in the Hadoop cluster. The input is the text data, and the output is the text data stored in the Hadoop cluster.

[0284] Specific behavior:

[0285] The server sends the text data to the Hadoop cluster.

[0286] The Hadoop cluster stores the text data.

[0287] Step 4: Analyze the text data

[0288] The server analyzes the accumulated text data using a generative AI model (GPT-4). The input is the text data stored in the Hadoop cluster, and the output is keywords and phrases as the analysis results.

[0289] Specific behavior:

[0290] The server retrieves the text data from the Hadoop cluster.

[0291] The server inputs the text data into the generative AI model.

[0292] The AI ​​model analyzes the text data and extracts frequently used keywords and phrases.

[0293] The server receives the analysis results.

[0294] Step 5: Optimize service delivery

[0295] The server optimizes the service provided to the user based on the analysis results of the AI ​​model. The input is keywords or phrases as the analysis results, and the output is the optimized service provided.

[0296] Specific behavior:

[0297] The server identifies the user's needs based on the analysis results.

[0298] The server suggests appropriate services based on the identified needs.

[0299] For example, if keywords such as "broken" and "return" are frequently used, the server will provide the user with guidance on return procedures and a service for replacing broken products.

[0300] (Application example 3)

[0301] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0302] With conventional telephone services, it was difficult to effectively utilize the contents of phone calls to optimize the provision of services to users. In addition, it was not possible to propose content in real time based on the contents of phone calls, making it difficult to provide services that meet the needs of users.

[0303] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means. In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope not infringing the Telecommunications Business Act, a means for providing advanced services using BigData and AI, and a means for analyzing the contents of the call in real time and automatically proposing content based on the contents after the call ends. This makes it possible to propose content in real time based on the contents of the call.

[0304] "SIP communication" is a communication protocol for making voice and video calls using the Internet Protocol.

[0305] "Telephone service" refers to a communication service for providing voice calls.

[0306] "Call content" refers to the audio information spoken during a telephone call.

[0307] "Intangible information assets" refers to information or data stored in digital form.

[0308] The Telecommunications Business Act is a law concerning the operation of telecommunications businesses.

[0309] "Service Delivery Infrastructure" means the technical and physical infrastructure for delivering a service.

[0310] "Big Data" refers to large amounts of data and the technology to extract useful information by analyzing it.

[0311] "AI" stands for artificial intelligence, a technology that allows machines to imitate human intelligence and learn and reason.

[0312] An "advanced service" is a service that provides higher added value than conventional services.

[0313] "Real-time analysis" means processing data instantly and providing results immediately.

[0314] "Content" means the information and entertainment provided to Users.

[0315] "Automatic suggestion" means that the system provides appropriate information or services without user interaction.

[0316] The system for carrying out this invention is configured as follows: The server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, and a means for analyzing the contents of the call in real time and automatically proposing content based on the contents after the call is ended.

[0317] First, when a user starts a call, the terminal makes the call using SIP communication. Voice data during the call is sent to the server in real time, and the server stores this voice data as an intangible information asset. Next, the server converts the voice data into text using voice recognition software (for example, Google's voice recognition API). This text data is stored as BigData.

[0318] The server analyzes the stored text data using AI (e.g., natural language processing models). As a result of the analysis, certain keywords and call patterns are extracted. Based on this, the server generates content to optimize the provision of services to users. For example, if a user talks about movies during a call, the server will suggest reviews, trailers, viewing links, etc. related to the movie.

[0319] As a concrete example, if a user talks about "recent movies" during a call, the server can extract keywords such as "movies," "reviews," and "trailers," and suggest related content to the user based on those keywords.

[0320] Here is an example of a prompt to be input to the generative AI model:

[0321] "When a user is on a call and they're talking about 'recent movies,' extract keywords to suggest related content."

[0322] In this way, real-time content suggestions based on the contents of the call are possible, and services that meet the needs of the user are provided.

[0323] The flow of the specific process in the application example 3 will be described with reference to FIG.

[0324] Step 1:

[0325] When a user initiates a call, the terminal uses SIP communication to make the call. The input is the user's voice, and the output is the voice data sent to the server in real time. The terminal uses a microphone to capture the voice and sends it to the server via the SIP protocol.

[0326] Step 2:

[0327] The server stores the received voice data as an intangible information asset. The input is the voice data sent from the terminal, and the output is the voice data stored on the server. The server uses a database to store the voice data.

[0328] Step 3:

[0329] The server uses speech recognition software (e.g. Google's speech recognition API) to convert the voice data to text. The input is the stored voice data, and the output is the text data. The server sends the voice data to the speech recognition API and receives the text data in return.

[0330] Step 4:

[0331] The server stores the text data as BigData. The input is the text data obtained from the speech recognition software, and the output is the text data stored on the server. The server uses a database to store the text data.

[0332] Step 5:

[0333] The server analyzes the stored text data using AI (e.g., a natural language processing model). The input is the stored text data, and the output is the extracted keywords and call patterns. The server inputs the text data into the natural language processing model and extracts keywords and call patterns.

[0334] Step 6:

[0335] The server generates content to optimize service provision to users based on the extracted keywords and call patterns. The input is the extracted keywords and call patterns, and the output is the generated content. The server uses a generative AI model to generate relevant content.

[0336] Step 7:

[0337] The server proposes the generated content to the user. The input is the generated content and the output is the content provided to the user. The server transmits the content to the user's terminal and the user receives the content through the terminal.

[0338] In this way, real-time content suggestions based on the contents of the call are possible, and services that meet the needs of the user are provided.

[0339] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0340] "Example 1"

[0341] One embodiment of the present invention is a system that combines an emotion engine with a telephone service using SIP communication. This system recognizes emotions from the user's voice during a call and treats the information as an intangible information asset. Specifically, emotions are analyzed from the user's tone of voice, speaking style, choice of words, etc., and the results are stored in a database. This emotion data is used to increase user satisfaction and improve services.

[0342] "Example 2"

[0343] Another embodiment of the present invention is a system that uses an emotion engine to analyze the emotional state of a user and uses the results as reference information for providing services. Specifically, when a user expresses an emotion such as joy, anger, or sadness, a service is provided based on that information. For example, when a user expresses anger, a response such as contacting customer support is made based on that information.

[0344] "Example 3"

[0345] Furthermore, as another embodiment of the present invention, there is a system that optimizes the provision of services to a user based on the emotion recognition result by the emotion engine. Specifically, the system provides optimal services according to the user's emotional state. For example, if the user expresses an emotion of joy, the system responds by recommending related services or products based on that information.

[0346] The process flow of each embodiment will be described below.

[0347] "Example 1"

[0348] Step 1: A user initiates a telephone call using SIP communication.

[0349] Step 2: The emotion engine recognizes emotions from the user's voice during the call.

[0350] Step 3: The emotion information recognized by the emotion engine is stored in a database.

[0351] "Example 2"

[0352] Step 1: A user initiates a telephone call using SIP communication.

[0353] Step 2: The emotion engine analyzes the user's emotional state during the call.

[0354] Step 3: Use the analysis results as reference information for service provision.

[0355] "Example 3"

[0356] Step 1: A user initiates a telephone call using SIP communication.

[0357] Step 2: The emotion engine analyzes the user's emotional state during the call.

[0358] Step 3: Optimize service provision to users based on the analysis results.

[0359] Example 1

[0360] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0361] Conventional telephone service systems have limited means of effectively utilizing call content, making it difficult to digitize call content for analysis and storage. It is also difficult to recognize user emotions in real time and provide services based on them. Another issue is building a new service provision infrastructure without violating the Telecommunications Business Act. To solve these issues, a system is needed that treats call content as an intangible information asset and provides advanced services that utilize BigData and AI.

[0362] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0363] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for converting the contents of the call into digital data and analyzing and storing the data, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, a means for recognizing emotions from the voice of a user during a call and treating the information as an intangible information asset, and a means for analyzing emotions from the tone of the user's voice, speaking style, choice of words, etc., and storing the results in a database. This makes it possible to effectively utilize the contents of the call and optimize service provision by recognizing the user's emotions in real time.

[0364] "SIP communication" is a communications technology for establishing and managing voice and video calls using the Session Initiation Protocol.

[0365] "Telephone service" refers to a communication service for providing voice calls.

[0366] "Intangible information assets" are information data that have value even though they do not have a physical form.

[0367] "Digitization" is the process of converting information such as audio and video into digital data.

[0368] "Analysis" is the process of examining data in detail to find meaning and patterns.

[0369] "Preservation" is the process of recording and storing data for later use.

[0370] The Telecommunications Business Act is a law concerning the operation and regulation of telecommunications businesses.

[0371] "Service provision infrastructure" means the technical and operational infrastructure for providing a service.

[0372] "Big Data" refers to large amounts of data, as well as the technologies and methods for collecting, storing, and analyzing that data.

[0373] "AI" is an abbreviation for artificial intelligence, a technology that allows machines to imitate human intelligence to learn and make inferences.

[0374] An "emotion engine" is software that analyzes and recognizes emotions from voice and text.

[0375] "Tone" refers to characteristics of voice such as pitch, intensity, and sound quality.

[0376] A "database" is a system for systematically storing and managing data.

[0377] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0378] This invention is a system that provides telephone services using SIP communications and treats the contents of phone calls as intangible information assets. This system digitizes the contents of phone calls, analyzes and stores the data, and makes effective use of the contents of phone calls. It also has the function of building a new service provision infrastructure within the scope of the Telecommunications Business Act. It also has the function of providing advanced services that utilize BigData and AI.

[0379] Hardware and Software Used

[0380] Hardware: Servers, terminals

[0381] Software: SIP communication protocol, database management systems (e.g. MySQL), AI analysis tools (e.g. TensorFlow), emotion engines

[0382] Specific operation of the system

[0383] 1. Start a call:

[0384] A user uses a terminal to make a call via SIP communication.

[0385] The terminal sends a call request to the server using the SIP protocol.

[0386] The server accepts the call request and establishes the call.

[0387] Specific behavior:

[0388] The user enters a phone number into the dial pad of the smartphone and presses the call button.

[0389] The terminal sends a SIP INVITE message to the server.

[0390] The server responds with a SIP 200 OK message and the call is established.

[0391] 2. Digitization of call contents:

[0392] The server captures the contents of the call as audio data in real time.

[0393] The server uses speech recognition software (e.g. Google Speech-to-Text API) to convert the voice data into text data.

[0394] Specific behavior:

[0395] The server captures the audio stream of the call and sends it to the speech recognition software.

[0396] Speech recognition software converts the speech "Hello, how can I help you today?" into text.

[0397] 3. Data storage:

[0398] The server stores the converted text data in a database.

[0399] The server manages the data using a database management system (e.g. MySQL).

[0400] Specific behavior:

[0401] The server uses the INSERT statement to save the text data "Hello, how are you today?" into the MySQL database.

[0402] 4. Sentiment analysis:

[0403] The server uses an emotion engine to analyze emotions based on the user's tone of voice, speaking style, choice of words, etc.

[0404] The server stores the analysis results in a database.

[0405] Specific behavior:

[0406] The server sends the text data "Hello, how are you today?" to the emotion engine.

[0407] The emotion engine analyzes it as a "neutral emotion" and stores the result in a database.

[0408] 5. Leverage data:

[0409] The server provides new services to users based on the analysis results.

[0410] The server refers to the user's emotional data and takes appropriate action.

[0411] Specific behavior:

[0412] The server will provide standard customer support to users whose sentiment is analyzed as "neutral."

[0413] If a server is analyzed as "unhappy," an alert is sent to customer support to promptly respond.

[0414] 6. Advanced service provision:

[0415] The server uses BigData and AI to accumulate and analyze data on call content to optimize service provision.

[0416] The server uses AI analytics tools (e.g. TensorFlow) to analyze the data and identify patterns and trends.

[0417] Specific behavior:

[0418] The server feeds a large amount of call data into TensorFlow and trains a model to identify common issues.

[0419] The server uses the trained model to analyze new call data in real time to help improve the service.

[0420] Examples of prompt statements

[0421] Examples of prompt sentences to be input to a generative AI model include the following:

[0422] "Create a program that saves the contents of a user's phone call as text data and analyzes the emotions using an emotion engine."

[0423] "Please design a system that will optimize the services provided to users by accumulating call content as BigData and analyzing it using AI."

[0424] In this way, a system is realized that provides advanced telephone services in cooperation with users, servers, and terminals.

[0425] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0426] Step 1:

[0427] Starting a call

[0428] Input: A user uses a terminal to enter a phone number and presses call.

[0429] Specific operation: The user enters a phone number on the dial pad of the smartphone and presses the call button. The device sends a SIP INVITE message to the server.

[0430] Data processing: The terminal sends a call request to the server using the SIP protocol.

[0431] Output: The server returns a SIP 200 OK message and the call is established.

[0432] Step 2:

[0433] Digitization of call contents

[0434] Input: The server receives the audio data of the call.

[0435] What happens: The server captures the audio stream of the call and sends it to speech recognition software.

[0436] Data processing: The server uses voice recognition software (e.g. Google Speech-to-Text API) to convert the voice data into text data.

[0437] Output: Speech recognition software converts the speech "Hello, how is your day?" into text.

[0438] Step 3:

[0439] Data storage

[0440] Input: The server receives the converted text data.

[0441] Specific operation: The server uses the INSERT statement to save the text data "Hello, how are you today?" into the MySQL database.

[0442] Data processing: The server manages the data using a database management system (e.g. MySQL).

[0443] Output: The text data is stored in a database.

[0444] Step 4:

[0445] sentiment analysis

[0446] Input: The server receives the stored text data.

[0447] Specific operation: The server sends text data "Hello, how are you today?" to the emotion engine.

[0448] Data processing: The emotion engine analyzes emotions from the user's tone of voice, speaking style, choice of words, etc.

[0449] Output: The sentiment engine analyzes it as “neutral sentiment” and stores the result in the database.

[0450] Step 5:

[0451] Leveraging Data

[0452] Input: The server receives the sentiment analysis results.

[0453] Specific Action: Provide standard customer support to users whose server has analyzed the user as having a "neutral sentiment".

[0454] Data processing: The server refers to the user's emotional data and takes appropriate action.

[0455] Output: The server sends an alert to customer support to promptly respond.

[0456] Step 6:

[0457] Advanced service provision

[0458] Input: The server receives a bunch of call data.

[0459] How it works: A server feeds a lot of call data into TensorFlow and trains a model to identify common issues.

[0460] Data processing: The server uses AI analysis tools (e.g. TensorFlow) to analyze the data and identify patterns and trends.

[0461] Output: The server uses the trained model to analyze new call data in real time and help improve the service.

[0462] (Application example 1)

[0463] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0464] In conventional telephone services, the contents of calls were often simply treated as voice data, and it was not possible to effectively utilize the contents of the calls or the emotions of the users. In addition, there was a lack of a mechanism to provide personalized content based on the user's emotions, making it difficult to improve the user experience. Furthermore, there was a demand for advanced technology to analyze the contents of calls and provide the most optimal services to users.

[0465] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0466] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the calls as intangible information assets, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, and a means for analyzing the contents of the calls and emotional data and recommending personalized content. This makes it possible to effectively utilize the contents of the calls and the emotional data and provide optimal content to users.

[0467] "SIP communication" is a technology that uses the Session Initiation Protocol to enable multimedia communication such as voice and video.

[0468] "Telephone service" refers to a communication service that provides voice calls.

[0469] "Intangible information assets" are information that is stored as digital data and has value.

[0470] The Telecommunications Business Act is a law that regulates the operation of telecommunications businesses in Japan.

[0471] "Service provision infrastructure" means the technical and operational infrastructure for providing a service.

[0472] "Big Data" refers to large amounts of data and the technology to gain new insights through analysis of that data.

[0473] "AI" stands for artificial intelligence, a technology that allows machines to mimic human intelligence.

[0474] An "advanced service" is a service that has more advanced technology or functions than conventional services.

[0475] "Emotion data" is data that indicates the emotional state of a user.

[0476] "Personalized content" is content that is provided according to the preferences and circumstances of individual users.

[0477] "Content recommendation" is a technology that suggests appropriate content to users.

[0478] A system for carrying out this invention has the following configuration: The server includes a means for providing telephone services using SIP communication, a means for treating the contents of the calls as intangible information assets, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, and a means for analyzing the contents of the calls and emotion data and recommending personalized content.

[0479] Hardware and Software Used

[0480] Hardware: Smartphone, head-mounted display

[0481] Software: Python, SpeechRecognition library, EmotionRecognizer library, ContentRecommender library

[0482] Data processing and calculation

[0483] The server first uses SIP communication to make a voice call. The contents of the call are acquired as voice data and converted to text data using the SpeechRecognition library. Next, the EmotionRecognizer library is used to recognize the user's emotions from the text data. This emotion data is stored in a database as an intangible information asset.

[0484] In addition, the ContentRecommender library is used to recommend the most suitable content to the user based on the recognized emotion data, and the recommended content is displayed on the user's smartphone or head-mounted display.

[0485] Examples

[0486] For example, if a user is feeling stressed during a call with a friend, the server can recognize that emotion and recommend relaxing music or a relaxation video. In this way, the user experience can be improved by providing personalized content based on the user's emotions.

[0487] Examples of prompt statements

[0488] Build an application that analyzes user conversations, recognizes emotions, and recommends the best content based on those emotions. Use the SpeechRecognition library for speech recognition, the EmotionRecognizer library for emotion recognition, and the ContentRecommender library for content recommendation.

[0489] In this way, it is possible to effectively utilize the call content and emotion data to provide optimal content to users.

[0490] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0491] Step 1:

[0492] The server uses SIP communication to make a voice call. When a user starts a call, voice data is acquired in real time. This voice data becomes the input.

[0493] Step 2:

[0494] The server converts the acquired voice data into text data using the SpeechRecognition library. This process outputs the voice data as text data. Specifically, the server reads the voice file, analyzes it with a voice recognition engine, and outputs it as text.

[0495] Step 3:

[0496] The server converts the converted text data into emotion data using the EmotionRecognizer library. This process outputs the text data as emotion data. Specifically, the server analyzes the content of the text and identifies emotions using an emotion recognition model.

[0497] Step 4:

[0498] The server saves the emotion data in a database as an intangible information asset. This process stores the emotion data in the database. Specifically, the server establishes a database connection and saves the emotion data in an appropriate format.

[0499] Step 5:

[0500] The server uses the ContentRecommender library to recommend optimal content to users based on their emotional data. This process inputs emotional data and outputs recommended content. Specifically, the server analyzes the emotional data and selects content that is appropriate for the user's emotional state.

[0501] Step 6:

[0502] The server transmits the recommended content to the user's device (such as a smartphone or a head-mounted display). Through this process, the recommended content is displayed on the user's device. Specifically, the content data is transmitted via a network and displayed on the user's device.

[0503] In this way, it is possible to effectively utilize the contents of the call and the emotion data to provide the user with the most suitable content.

[0504] Example 2

[0505] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0506] In conventional telephone services, it was difficult to efficiently analyze the contents of calls and provide appropriate services to users. In addition, there was a lack of technology to identify specific call patterns and optimize services based on them. This meant that users' needs could not be quickly and accurately responded to, resulting in a decline in the quality of service.

[0507] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0508] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the calls as intangible information assets, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using large-scale data and artificial intelligence, a means for recording the contents of the calls as voice data and transmitting them to the server in real time, a means for converting the voice data into text data, a means for analyzing the text data using natural language processing technology, a means for identifying a specific call pattern based on the analysis results, and a means for providing a service to a user based on the identified call pattern. This makes it possible to quickly provide an appropriate service to a user by efficiently analyzing the contents of the calls and identifying a specific call pattern.

[0509] "SIP communication" is a communication technology for making voice and video calls using the Session Initiation Protocol.

[0510] "Telephone Services" refers to systems and platforms for providing voice calls.

[0511] "Intangible information assets" refers to information and data stored in digital format, including, for example, telephone conversation content and text data.

[0512] The Telecommunications Business Act is a law concerning the operation and regulation of telecommunications businesses, and provides the legal framework for the provision of telecommunications services.

[0513] "Service Provision Platform" refers to the technical and operational infrastructure for providing Services to Users.

[0514] "Big data" refers to very large amounts of data, also known as big data, including call records and user behavior data.

[0515] "Artificial intelligence" refers to technology that enables computer systems to mimic human intelligence to learn and reason.

[0516] "Audio data" refers to data that records audio in digital format.

[0517] "Text data" refers to data in which character information is recorded in digital format.

[0518] "Natural language processing technology" refers to technology that enables computers to understand, interpret, and generate human language.

[0519] "Call patterns" refers to certain tendencies or characteristics regarding calls, such as the content of calls or the time of day calls are made.

[0520] "User" refers to any individual or legal entity that uses the telephone service.

[0521] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0522] This invention is a system that provides telephone services using SIP communication, treats the contents of calls as intangible information assets, and builds a new service provision infrastructure within the scope of the Telecommunications Business Law. Furthermore, it aims to provide advanced services by utilizing large-scale data and artificial intelligence.

[0523] Server Processing

[0524] The server provides a telephone service using SIP communication. This service has a function of recording the contents of a call as voice data and sending it to the server in real time. The recorded voice data is converted into text data using a voice recognition engine (e.g., Google Speech-to-Text API). The server then analyzes the text data using natural language processing technology (e.g., NLTK, spaCy) to identify specific call patterns. Services are provided to the user based on the identified call patterns.

[0525] Terminal handling

[0526] The device starts recording as soon as the user starts a call. The recorded data is sent to the server in real time. An application with recording function is installed on the device, and recording starts automatically when a call starts. This application has the function of uploading the recorded data to the server in real time. The hardware used includes smartphones and tablets.

[0527] User Action

[0528] The user simply uses the phone as they normally would. If certain keywords are used frequently during calls, or if calls are concentrated at certain times of the day, the server will identify the patterns. The user can then receive services provided by the server after the call. For example, if a certain keyword is used frequently, the user can receive customer support suggestions based on that content.

[0529] Examples

[0530] For example, if a user calls customer support and frequently uses keywords such as "return" or "refund," the server can identify this and automatically provide the user with instructions on how to process a return. Also, if a user frequently calls at night, the server can provide support services tailored to that time period.

[0531] Example prompts for generative AI models

[0532] "If a user calls customer support and frequently uses keywords like 'returns' or 'refunds' during the call, explain how you can identify that calling pattern and provide the user with the appropriate service."

[0533] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0534] Step 1:

[0535] The device starts recording as soon as the user starts a call. The recorded data is sent to the server in real time. Specifically, the recording application installed on the device detects the start of a call and automatically starts recording. The input is the call audio, and the output is the recorded audio data.

[0536] Step 2:

[0537] The server receives the voice data sent from the terminal. It converts the received voice data into text data using a voice recognition engine (e.g., Google Speech-to-Text API). The input is voice data, and the output is text data. Specifically, it calls the voice recognition API to convert the voice data into text.

[0538] Step 3:

[0539] The server analyzes the converted text data using natural language processing technology (e.g., NLTK, spaCy). The input is the text data, and the output is the analysis results. Specifically, keywords are extracted from the text data and their frequency is calculated. The time of the call is also recorded.

[0540] Step 4:

[0541] The server identifies specific call patterns based on the analysis results. The input is the analysis results, and the output is the identified call patterns. Specifically, the server identifies calls in which specific keywords are frequently used or calls concentrated in specific time periods.

[0542] Step 5:

[0543] The server provides services to users based on the identified calling patterns. The input is the identified calling patterns, and the output is the services to be provided. Specifically, if a particular keyword is used frequently, the server suggests customer support based on the content of the keyword. In addition, for users who make frequent calls at night, the server provides support services tailored to that time period.

[0544] (Application example 2)

[0545] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0546] In conventional telephone services, it was difficult to analyze call content in real time and identify specific keywords or emotional states. In addition, there was no system that suggested appropriate responses based on the user's emotional state, so improving the user experience was a challenge.

[0547] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0548] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, a means for converting the contents of the call into text data in real time and identifying specific keywords and phrases, a means for analyzing the emotional state of the user and proposing an appropriate response using an emotion engine, and a means for identifying specific call patterns and optimizing services for the user. This enables real-time analysis of the contents of the call and identification of the emotional state, thereby realizing appropriate responses to the user and optimization of services.

[0549] "SIP communication" is a communications technology for making voice and video calls using the Session Initiation Protocol.

[0550] "Telephone service" refers to a system or platform for providing voice calls.

[0551] "Intangible information assets" are information or data stored in digital form that have value despite not having a physical form.

[0552] The Telecommunications Business Act is a law that sets out regulations and rules concerning telecommunications businesses.

[0553] "Service Provisioning Platform" means the technical and operational infrastructure for providing the Services.

[0554] "Big Data" refers to large amounts of data, and is a concept that includes technologies and methods for collecting, storing, and analyzing that data.

[0555] "AI" is an abbreviation for artificial intelligence, a technology that allows machines to imitate human intelligence to learn and make inferences.

[0556] "Real-time" refers to instantaneous processing or reaction without delay.

[0557] "Text data" is data that is stored as character information.

[0558] A "keyword" is a word or phrase that is important for locating or identifying specific information.

[0559] A "phrase" is a short sentence or expression that has meaning and is made up of multiple words.

[0560] An "emotion engine" is a technology that analyzes the user's emotional state and suggests appropriate responses based on the results.

[0561] "Call patterns" refer to certain actions or tendencies that are repeated within a call.

[0562] "Optimization" refers to making something the most effective for a particular purpose.

[0563] The system for carrying out the present invention is configured as follows: The server includes means for providing telephone services using SIP communication, means for treating call contents as intangible information assets, means for building a new service provision infrastructure within the scope of the Telecommunications Business Act, means for providing advanced services using BigData and AI, means for converting call contents into text data in real time and identifying specific keywords and phrases, means for analyzing the emotional state of the user and proposing appropriate responses using an emotion engine, and means for identifying specific call patterns and optimizing services for the user.

[0564] The server converts the call into text data in real time using a speech recognition library (e.g., the speech_recognition library). The converted text data is analyzed to identify specific keywords and phrases. For example, if the keyword "return" is frequently used in customer support calls, the server can identify this keyword and provide the user with details on how to process a return.

[0565] In addition, the server can use an emotion engine (e.g., TextBlob library) to analyze the user's emotional state. If the user expresses anger, the server can use that information to suggest an appropriate response. For example, if the user expresses anger, a notification can be automatically sent to customer support to facilitate a quick response.

[0566] The server also uses BigData and AI technology to identify specific call patterns, for example, call patterns concentrated in certain times of the day, and can optimize services for users based on that information.

[0567] As a concrete example, if the keyword "return" is frequently used in customer support calls, the server can identify the keyword "return" and provide the user with details on how to proceed with a return.

[0568] Examples of prompts to be input to a generative AI model include:

[0569] If a user calls customer support and frequently uses the keyword "returns," what would you suggest?

[0570] This prompt can then be used to have a generative AI model suggest an appropriate response.

[0571] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0572] Step 1:

[0573] The server receives a user's call using SIP communication. The input is voice data and the output is a stream of voice data. The server uses a speech recognition library (e.g., speech_recognition library) to process this voice data in real time.

[0574] Step 2:

[0575] The server converts the voice data into text data using a voice recognition library. The input is voice data, and the output is text data. Through this conversion process, the contents of the call are obtained as text information.

[0576] Step 3:

[0577] The server analyzes the converted text data to identify specific keywords or phrases. The input is the text data, and the output is a list of identified keywords or phrases. For example, it checks whether the text data contains keywords such as "return" or "claim."

[0578] Step 4:

[0579] The server analyzes the text data using an emotion engine (e.g., the TextBlob library) to identify the user's emotional state. The input is the text data, and the output is the emotional state (e.g., happy, anger, sadness, etc.). Through this analysis, the user's emotional state is understood.

[0580] Step 5:

[0581] The server suggests appropriate responses based on the identified keywords and emotional state. The input is a list of keywords and an emotional state, and the output is a list of suggested responses. For example, if a user uses the keyword "return" and indicates an angry emotion, the server automatically notifies customer support to promptly respond.

[0582] Step 6:

[0583] The server uses BigData and AI technology to identify call patterns. The input is past call data, and the output is the identified call patterns. For example, it can identify call patterns that are concentrated in a certain time period and use that information to optimize services for users.

[0584] Step 7:

[0585] The server uses a generative AI model to generate an optimal response for the user. The inputs are the prompt text and the call data, and the output is the generated suggested response. For example, using the following prompt text:

[0586] If a user calls customer support and frequently uses the keyword "returns," what would you suggest?

[0587] Based on this prompt, the generative AI model suggests an appropriate response.

[0588] Example 3

[0589] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0590] In conventional telephone services, it was difficult to effectively utilize the contents of phone calls to optimize the provision of services to users. In addition, there was a lack of means to accurately grasp the user's emotions and needs from the contents of phone calls and provide appropriate services based on that. This meant that improvements in user satisfaction and personalization of services were not fully realized.

[0591] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0592] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using large-scale data and artificial intelligence, a means for collecting the contents of the call in real time and converting the voice into text using voice recognition software, a means for storing the converted text data in a database, a means for extracting specific keywords from the text data using a natural language processing model, a means for recognizing the user's emotions from the contents of the call using an emotion engine, and a means for providing the optimal service to the user based on the extracted keywords and the emotion recognition result. This makes it possible to effectively utilize the contents of the call and provide the optimal service based on the user's needs and emotions.

[0593] "SIP communication" is a communications technology for making voice and video calls using the Session Initiation Protocol.

[0594] "Telephone service" refers to a communication service for providing voice calls.

[0595] "Intangible information assets" are information or data stored in digital form that have value despite not having a physical form.

[0596] The Telecommunications Business Act is a law concerning the operation and regulation of telecommunications businesses.

[0597] "Service provision infrastructure" means the technical and operational infrastructure for providing a service.

[0598] "Large-scale data" refers to a very large amount of data set, which is difficult to process using conventional data processing techniques.

[0599] "Artificial intelligence" is the technology that enables computer systems to mimic human intelligence by learning, reasoning, and self-correcting.

[0600] "Speech recognition software" means software for converting speech into text.

[0601] A "database" is a system for efficiently storing, managing, and searching data.

[0602] A "natural language processing model" is an algorithm or technology for understanding and generating human language.

[0603] "Keywords" are important words or phrases used to represent specific information.

[0604] An "emotion engine" is a technology for recognizing emotions from text and voice.

[0605] "User" means an individual or organization that uses the system or services.

[0606] An "optimal service" is one that best suits the user's needs and circumstances.

[0607] This invention is a system that provides telephone services using SIP communication, treats the contents of calls as intangible information assets, and builds a new service provision infrastructure within the scope of the Telecommunications Business Law. Furthermore, it utilizes large-scale data and artificial intelligence to provide advanced services to users.

[0608] Hardware and Software Used

[0609] The server uses the following hardware and software:

[0610] SIP communication module: Manages voice calls using SIP communication.

[0611] Speech recognition software: Converts voice data into text (e.g., Google Cloud Speech-to-Text API).

[0612] Database system: Stores call content and metadata (e.g. MySQL, PostgreSQL).

[0613] Natural Language Processing models: Extracting keywords from text data (e.g. SpaCy, NLTK).

[0614] Emotion engine: Recognize user emotions from call content (e.g. Microsoft® Azure® Emotion API).

[0615] Artificial intelligence models: Analyze large-scale data and provide optimal services to users (e.g. TensorFlow, PyTorch).

[0616] Data processing and calculation

[0617] The server collects the call contents in real time and converts the voice to text using speech recognition software. The converted text data is stored in a database. It then uses a natural language processing model to extract specific keywords from the text data. It also uses an emotion engine to recognize the user's emotions from the call contents. Finally, it uses an artificial intelligence model to provide optimal services to the user based on the extracted keywords and emotion recognition results.

[0618] Examples

[0619] Example 1: Service provision by keyword extraction

[0620] If a user frequently mentions "I want a new smartphone" during a call, the server will extract that keyword and provide the user with information about the latest smartphones and links to purchase them.

[0621] Example prompt:

[0622] "Please extract frequently used keywords from the following call: Call: 'I've been thinking about going on a trip lately. Any recommendations on where to go?'"

[0623] Example 2: Service provision using emotion recognition

[0624] If the user expresses a happy emotion during the call, the server can use that information to recommend special discount coupons or related products to the user.

[0625] Example prompt:

[0626] "Recognize the user's emotions in the following call: 'Yay! Finally, the project is successful!'"

[0627] In this way, the server effectively utilizes the contents of the call to provide the most suitable service based on the needs and feelings of the user. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0628] Step 1:

[0629] Call data collection

[0630] The server uses SIP communication to monitor the call between the user and the terminal in real time. When the call is started, the server captures the voice data. The input is the voice call between the user and the terminal, and the output is the captured voice data. In concrete terms, the server uses the SIP communication module to monitor the call and collect the voice data.

[0631] Step 2:

[0632] Converting audio data to text

[0633] The server converts the captured voice data into text using voice recognition software (e.g., Google Cloud Speech-to-Text API). The input is the voice data, and the output is the converted text data. In concrete terms, the server calls the voice recognition software and converts the voice data into text.

[0634] Step 3:

[0635] Data storage

[0636] The server stores the converted text data in a database. The input is the text data and call metadata (date and time, call recipient, etc.), and the output is the data stored in the database. In concrete terms, the server uses a database system to store the text data and metadata.

[0637] Step 4:

[0638] Keyword Extraction

[0639] The server analyzes the stored text data and extracts specific keywords using a natural language processing model (e.g., SpaCy, NLTK). The input is the text data, and the output is the extracted keywords. In concrete terms, the server invokes the natural language processing model to extract keywords from the text data.

[0640] Step 5:

[0641] emotion recognition

[0642] The server uses an emotion engine (e.g. Microsoft Azure Emotion API) to recognize the user's emotion from the call content. The input is text data, and the output is the emotion recognition result. In concrete terms, the server calls the emotion engine, analyzes the text data, and recognizes the user's emotion.

[0643] Step 6:

[0644] Optimizing service delivery

[0645] The server uses an artificial intelligence model to provide the optimal service to the user based on the extracted keywords and emotion recognition results. The input is the keywords and emotion recognition results, and the output is the optimal service provided to the user. In specific operations, the server uses the artificial intelligence model to analyze the data, identify the optimal service or product for the user, and send a notification.

[0646] In this way, the server can effectively utilize the contents of the call and provide optimal services based on the user's needs and emotions.

[0647] (Application example 3)

[0648] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0649] In conventional telephone services, it was difficult to optimize the service provided to users by utilizing the contents of the call. In addition, services were not provided that took into account the user's emotional state, so user satisfaction could not be improved. Furthermore, there was a lack of technology to treat the contents of the call as an intangible information asset and to properly analyze and utilize it. In order to solve these issues, a system is needed that can efficiently analyze the contents of the call and provide optimal services based on the user's emotional state.

[0650] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0651] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, a means for converting the contents of the call into text, a means for analyzing the emotions of the text, and a means for recommending products and services based on the results of the emotion analysis. This makes it possible to efficiently analyze the contents of the call and provide optimal services based on the emotional state of the user.

[0652] "SIP communication" is a communications technology that uses the Session Initiation Protocol to make voice and video calls.

[0653] "Telephone service" refers to a communication service for providing voice calls.

[0654] "Intangible information assets" are information or data stored in digital form, and are assets that have value despite not having a physical form.

[0655] The Telecommunications Business Act is a law concerning the operation and regulation of telecommunications businesses.

[0656] "Service provision infrastructure" means the technical and operational infrastructure for providing a service.

[0657] "Big Data" refers to large volumes of diverse data sets, and is a technology that creates new knowledge and value by analyzing them.

[0658] "AI" is an abbreviation for artificial intelligence, a technology that allows machines to imitate human intelligence to learn and make inferences.

[0659] "Means for converting the contents of a voice call into text" refers to technology for converting the contents of a voice call into text information.

[0660] A "means for analyzing text sentiment" is a technique for extracting sentiment from text data and assessing the emotional state.

[0661] "Means for recommending products and services based on the results of sentiment analysis" refers to technology that suggests products and services suitable for users based on the results of sentiment analysis.

[0662] The following system configuration and processing procedure will be described as an embodiment of the present invention.

[0663] System Configuration

[0664] This system uses the following hardware and software:

[0665] Hardware: Smartphones, servers

[0666] Software: Python, SpeechRecognition library, TextBlob library, Requests library

[0667] Processing Steps

[0668] 1. Voice Recognition:

[0669] A user calls customer support using a smartphone. The contents of the call are recorded on the smartphone and saved as an audio file. The server converts the audio file into text using the SpeechRecognition library. Through this process, the contents of the call are obtained as text information.

[0670] 2. Sentiment analysis:

[0671] The server analyzes the sentiment of the text data using the TextBlob library. Sentiment analysis evaluates the emotional state of the text, such as positive, negative, or neutral, and obtains the user's emotional state as numerical data.

[0672] 3. Product recommendation:

[0673] The server recommends appropriate products and services based on the results of sentiment analysis. Using the Requests library, it retrieves product information from external APIs and suggests the best products and services to the user. This process makes it possible to provide personalized services according to the user's emotional state.

[0674] Examples

[0675] For example, if a user calls customer support and says, "I've been busy at work lately and I'm feeling stressed," the server converts the call content into text and performs sentiment analysis. If the result of the sentiment analysis indicates a negative emotion, the server recommends a product that will help the user relax (e.g., an aroma candle or a massage machine).

[0676] Examples of prompt statements

[0677] Please write a Python program that converts the conversation of the user who called customer support into text, analyzes the sentiment of the text, and recommends appropriate products. The libraries used are SpeechRecognition, TextBlob, and Requests.

[0678] The above is a specific embodiment for carrying out the present invention.

[0679] The flow of the specific process in the application example 3 will be described with reference to FIG.

[0680] Step 1:

[0681] A user calls customer support using a smartphone. The contents of the call are recorded on the smartphone and saved as an audio file.

[0682] Input: User's call audio

[0683] Output: Audio file (e.g. .wav format)

[0684] What happens: A user calls customer support and the conversation is saved using the smartphone's recording function.

[0685] Step 2:

[0686] The server converts the audio file into text using the SpeechRecognition library.

[0687] Input: Audio file

[0688] Output: Text data

[0689] How it works: The server receives the audio file and converts it into text using the SpeechRecognition library. Specifically, it reads the audio file and converts it into text using Google's speech recognition API.

[0690] Step 3:

[0691] The server analyzes the sentiment of the text data using the TextBlob library.

[0692] Input: Text data

[0693] Output: Sentiment score (positive, negative, neutral)

[0694] What it does: The server passes the text data to the TextBlob library for sentiment analysis. TextBlob calculates the text's positive, negative, or neutral sentiment score and returns the result.

[0695] Step 4:

[0696] The server recommends appropriate products and services based on the results of the sentiment analysis.

[0697] Input: Sentiment score

[0698] Output: A list of recommended products

[0699] Specific operation: The server sends a request to an external API based on the emotion score to obtain appropriate product information. For example, if the emotion score is negative, the server sends an API request to recommend relaxing products and presents the obtained product information to the user.

[0700] The above are the specific process steps for carrying out the present invention.

[0701] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0702] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0703] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0704] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0705] [Second embodiment]

[0706] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0707] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0708] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0709] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0710] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0711] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0712] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0713] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0714] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0715] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0716] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0717] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0718] "Example 1"

[0719] As an example of the embodiment of the present invention, there is a system that provides telephone services using SIP communication. This system has a function to handle the contents of calls as intangible information assets. Specifically, the contents of calls are digitized, and the data is analyzed and stored to make effective use of the contents of calls. In addition, the system has a function to build a new service provision infrastructure within the scope that does not violate the Telecommunications Business Act. Specifically, new services are provided to users based on data on the contents of calls. Furthermore, the system has a function to provide advanced services that utilize BigData and AI. Specifically, data on the contents of calls is accumulated as BigData, and the data is analyzed using AI to optimize the provision of services to users.

[0720] "Example 2"

[0721] Another embodiment of the present invention is a telephone service using SIP communication with a function for identifying specific call patterns. This system digitizes the contents of calls and identifies specific call patterns by analyzing the data. For example, it identifies call patterns such as calls where a specific keyword is frequently used or calls concentrated in a specific time period, and provides services to users based on the identified call patterns.

[0722] "Example 3"

[0723] Another embodiment of the present invention is a telephone service using SIP communication that has a function of optimizing the provision of services to users based on the content of the calls. This system optimizes the provision of services to users by accumulating data on the content of the calls as BigData and analyzing the data using AI. For example, the system optimizes the provision of services based on the content of the calls by analyzing calls in which a specific keyword is frequently used and providing the user with services related to the keyword.

[0724] The process flow of each embodiment will be described below.

[0725] "Example 1"

[0726] Step 1: Make a telephone call using SIP communication.

[0727] Step 2: The contents of the calls are digitized and treated as an intangible information asset.

[0728] Step 3: Analyze and store the call content data to make effective use of the call content.

[0729] Step 4: Build a new service provision infrastructure within the scope of the Telecommunications Business Act. Step 5: Provide new services to users based on the data on call content.

[0730] Step 6: Accumulate the call content data as BigData and analyze the data using AI.

[0731] Step 7: Optimize the provision of services to users based on the results of AI analysis.

[0732] "Example 2"

[0733] Step 1: Make a telephone call using SIP communication.

[0734] Step 2: Record the call contents and analyze the data.

[0735] Step 3: Identify specific call patterns from the analysis results.

[0736] Step 4: Provide services to the user based on the identified call patterns.

[0737] Step 1: Make a telephone call using SIP communication.

[0738] Step 2: Accumulate the call content data as BigData.

[0739] Step 3: Use AI to analyze the accumulated data.

[0740] Step 4: Optimize the provision of services to users based on the results of AI analysis.

[0741] Example 1

[0742] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0743] Conventional telephone service systems had limited means of effectively utilizing call content, making it difficult to digitize call content for analysis and storage. It was also difficult to build a new service provision infrastructure without violating the Telecommunications Business Act, and services provided to users were not fully optimized. Furthermore, there was a demand for advanced technology to capture call content in real time, store it as voice data, and convert it into text data for analysis.

[0744] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0745] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the calls as intangible information assets, a means for converting the contents of the calls into digital data and analyzing and storing the data, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing new services to users based on the data of the calls, a means for providing advanced services using BigData and AI, a means for capturing the contents of the calls in real time and storing them as voice data, a means for converting the voice data into text data using voice recognition software, a means for storing the converted text data in a database, a means for analyzing the accumulated data with an AI model, and a means for providing services to users based on the analysis results. This makes it possible to effectively utilize the contents of the calls and optimize the provision of services to users.

[0746] "SIP communication" is a communications technology for making voice and video calls using the Session Initiation Protocol.

[0747] "Telephone services" refers to all services for providing voice calls.

[0748] "Intangible information assets" refers to information data that has value but does not have a physical form.

[0749] "Digitization" refers to the conversion of information such as audio and video into digital data.

[0750] "Analysis" refers to the processing of data to extract useful information.

[0751] "Storage" refers to keeping data for an extended period of time.

[0752] The Telecommunications Business Act is a law concerning the operation of telecommunications businesses.

[0753] "Service Provision Infrastructure" means the technical and physical infrastructure for providing the Services.

[0754] "Big Data" refers to large amounts of data and includes the technologies to collect, store, and analyze that data.

[0755] "AI" refers to artificial intelligence and includes technologies such as machine learning and natural language processing.

[0756] "Real-time" refers to processing that occurs immediately, without delay.

[0757] "Capture" refers to obtaining data.

[0758] "Audio data" refers to data that represents audio in digital form.

[0759] "Speech recognition software" means software for converting speech into text.

[0760] "Text data" refers to data that represents character information in digital form.

[0761] "Database" refers to a system for efficiently storing, managing, and retrieving data.

[0762] "AI model" refers to a model for analyzing data using artificial intelligence algorithms.

[0763] "Analysis Results" refers to information obtained through data analysis.

[0764] "Service provision" refers to providing useful functions and information to users.

[0765] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0766] The present invention is a system that provides telephone services using SIP communication and handles call content as intangible information assets. A specific embodiment of this system will be described below.

[0767] 1. Generate the system program

[0768] The server generates a program for providing telephone services using SIP communication. This program has the function of converting the contents of the call into data and analyzing and saving that data.

[0769] 2. Explain the program's processing in natural language

[0770] The server captures the call in real time and stores it as voice data using SIP server software such as Asterisk, which is then converted to text data using voice recognition software (e.g., Google Cloud Speech-to-Text API).

[0771] The terminal receives the converted text data and stores it in a database (e.g., MySQL). The stored data is accumulated using BigData technology (e.g., Hadoop).

[0772] The server analyzes the accumulated data using AI models (for example, natural language processing models using TensorFlow) to provide new services to users, such as customized suggestions based on the content of the call, call summaries, and sentiment analysis.

[0773] 3. Add examples to your description

[0774] As a concrete example, consider a case where a user calls customer support and asks, "How do I return an item?"

[0775] The server captures the contents of the call and stores them as voice data. It then uses voice recognition software to convert the voice data into text data such as "Please tell me how to return the product." The converted text data is stored in a database and accumulated using BigData technology.

[0776] The AI ​​model analyzes this text data and provides users with detailed guidelines on how to return items. It can also predict frequently asked questions based on past call data and provide FAQs in advance.

[0777] Example of a prompt to be input to a generative AI model:

[0778] "When a user calls customer support to ask how to return an item, digitize the conversation, analyze it, and explain what kind of service you would provide to that user."

[0779] In this way, a system is constructed in which the server, terminals, and users work together to effectively utilize call contents and provide new services.

[0780] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0781] Step 1:

[0782] The server captures calls between users in real time using SIP communication. The input is the call audio between user A and user B, and the output is the captured audio data. Specifically, the server uses SIP server software such as Asterisk to obtain the audio data of the call. For example, when user A says, "Please tell me how to return a product," the audio is captured.

[0783] Step 2:

[0784] The server temporarily stores the captured voice data. The input is the voice data captured in step 1, and the output is the stored voice data. The data is stored in storage within the server (e.g., cloud storage). Specifically, the data is stored as an audio file, and the call date and time and caller information are included in the file name.

[0785] Step 3:

[0786] The server converts the stored voice data into text data using voice recognition software. The input is the voice data stored in step 2, and the output is the converted text data. Specifically, the voice data is converted into text using Google Cloud Speech-to-Text API. For example, the voice saying "Please tell me how to return the product" is converted into text "Please tell me how to return the product."

[0787] Step 4:

[0788] The terminal receives the converted text data and stores it in a database. The input is the text data converted in step 3, and the output is the text data stored in the database. Specifically, it is stored in a MySQL database, and when it is stored, the call date and time and caller information are also stored.

[0789] Step 5:

[0790] The server accumulates and manages the saved text data using BigData technology. The input is the text data saved in step 4, and the output is the accumulated data. Specifically, the data is saved in distributed storage using Hadoop and indexed for efficient access.

[0791] Step 6:

[0792] The server analyzes the accumulated data using an AI model. The input is the data accumulated in step 5, and the output is the analysis results. Specifically, the data is analyzed using a natural language processing model that uses TensorFlow, and sentiment analysis, keyword extraction, and trend analysis are performed.

[0793] Step 7:

[0794] The user receives new services based on the analysis results. The input is the analysis results obtained in step 6, and the output is the service provided to the user. Specifically, this includes customized suggestions based on the content of the call, a summary of the call, and the provision of FAQs. For example, if a user says, "Please tell me how to return an item," the server will provide "detailed guidelines on how to return the item." In addition, based on past call data, the server can predict what questions users will frequently ask and provide FAQs in advance.

[0795] (Application example 1)

[0796] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0797] In the conventional telephone service system, the means to effectively utilize the call contents were limited, and the service provision to users was not sufficiently optimized. In addition, the function to analyze the call contents in real time and recommend the content related to the user was lacking, so an improvement of the user experience was required.

[0798] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0799] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the calls as intangible information assets, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, a means for converting the contents of the calls into digital data in real time and analyzing them using AI, and a means for recommending content related to the user based on the analysis results. This makes it possible to provide advanced services based on the contents of the calls and recommend optimal content to the user.

[0800] "SIP communication" is a communications technology for making voice and video calls using the Session Initiation Protocol.

[0801] "Telephone service" means a system or means for providing voice communications.

[0802] "Call Content" refers to voice data and information exchanged through telephone services.

[0803] "Intangible information assets" are information or data stored in digital form that have value despite not having a physical form.

[0804] The Telecommunications Business Act is a law concerning the operation and regulation of telecommunications businesses.

[0805] "Service Provisioning Infrastructure" means the technical and operational infrastructure for providing the Services.

[0806] "Big Data" refers to large volumes of diverse data sets, and is a technology that creates new knowledge and value by analyzing them.

[0807] "AI" is an abbreviation for artificial intelligence, a technology that allows machines to imitate human intelligence to learn and make inferences.

[0808] "Digitization in real time" refers to instantly converting the contents of a call into digital data.

[0809] "Analysis" is the act of processing data to extract useful information.

[0810] "Content" refers collectively to information and entertainment provided to Users.

[0811] A "recommendation" is the act of presenting the best option based on certain criteria.

[0812] The system for carrying out this invention has the following configuration: The server includes means for providing telephone services using SIP communication, means for treating the contents of the calls as intangible information assets, means for building a new service provision infrastructure within the scope of the Telecommunications Business Act, means for providing advanced services using BigData and AI, means for converting the contents of the calls into digital data in real time and analyzing them using AI, and means for recommending content relevant to the user based on the analysis results.

[0813] Specifically, when a user makes a call using a smartphone, the contents of the call are digitized in real time. This data is sent to a server and analyzed using an AI model. For the analysis, a voice recognition library (e.g., the speech_recognition library) is used to convert the contents of the call into text. The converted text is sent to the API endpoint of the AI ​​model for detailed analysis.

[0814] The server recommends content relevant to the user based on the analysis results. For example, if a user says, "I want to talk about recent movies," the AI ​​model analyzes this text and recommends the latest movie information and reviews. This recommendation is notified to the user's smartphone.

[0815] The hardware used is the smartphone's microphone and speaker, and the server's processing power. The software uses the speech_recognition library and the API of the AI ​​model.

[0816] As a concrete example, if a user says, "I want to talk about recent movies," the AI ​​model analyzes this text and recommends the latest movie information and reviews. The following is an example of a prompt sentence to be input to the generative AI model.

[0817] "If a user says, 'I want to talk about recent movies,' recommend the latest movie information and reviews."

[0818] In this way, advanced services based on the content of the call and optimal content recommendations for the user can be realized.

[0819] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0820] Step 1:

[0821] A user starts a call using a smartphone. The smartphone's microphone captures the voice and digitizes the call content in real time. The input is the user's voice, and the output is digital voice data.

[0822] Step 2:

[0823] The smartphone transmits digital voice data to the server. The server converts the voice data into text using a voice recognition library (e.g., speech_recognition library). The input is the digital voice data and the output is the text data.

[0824] Step 3:

[0825] The server sends the converted text data to the API endpoint of the AI ​​model. The AI ​​model analyzes the text data and extracts the user's intent and interests. The input is the text data, and the output is the analysis result.

[0826] Step 4:

[0827] The server recommends content relevant to the user based on the analysis results. For example, if a user says, "I want to talk about recent movies," the AI ​​model recommends the latest movie information and reviews. The input is the analysis results, and the output is the recommended content.

[0828] Step 5:

[0829] The server notifies the user of the recommended content to the smartphone. The smartphone receives the notification and displays it to the user. The input is the recommended content, and the output is the notification to the user.

[0830] In this way, advanced services based on the contents of the call and optimal content recommendations to the user are realized.

[0831] Example 2

[0832] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0833] In conventional telephone services using SIP communications, it was difficult to efficiently analyze call content and identify specific call patterns. In addition, there was a lack of means to digitize call content in real time and use that data to optimize services for users. This made it difficult to respond quickly and accurately to user needs, resulting in a decline in the quality of service.

[0834] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0835] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using large amounts of data and artificial intelligence, a means for capturing the contents of the call in real time and converting the voice data into text data, a means for storing the converted text data, a means for analyzing the stored text data and extracting patterns such as the frequency of appearance of specific keywords and the time period of the call, and a means for transmitting the analysis results to a user terminal and visually displaying them. This makes it possible to efficiently analyze the contents of the call and identify specific call patterns, thereby optimizing the provision of services to users.

[0836] "SIP communication" is a technology that uses the Session Initiation Protocol to manage multimedia communications such as voice and video.

[0837] "Telephone service" refers to a communication service for providing voice calls.

[0838] "Intangible information assets" are information or data stored in digital form, and are assets that have value despite not having a physical form.

[0839] The Telecommunications Business Act is a law concerning the operation and regulation of telecommunications businesses.

[0840] "Service Provisioning Platform" means the technical and operational infrastructure for providing the Services.

[0841] "Massive data", also known as big data, refers to a very large data set, the amount of data that cannot be handled by conventional data processing techniques.

[0842] "Artificial intelligence" is a technology that enables machines to mimic human intelligence and learn, reason, and solve problems.

[0843] "Real-time" refers to data processing and communication occurring immediately and without delay.

[0844] "Audio data" refers to data in which audio is recorded in digital format.

[0845] "Text data" refers to data in which character information is recorded in digital format.

[0846] A "database" is a system for efficiently storing, searching, and managing data.

[0847] "Natural language processing" is a technology that enables computers to understand and generate human language.

[0848] A "keyword" is a word or phrase that is important when searching for or analyzing specific information.

[0849] A "user terminal" is a device such as a computer or smartphone that is directly operated by a user.

[0850] "Visual display" refers to presenting data or information to a user in an easily viewable form, such as in a graph or chart.

[0851] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0852] This invention is a system for telephone services using SIP communication that digitizes call contents in real time, analyzes the data to identify specific call patterns, and optimizes services for users. A specific embodiment of this system is described below.

[0853] Hardware and Software Used

[0854] The server uses SIP server software (e.g., Asterisk or FreeSWITCH) to capture calls made through SIP communication in real time, and uses a speech recognition API (e.g., Google Cloud Speech-to-Text API) to convert the captured voice data into text data, which is then stored in a relational database (e.g., MySQL or PostgreSQL).

[0855] The server uses Python natural language processing libraries (e.g., NLTK and spaCy) to analyze the stored text data, extracting patterns such as the frequency of occurrence of certain keywords or the time of day of calls.

[0856] The terminal uses JavaScript libraries (e.g. Chart.js and D3.js) to receive the analysis results sent from the server and visually display them to the user.

[0857] Examples

[0858] Consider the case where the user is an administrator of a customer support center. The user wants to identify calls that frequently use certain keywords (e.g., "complaint" or "returned goods"). The server automatically extracts calls that contain these keywords and sends the analysis results to the terminal. The terminal displays a graph to the user showing which time periods these calls are concentrated.

[0859] Example of a prompt to be input to a generative AI model:

[0860] "Analyze call data from your customer support center and identify calls that frequently use certain keywords (e.g., 'claims' or 'returns'). Also, create a graph to show which times of day these calls are concentrated."

[0861] In this way, users can easily keep track of calling patterns and take appropriate measures.

[0862] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0863] Step 1:

[0864] The server captures calls made through SIP communication in real time. Specifically, it uses SIP server software (e.g., Asterisk or FreeSWITCH) to obtain the voice data of calls. The input is the call data from SIP communication, and the output is the captured voice data.

[0865] Step 2:

[0866] The server converts the captured voice data into text data. This conversion is done using a voice recognition API (e.g., Google Cloud Speech-to-Text API). The server sends the voice data to the API and receives the returned text data. The input is the captured voice data, and the output is the converted text data.

[0867] Step 3:

[0868] The server stores the converted text data in a database. Specifically, a relational database (e.g., MySQL or PostgreSQL) is used. The text data is stored together with the call timestamp and caller information. The input is the converted text data, and the output is the text data stored in the database.

[0869] Step 4:

[0870] The server analyzes the stored text data. It uses Python natural language processing libraries (e.g., NLTK and spaCy) to extract patterns such as the frequency of occurrence of specific keywords or the time of day of calls. The input is the text data stored in the database, and the output is the analysis results.

[0871] Step 5:

[0872] The server sends the analysis results to the user terminal. Specifically, it uses a RESTful API to send data in JSON format. The input is the analysis results, and the output is the data sent to the user terminal.

[0873] Step 6:

[0874] The terminal visually displays the received analysis results to the user. It uses JavaScript libraries (e.g. Chart.js and D3.js) to display graphs and lists of calls where specific keywords are frequently used or where calls are concentrated in a specific time period. The input is the analysis results sent from the server, and the output is the visually displayed data.

[0875] (Application example 2)

[0876] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0877] With conventional telephone services, it was difficult to analyze call content in real time and identify specific keywords or patterns. In addition, there was a lack of functionality to detect dangerous call patterns and send warnings to users, so sufficient measures could not be taken against risks such as fraud and blackmail. Furthermore, there was a lack of functionality to analyze call history and provide reports on dangerous call patterns, making it difficult to ensure the safety of users.

[0878] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0879] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, a means for converting the contents of the call into data in real time and identifying specific keywords or patterns, a means for sending a warning to the user based on the identified keywords or patterns, and a means for analyzing the call history and providing a report of dangerous call patterns. This makes it possible to detect specific keywords or patterns in real time during a call and immediately send a warning to the user. In addition, by analyzing the call history and providing a report of dangerous call patterns, the safety of the user can be ensured.

[0880] "SIP communication" is a communication method that uses the Session Initiation Protocol to make voice and video calls.

[0881] "Telephone service" refers to a communication service for providing voice calls.

[0882] An "intangible information asset" is information stored as digital data; it is an asset that has value despite not having a physical form.

[0883] The Telecommunications Business Act is a law concerning the operation of telecommunications businesses, and sets out regulations concerning the provision of telecommunications services.

[0884] "Service provision infrastructure" means the technical and operational infrastructure for providing a service.

[0885] "Big Data" refers to large amounts of data, and is a concept that includes technologies and methods for collecting, storing, and analyzing that data.

[0886] "AI" is an abbreviation for artificial intelligence, a technology that allows machines to imitate human intelligence to learn and make inferences.

[0887] An "advanced service" is a service that has more advanced functions and performance than conventional services.

[0888] "Real-time" refers to immediate processing or response without delay.

[0889] "Digitization" refers to converting information into digital data.

[0890] A "keyword" is a word that has a particular meaning or importance.

[0891] A "pattern" is a collection of data that has a particular regularity or tendency.

[0892] A "warning" is a notification that warns of a possible danger or problem.

[0893] A "call history" is a record of past calls.

[0894] A "report" is a report that summarizes analysis results and information.

[0895] A system for carrying out this invention is configured as follows: The server includes means for providing telephone services using SIP communication, means for treating call contents as intangible information assets, means for building a new service provision infrastructure within the scope of the Telecommunications Business Act, means for providing advanced services using BigData and AI, means for converting call contents into data in real time and identifying specific keywords or patterns, means for sending a warning to the user based on the identified keywords or patterns, and means for analyzing call history and providing a report on risky call patterns.

[0896] Program processing explanation

[0897] Hardware used

[0898] Smartphone (microphone, speaker)

[0899] Software used

[0900] Python

[0901] SpeechRecognition Library

[0902] Google Speech Recognition API

[0903] Data processing and calculation

[0904] The server first captures the call audio through the smartphone's microphone. The captured audio data is converted to text data using the SpeechRecognition library. The converted text data is then analyzed to identify specific keywords and patterns. This analysis includes matching keywords using regular expressions.

[0905] Based on the identified keywords and patterns, the server sends a warning to the user in real time via the smartphone's notification function, and the call history is stored in a database for later analysis. The analysis results are then provided to the user as a report of risky calling patterns.

[0906] Examples

[0907] For example, if the keyword "scam" is detected during a user's call, the server will immediately display a warning and record the call content and the detected keyword. This warning will be sent to the user through the smartphone's notification function. The call history will also be stored in a database for later analysis. The analysis results will be provided to the user as a report of risky calling patterns.

[0908] Examples of prompt statements

[0909] "Write a Python program that will alert the user and record the call if certain keywords (e.g. scam, blackmail) are detected during the call."

[0910] In this way, it is possible to detect certain keywords or patterns during calls in real time and send immediate warnings to users, and also analyze call history to provide reports on risky calling patterns to ensure user safety.

[0911] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0912] Step 1:

[0913] The server captures the call audio through the smartphone's microphone. The input is the call audio data, and the output is an audio data file. This audio data is stored for analysis in subsequent processing steps.

[0914] Step 2:

[0915] The server converts the acquired voice data into text data using the SpeechRecognition library. The input is a voice data file, and the output is text data. This conversion process converts the voice data into an analyzable text format.

[0916] Step 3:

[0917] The server analyzes the converted text data to identify specific keywords or patterns. The input is the text data, and the output is a list of identified keywords or patterns. This analysis includes matching keywords using regular expressions.

[0918] Step 4:

[0919] The server sends alerts to users based on the identified keywords and patterns. The input is a list of identified keywords and patterns, and the output is an alert notification to the user. This notification is done in real time through the smartphone notification feature.

[0920] Step 5:

[0921] The server stores the call history in a database. The input is the text data of the call and a list of identified keywords or patterns, and the output is the call history stored in the database. This storage process accumulates data for later analysis of the call history.

[0922] Step 6:

[0923] The server periodically analyzes the stored call history and provides a report on risky calling patterns. The input is the call history stored in the database, and the output is a report of the analysis results. This report is provided to the user and is used to understand risky calling patterns.

[0924] Example 3

[0925] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0926] With conventional telephone services, it was difficult to efficiently analyze the contents of phone calls and optimize the services provided to users. In addition, there was a lack of appropriate means to treat the contents of phone calls as intangible information assets, and there was a need to build a new service provision infrastructure that did not violate the Telecommunications Business Act. Furthermore, no specific method had been established for providing advanced services that utilize large amounts of data and artificial intelligence.

[0927] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0928] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using large amounts of data and artificial intelligence, a means for capturing the contents of the call in real time and converting the voice data into text data, a means for storing and analyzing the converted text data, and a means for optimizing the provision of services to users based on the analysis results. This makes it possible to efficiently analyze the contents of the call and optimize the provision of services to users.

[0929] "SIP communication" is a communications technology for making voice and video calls using the Session Initiation Protocol.

[0930] "Telephone service" means a system or means for providing voice communications.

[0931] "Intangible information assets" are information or data stored in digital form, and are assets that have value despite not having a physical form.

[0932] The Telecommunications Business Act is a law concerning the operation and regulation of telecommunications businesses.

[0933] "Service Provisioning Platform" means the technical and operational infrastructure for providing the Services.

[0934] "Massive data" refers to very large data sets, usually referred to as big data.

[0935] "Artificial intelligence" is a technology that allows computers to imitate human intelligence to learn and make inferences.

[0936] "Real-time" refers to immediate processing or response without delay.

[0937] "Audio data" refers to data in which audio is recorded in digital format.

[0938] "Text data" refers to data in which character information is recorded in digital format.

[0939] "Storage" refers to storing data and making it available for later use.

[0940] "Analysis" is the process of examining data in detail to find meaning and patterns.

[0941] "Optimization" refers to finding the most efficient way to achieve a particular goal.

[0942] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0943] This invention is a system that provides telephone services using SIP communication, treats the contents of calls as intangible information assets, and builds a new service provision infrastructure within the scope of the Telecommunications Business Law. Furthermore, it provides advanced services by utilizing large amounts of data and artificial intelligence, captures the contents of calls in real time, converts the voice data into text data, and accumulates and analyzes the converted text data, optimizing the provision of services to users.

[0944] Hardware and Software Used

[0945] The server uses the following hardware and software:

[0946] SIP communication module: A communication technology for making voice calls using SIP communication.

[0947] Speech recognition API: An API for converting voice data into text data. Specifically, we use the Google Cloud Speech-to-Text API.

[0948] Data storage system: A distributed file system for efficiently storing and managing large amounts of text data. Specifically, it uses a Hadoop cluster.

[0949] Generative AI model: An AI model for analyzing text data and extracting frequently used keywords and phrases. Specifically, we use OpenAI's GPT-4.

[0950] Program processing explanation

[0951] The server captures the conversation between the user and the device in real time. When the conversation starts, the server receives the voice data in streaming format and converts it into text data using the Google Cloud Speech-to-Text API. The converted text data is stored in a Hadoop cluster.

[0952] Next, the server analyzes the accumulated text data using a generative AI model (GPT-4). The AI ​​model extracts frequently used keywords and phrases from the text data and identifies the user's needs and problems. Based on the analysis results, the server optimizes the service provision to the user. For example, if a certain keyword is frequently used, it automatically suggests services related to that keyword.

[0953] Examples

[0954] As a concrete example, consider the case where a user calls customer support and says, "The product was broken and I want to return it. What should I do?" The contents of this call are captured in real time and converted into text data using the Google Cloud Speech-to-Text API. The data is then stored in a Hadoop cluster and analyzed by a generative AI model (GPT-4).

[0955] As a result of the analysis, it is found that keywords such as "broken" and "return" are frequently used. Based on this information, the server can provide the user with information on the return procedure and a service to exchange the broken product.

[0956] Examples of prompt statements

[0957] Example prompt:

[0958] Please enter the user's call details in the following format:

[0959] "The item was broken and I want to return it. What should I do?"

[0960] By inputting this prompt sentence into the generative AI model, the AI ​​proposes an appropriate service provision method. For example, it instructs the server to take specific actions such as "providing detailed information on the return procedure" or "arranging for a replacement product." The flow of the specific process in the third embodiment will be explained with reference to FIG. 15.

[0961] Program processing flow

[0962] Step 1: Capture the call

[0963] The server captures the contents of the call between the user and the terminal in real time. When the call starts, the server receives the voice data in streaming format through SIP communication. The input is the user's voice data, and the output is the voice data sent to the server.

[0964] Specific behavior:

[0965] A user calls customer support.

[0966] The server captures the contents of the call using SIP communication.

[0967] Step 2: Convert audio to text

[0968] The server converts the received voice data into text data using the Google Cloud Speech-to-Text API. The input is the voice data sent to the server, and the output is the text data.

[0969] Specific behavior:

[0970] The server sends the audio data to the Google Cloud Speech-to-Text API.

[0971] The API converts the audio data into text data.

[0972] The server receives the converted text data.

[0973] Step 3: Accumulating text data

[0974] The server accumulates the converted text data in the Hadoop cluster. The input is the text data, and the output is the text data stored in the Hadoop cluster.

[0975] Specific behavior:

[0976] The server sends the text data to the Hadoop cluster.

[0977] The Hadoop cluster stores the text data.

[0978] Step 4: Analyze the text data

[0979] The server analyzes the accumulated text data using a generative AI model (GPT-4). The input is the text data stored in the Hadoop cluster, and the output is keywords and phrases as the analysis results.

[0980] Specific behavior:

[0981] The server retrieves the text data from the Hadoop cluster.

[0982] The server inputs the text data into the generative AI model.

[0983] The AI ​​model analyzes the text data and extracts frequently used keywords and phrases.

[0984] The server receives the analysis results.

[0985] Step 5: Optimize service delivery

[0986] The server optimizes the service provided to the user based on the analysis results of the AI ​​model. The input is keywords or phrases as the analysis results, and the output is the optimized service provided.

[0987] Specific behavior:

[0988] The server identifies the user's needs based on the analysis results.

[0989] The server suggests appropriate services based on the identified needs.

[0990] For example, if keywords such as "broken" and "return" are frequently used, the server will provide the user with guidance on return procedures and a service for replacing broken products.

[0991] (Application example 3)

[0992] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0993] With conventional telephone services, it was difficult to effectively utilize the contents of phone calls to optimize the provision of services to users. In addition, it was not possible to propose content in real time based on the contents of phone calls, making it difficult to provide services that meet the needs of users.

[0994] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means. In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope not infringing the Telecommunications Business Act, a means for providing advanced services using BigData and AI, and a means for analyzing the contents of the call in real time and automatically proposing content based on the contents after the call ends. This makes it possible to propose content in real time based on the contents of the call.

[0995] "SIP communication" is a communication protocol for making voice and video calls using the Internet Protocol.

[0996] "Telephone service" refers to a communication service for providing voice calls.

[0997] "Call content" refers to the audio information spoken during a telephone call.

[0998] "Intangible information assets" refers to information or data stored in digital form.

[0999] The Telecommunications Business Act is a law concerning the operation of telecommunications businesses.

[1000] "Service Delivery Infrastructure" means the technical and physical infrastructure for delivering a service.

[1001] "Big Data" refers to large amounts of data and the technology to extract useful information by analyzing it.

[1002] "AI" stands for artificial intelligence, a technology that allows machines to imitate human intelligence and learn and reason.

[1003] An "advanced service" is a service that provides higher added value than conventional services.

[1004] "Real-time analysis" means processing data instantly and providing results immediately.

[1005] "Content" means the information and entertainment provided to Users.

[1006] "Automatic suggestion" means that the system provides appropriate information or services without user interaction.

[1007] The system for carrying out this invention is configured as follows: The server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, and a means for analyzing the contents of the call in real time and automatically proposing content based on the contents after the call is ended.

[1008] First, when a user starts a call, the terminal makes the call using SIP communication. Voice data during the call is sent to the server in real time, and the server stores this voice data as an intangible information asset. Next, the server converts the voice data into text using voice recognition software (for example, Google's voice recognition API). This text data is stored as BigData.

[1009] The server analyzes the stored text data using AI (e.g., natural language processing models). As a result of the analysis, certain keywords and call patterns are extracted. Based on this, the server generates content to optimize the provision of services to users. For example, if a user talks about movies during a call, the server will suggest reviews, trailers, viewing links, etc. related to the movie.

[1010] As a concrete example, if a user talks about "recent movies" during a call, the server can extract keywords such as "movies," "reviews," and "trailers," and suggest related content to the user based on those keywords.

[1011] Here is an example of a prompt to be input to the generative AI model:

[1012] "When a user is on a call and they're talking about 'recent movies,' extract keywords to suggest related content."

[1013] In this way, real-time content suggestions based on the contents of the call are possible, and services that meet the needs of the user are provided.

[1014] The flow of the specific process in the application example 3 will be described with reference to FIG.

[1015] Step 1:

[1016] When a user initiates a call, the terminal uses SIP communication to make the call. The input is the user's voice, and the output is the voice data sent to the server in real time. The terminal uses a microphone to capture the voice and sends it to the server via the SIP protocol.

[1017] Step 2:

[1018] The server stores the received voice data as an intangible information asset. The input is the voice data sent from the terminal, and the output is the voice data stored on the server. The server uses a database to store the voice data.

[1019] Step 3:

[1020] The server uses speech recognition software (e.g. Google's speech recognition API) to convert the voice data to text. The input is the stored voice data, and the output is the text data. The server sends the voice data to the speech recognition API and receives the text data in return.

[1021] Step 4:

[1022] The server stores the text data as BigData. The input is the text data obtained from the speech recognition software, and the output is the text data stored on the server. The server uses a database to store the text data.

[1023] Step 5:

[1024] The server analyzes the stored text data using AI (e.g., a natural language processing model). The input is the stored text data, and the output is the extracted keywords and call patterns. The server inputs the text data into the natural language processing model and extracts keywords and call patterns.

[1025] Step 6:

[1026] The server generates content to optimize service provision to users based on the extracted keywords and call patterns. The input is the extracted keywords and call patterns, and the output is the generated content. The server uses a generative AI model to generate relevant content.

[1027] Step 7:

[1028] The server proposes the generated content to the user. The input is the generated content and the output is the content provided to the user. The server transmits the content to the user's terminal and the user receives the content through the terminal.

[1029] In this way, real-time content suggestions based on the contents of the call are possible, and services that meet the needs of the user are provided.

[1030] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1031] "Example 1"

[1032] One embodiment of the present invention is a system that combines an emotion engine with a telephone service using SIP communication. This system recognizes emotions from the user's voice during a call and treats the information as an intangible information asset. Specifically, emotions are analyzed from the user's tone of voice, speaking style, choice of words, etc., and the results are stored in a database. This emotion data is used to increase user satisfaction and improve services.

[1033] "Example 2"

[1034] Another embodiment of the present invention is a system that uses an emotion engine to analyze the emotional state of a user and uses the results as reference information for providing services. Specifically, when a user expresses an emotion such as joy, anger, or sadness, a service is provided based on that information. For example, when a user expresses anger, a response such as contacting customer support is made based on that information.

[1035] "Example 3"

[1036] Furthermore, as another embodiment of the present invention, there is a system that optimizes the provision of services to a user based on the emotion recognition result by the emotion engine. Specifically, the system provides optimal services according to the user's emotional state. For example, if the user expresses an emotion of joy, the system responds by recommending related services or products based on that information.

[1037] The process flow of each embodiment will be described below.

[1038] "Example 1"

[1039] Step 1: A user initiates a telephone call using SIP communication.

[1040] Step 2: The emotion engine recognizes emotions from the user's voice during the call.

[1041] Step 3: The emotion information recognized by the emotion engine is stored in a database.

[1042] "Example 2"

[1043] Step 1: A user initiates a telephone call using SIP communication.

[1044] Step 2: The emotion engine analyzes the user's emotional state during the call.

[1045] Step 3: Use the analysis results as reference information for service provision.

[1046] "Example 3"

[1047] Step 1: A user initiates a telephone call using SIP communication.

[1048] Step 2: The emotion engine analyzes the user's emotional state during the call.

[1049] Step 3: Optimize service provision to users based on the analysis results.

[1050] Example 1

[1051] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1052] Conventional telephone service systems have limited means of effectively utilizing call content, making it difficult to digitize call content for analysis and storage. It is also difficult to recognize user emotions in real time and provide services based on them. Another issue is building a new service provision infrastructure without violating the Telecommunications Business Act. To solve these issues, a system is needed that treats call content as an intangible information asset and provides advanced services that utilize BigData and AI.

[1053] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1054] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for converting the contents of the call into digital data and analyzing and storing the data, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, a means for recognizing emotions from the voice of a user during a call and treating the information as an intangible information asset, and a means for analyzing emotions from the tone of the user's voice, speaking style, choice of words, etc., and storing the results in a database. This makes it possible to effectively utilize the contents of the call and optimize service provision by recognizing the user's emotions in real time.

[1055] "SIP communication" is a communications technology for establishing and managing voice and video calls using the Session Initiation Protocol.

[1056] "Telephone service" refers to a communication service for providing voice calls.

[1057] "Intangible information assets" are information data that have value even though they do not have a physical form.

[1058] "Digitization" is the process of converting information such as audio and video into digital data.

[1059] "Analysis" is the process of examining data in detail to find meaning and patterns.

[1060] "Preservation" is the process of recording and storing data for later use.

[1061] The Telecommunications Business Act is a law concerning the operation and regulation of telecommunications businesses.

[1062] "Service provision infrastructure" means the technical and operational infrastructure for providing a service.

[1063] "Big Data" refers to large amounts of data, as well as the technologies and methods for collecting, storing, and analyzing that data.

[1064] "AI" is an abbreviation for artificial intelligence, a technology that allows machines to imitate human intelligence to learn and make inferences.

[1065] An "emotion engine" is software that analyzes and recognizes emotions from voice and text.

[1066] "Tone" refers to characteristics of voice such as pitch, intensity, and sound quality.

[1067] A "database" is a system for systematically storing and managing data.

[1068] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1069] This invention is a system that provides telephone services using SIP communications and treats the contents of phone calls as intangible information assets. This system digitizes the contents of phone calls, analyzes and stores the data, and makes effective use of the contents of phone calls. It also has the function of building a new service provision infrastructure within the scope of the Telecommunications Business Act. It also has the function of providing advanced services that utilize BigData and AI.

[1070] Hardware and Software Used

[1071] Hardware: Servers, terminals

[1072] Software: SIP communication protocol, database management systems (e.g. MySQL), AI analysis tools (e.g. TensorFlow), emotion engines

[1073] Specific operation of the system

[1074] 1. Start a call:

[1075] A user uses a terminal to make a call via SIP communication.

[1076] The terminal sends a call request to the server using the SIP protocol.

[1077] The server accepts the call request and establishes the call.

[1078] Specific behavior:

[1079] The user enters a phone number into the dial pad of the smartphone and presses the call button.

[1080] The terminal sends a SIP INVITE message to the server.

[1081] The server responds with a SIP 200 OK message and the call is established.

[1082] 2. Digitization of call contents:

[1083] The server captures the contents of the call as audio data in real time.

[1084] The server uses speech recognition software (e.g. Google Speech-to-Text API) to convert the voice data into text data.

[1085] Specific behavior:

[1086] The server captures the audio stream of the call and sends it to the speech recognition software.

[1087] Speech recognition software converts the speech "Hello, how can I help you today?" into text.

[1088] 3. Data storage:

[1089] The server stores the converted text data in a database.

[1090] The server manages the data using a database management system (e.g. MySQL).

[1091] Specific behavior:

[1092] The server uses the INSERT statement to save the text data "Hello, how are you today?" into the MySQL database.

[1093] 4. Sentiment analysis:

[1094] The server uses an emotion engine to analyze emotions based on the user's tone of voice, speaking style, choice of words, etc.

[1095] The server stores the analysis results in a database.

[1096] Specific behavior:

[1097] The server sends the text data "Hello, how are you today?" to the emotion engine.

[1098] The emotion engine analyzes it as a "neutral emotion" and stores the result in a database.

[1099] 5. Leverage data:

[1100] The server provides new services to users based on the analysis results.

[1101] The server refers to the user's emotional data and takes appropriate action.

[1102] Specific behavior:

[1103] The server will provide standard customer support to users whose sentiment is analyzed as "neutral."

[1104] If a server is analyzed as "unhappy," an alert is sent to customer support to promptly respond.

[1105] 6. Advanced service provision:

[1106] The server uses BigData and AI to accumulate and analyze data on call content to optimize service provision.

[1107] The server uses AI analytics tools (e.g. TensorFlow) to analyze the data and identify patterns and trends.

[1108] Specific behavior:

[1109] The server feeds a large amount of call data into TensorFlow and trains a model to identify common issues.

[1110] The server uses the trained model to analyze new call data in real time to help improve the service.

[1111] Examples of prompt statements

[1112] Examples of prompt sentences to be input to a generative AI model include the following:

[1113] "Create a program that saves the contents of a user's phone call as text data and analyzes the emotions using an emotion engine."

[1114] "Please design a system that will optimize the services provided to users by accumulating call content as BigData and analyzing it using AI."

[1115] In this way, a system is realized that provides advanced telephone services in cooperation with users, servers, and terminals.

[1116] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1117] Step 1:

[1118] Starting a call

[1119] Input: A user uses a terminal to enter a phone number and presses call.

[1120] Specific operation: The user enters a phone number on the dial pad of the smartphone and presses the call button. The device sends a SIP INVITE message to the server.

[1121] Data processing: The terminal sends a call request to the server using the SIP protocol.

[1122] Output: The server returns a SIP 200 OK message and the call is established.

[1123] Step 2:

[1124] Digitization of call contents

[1125] Input: The server receives the audio data of the call.

[1126] What happens: The server captures the audio stream of the call and sends it to speech recognition software.

[1127] Data processing: The server uses voice recognition software (e.g. Google Speech-to-Text API) to convert the voice data into text data.

[1128] Output: Speech recognition software converts the speech "Hello, how is your day?" into text.

[1129] Step 3:

[1130] Data storage

[1131] Input: The server receives the converted text data.

[1132] Specific operation: The server uses the INSERT statement to save the text data "Hello, how are you today?" into the MySQL database.

[1133] Data processing: The server manages the data using a database management system (e.g. MySQL).

[1134] Output: The text data is stored in a database.

[1135] Step 4:

[1136] sentiment analysis

[1137] Input: The server receives the stored text data.

[1138] Specific operation: The server sends text data "Hello, how are you today?" to the emotion engine.

[1139] Data processing: The emotion engine analyzes emotions from the user's tone of voice, speaking style, choice of words, etc.

[1140] Output: The sentiment engine analyzes it as “neutral sentiment” and stores the result in the database.

[1141] Step 5:

[1142] Leveraging Data

[1143] Input: The server receives the sentiment analysis results.

[1144] Specific Action: Provide standard customer support to users whose server has analyzed the user as having a "neutral sentiment".

[1145] Data processing: The server refers to the user's emotional data and takes appropriate action.

[1146] Output: The server sends an alert to customer support to promptly respond.

[1147] Step 6:

[1148] Advanced service provision

[1149] Input: The server receives a bunch of call data.

[1150] How it works: A server feeds a lot of call data into TensorFlow and trains a model to identify common issues.

[1151] Data processing: The server uses AI analysis tools (e.g. TensorFlow) to analyze the data and identify patterns and trends.

[1152] Output: The server uses the trained model to analyze new call data in real time and help improve the service.

[1153] (Application example 1)

[1154] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1155] In conventional telephone services, the contents of calls were often simply treated as voice data, and it was not possible to effectively utilize the contents of the calls or the emotions of the users. In addition, there was a lack of a mechanism to provide personalized content based on the user's emotions, making it difficult to improve the user experience. Furthermore, there was a demand for advanced technology to analyze the contents of calls and provide the most optimal services to users.

[1156] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1157] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the calls as intangible information assets, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, and a means for analyzing the contents of the calls and emotional data and recommending personalized content. This makes it possible to effectively utilize the contents of the calls and the emotional data and provide optimal content to users.

[1158] "SIP communication" is a technology that uses the Session Initiation Protocol to enable multimedia communication such as voice and video.

[1159] "Telephone service" refers to a communication service that provides voice calls.

[1160] "Intangible information assets" are information that is stored as digital data and has value.

[1161] The Telecommunications Business Act is a law that regulates the operation of telecommunications businesses in Japan.

[1162] "Service provision infrastructure" means the technical and operational infrastructure for providing a service.

[1163] "Big Data" refers to large amounts of data and the technology to gain new insights through analysis of that data.

[1164] "AI" stands for artificial intelligence, a technology that allows machines to mimic human intelligence.

[1165] An "advanced service" is a service that has more advanced technology or functions than conventional services.

[1166] "Emotion data" is data that indicates the emotional state of a user.

[1167] "Personalized content" is content that is provided according to the preferences and circumstances of individual users.

[1168] "Content recommendation" is a technology that suggests appropriate content to users.

[1169] A system for carrying out this invention has the following configuration: The server includes a means for providing telephone services using SIP communication, a means for treating the contents of the calls as intangible information assets, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, and a means for analyzing the contents of the calls and emotion data and recommending personalized content.

[1170] Hardware and Software Used

[1171] Hardware: Smartphone, head-mounted display

[1172] Software: Python, SpeechRecognition library, EmotionRecognizer library, ContentRecommender library

[1173] Data processing and calculation

[1174] The server first uses SIP communication to make a voice call. The contents of the call are acquired as voice data and converted to text data using the SpeechRecognition library. Next, the EmotionRecognizer library is used to recognize the user's emotions from the text data. This emotion data is stored in a database as an intangible information asset.

[1175] In addition, the ContentRecommender library is used to recommend the most suitable content to the user based on the recognized emotion data, and the recommended content is displayed on the user's smartphone or head-mounted display.

[1176] Examples

[1177] For example, if a user is feeling stressed during a call with a friend, the server can recognize that emotion and recommend relaxing music or a relaxation video. In this way, the user experience can be improved by providing personalized content based on the user's emotions.

[1178] Examples of prompt statements

[1179] Build an application that analyzes user conversations, recognizes emotions, and recommends the best content based on those emotions. Use the SpeechRecognition library for speech recognition, the EmotionRecognizer library for emotion recognition, and the ContentRecommender library for content recommendation.

[1180] In this way, it is possible to effectively utilize the call content and emotion data to provide optimal content to users.

[1181] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1182] Step 1:

[1183] The server uses SIP communication to make a voice call. When a user starts a call, voice data is acquired in real time. This voice data becomes the input.

[1184] Step 2:

[1185] The server converts the acquired voice data into text data using the SpeechRecognition library. This process outputs the voice data as text data. Specifically, the server reads the voice file, analyzes it with a voice recognition engine, and outputs it as text.

[1186] Step 3:

[1187] The server converts the converted text data into emotion data using the EmotionRecognizer library. This process outputs the text data as emotion data. Specifically, the server analyzes the content of the text and identifies emotions using an emotion recognition model.

[1188] Step 4:

[1189] The server saves the emotion data in a database as an intangible information asset. This process stores the emotion data in the database. Specifically, the server establishes a database connection and saves the emotion data in an appropriate format.

[1190] Step 5:

[1191] The server uses the ContentRecommender library to recommend optimal content to users based on their emotional data. This process inputs emotional data and outputs recommended content. Specifically, the server analyzes the emotional data and selects content that is appropriate for the user's emotional state.

[1192] Step 6:

[1193] The server transmits the recommended content to the user's device (such as a smartphone or a head-mounted display). Through this process, the recommended content is displayed on the user's device. Specifically, the content data is transmitted via a network and displayed on the user's device.

[1194] In this way, it is possible to effectively utilize the contents of the call and the emotion data to provide the user with the most suitable content.

[1195] Example 2

[1196] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[1197] In conventional telephone services, it was difficult to efficiently analyze the contents of calls and provide appropriate services to users. In addition, there was a lack of technology to identify specific call patterns and optimize services based on them. This meant that users' needs could not be quickly and accurately responded to, resulting in a decline in the quality of service.

[1198] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1199] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the calls as intangible information assets, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using large-scale data and artificial intelligence, a means for recording the contents of the calls as voice data and transmitting them to the server in real time, a means for converting the voice data into text data, a means for analyzing the text data using natural language processing technology, a means for identifying a specific call pattern based on the analysis results, and a means for providing a service to a user based on the identified call pattern. This makes it possible to quickly provide an appropriate service to a user by efficiently analyzing the contents of the calls and identifying a specific call pattern.

[1200] "SIP communication" is a communication technology for making voice and video calls using the Session Initiation Protocol.

[1201] "Telephone Services" refers to systems and platforms for providing voice calls.

[1202] "Intangible information assets" refers to information and data stored in digital format, including, for example, telephone conversation content and text data.

[1203] The Telecommunications Business Act is a law concerning the operation and regulation of telecommunications businesses, and provides the legal framework for the provision of telecommunications services.

[1204] "Service Provision Platform" refers to the technical and operational infrastructure for providing Services to Users.

[1205] "Big data" refers to very large amounts of data, also known as big data, including call records and user behavior data.

[1206] "Artificial intelligence" refers to technology that enables computer systems to mimic human intelligence to learn and reason.

[1207] "Audio data" refers to data that records audio in digital format.

[1208] "Text data" refers to data in which character information is recorded in digital format.

[1209] "Natural language processing technology" refers to technology that enables computers to understand, interpret, and generate human language.

[1210] "Call patterns" refers to certain tendencies or characteristics regarding calls, such as the content of calls or the time of day calls are made.

[1211] "User" refers to any individual or legal entity that uses the telephone service.

[1212] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1213] This invention is a system that provides telephone services using SIP communication, treats the contents of calls as intangible information assets, and builds a new service provision infrastructure within the scope of the Telecommunications Business Law. Furthermore, it aims to provide advanced services by utilizing large-scale data and artificial intelligence.

[1214] Server Processing

[1215] The server provides a telephone service using SIP communication. This service has a function of recording the contents of a call as voice data and sending it to the server in real time. The recorded voice data is converted into text data using a voice recognition engine (e.g., Google Speech-to-Text API). The server then analyzes the text data using natural language processing technology (e.g., NLTK, spaCy) to identify specific call patterns. Services are provided to the user based on the identified call patterns.

[1216] Terminal handling

[1217] The device starts recording as soon as the user starts a call. The recorded data is sent to the server in real time. An application with recording function is installed on the device, and recording starts automatically when a call starts. This application has the function of uploading the recorded data to the server in real time. The hardware used includes smartphones and tablets.

[1218] User Action

[1219] The user simply uses the phone as they normally would. If certain keywords are used frequently during calls, or if calls are concentrated at certain times of the day, the server will identify the patterns. The user can then receive services provided by the server after the call. For example, if a certain keyword is used frequently, the user can receive customer support suggestions based on that content.

[1220] Examples

[1221] For example, if a user calls customer support and frequently uses keywords such as "return" or "refund," the server can identify this and automatically provide the user with instructions on how to process a return. Also, if a user frequently calls at night, the server can provide support services tailored to that time period.

[1222] Example prompts for generative AI models

[1223] "If a user calls customer support and frequently uses keywords like 'returns' or 'refunds' during the call, explain how you can identify that calling pattern and provide the user with the appropriate service."

[1224] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1225] Step 1:

[1226] The device starts recording as soon as the user starts a call. The recorded data is sent to the server in real time. Specifically, the recording application installed on the device detects the start of a call and automatically starts recording. The input is the call audio, and the output is the recorded audio data.

[1227] Step 2:

[1228] The server receives the voice data sent from the terminal. It converts the received voice data into text data using a voice recognition engine (e.g., Google Speech-to-Text API). The input is voice data, and the output is text data. Specifically, it calls the voice recognition API to convert the voice data into text.

[1229] Step 3:

[1230] The server analyzes the converted text data using natural language processing technology (e.g., NLTK, spaCy). The input is the text data, and the output is the analysis results. Specifically, keywords are extracted from the text data and their frequency is calculated. The time of the call is also recorded.

[1231] Step 4:

[1232] The server identifies specific call patterns based on the analysis results. The input is the analysis results, and the output is the identified call patterns. Specifically, the server identifies calls in which specific keywords are frequently used or calls concentrated in specific time periods.

[1233] Step 5:

[1234] The server provides services to users based on the identified calling patterns. The input is the identified calling patterns, and the output is the services to be provided. Specifically, if a particular keyword is used frequently, the server suggests customer support based on the content of the keyword. In addition, for users who make frequent calls at night, the server provides support services tailored to that time period.

[1235] (Application example 2)

[1236] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1237] In conventional telephone services, it was difficult to analyze call content in real time and identify specific keywords or emotional states. In addition, there was no system that suggested appropriate responses based on the user's emotional state, so improving the user experience was a challenge.

[1238] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1239] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, a means for converting the contents of the call into text data in real time and identifying specific keywords and phrases, a means for analyzing the emotional state of the user and proposing an appropriate response using an emotion engine, and a means for identifying specific call patterns and optimizing services for the user. This enables real-time analysis of the contents of the call and identification of the emotional state, thereby realizing appropriate responses to the user and optimization of services.

[1240] "SIP communication" is a communications technology for making voice and video calls using the Session Initiation Protocol.

[1241] "Telephone service" refers to a system or platform for providing voice calls.

[1242] "Intangible information assets" are information or data stored in digital form that have value despite not having a physical form.

[1243] The Telecommunications Business Act is a law that sets out regulations and rules concerning telecommunications businesses.

[1244] "Service Provisioning Platform" means the technical and operational infrastructure for providing the Services.

[1245] "Big Data" refers to large amounts of data, and is a concept that includes technologies and methods for collecting, storing, and analyzing that data.

[1246] "AI" is an abbreviation for artificial intelligence, a technology that allows machines to imitate human intelligence to learn and make inferences.

[1247] "Real-time" refers to instantaneous processing or reaction without delay.

[1248] "Text data" is data that is stored as character information.

[1249] A "keyword" is a word or phrase that is important for locating or identifying specific information.

[1250] A "phrase" is a short sentence or expression that has meaning and is made up of multiple words.

[1251] An "emotion engine" is a technology that analyzes the user's emotional state and suggests appropriate responses based on the results.

[1252] "Call patterns" refer to certain actions or tendencies that are repeated within a call.

[1253] "Optimization" refers to making something the most effective for a particular purpose.

[1254] The system for carrying out the present invention is configured as follows: The server includes means for providing telephone services using SIP communication, means for treating call contents as intangible information assets, means for building a new service provision infrastructure within the scope of the Telecommunications Business Act, means for providing advanced services using BigData and AI, means for converting call contents into text data in real time and identifying specific keywords and phrases, means for analyzing the emotional state of the user and proposing appropriate responses using an emotion engine, and means for identifying specific call patterns and optimizing services for the user.

[1255] The server converts the call into text data in real time using a speech recognition library (e.g., the speech_recognition library). The converted text data is analyzed to identify specific keywords and phrases. For example, if the keyword "return" is frequently used in customer support calls, the server can identify this keyword and provide the user with details on how to process a return.

[1256] In addition, the server can use an emotion engine (e.g., TextBlob library) to analyze the user's emotional state. If the user expresses anger, the server can use that information to suggest an appropriate response. For example, if the user expresses anger, a notification can be automatically sent to customer support to facilitate a quick response.

[1257] The server also uses BigData and AI technology to identify specific call patterns, for example, call patterns concentrated in certain times of the day, and can optimize services for users based on that information.

[1258] As a concrete example, if the keyword "return" is frequently used in customer support calls, the server can identify the keyword "return" and provide the user with details on how to proceed with a return.

[1259] Examples of prompts to be input to a generative AI model include:

[1260] If a user calls customer support and frequently uses the keyword "returns," what would you suggest?

[1261] This prompt can then be used to have a generative AI model suggest an appropriate response.

[1262] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1263] Step 1:

[1264] The server receives a user's call using SIP communication. The input is voice data and the output is a stream of voice data. The server uses a speech recognition library (e.g., speech_recognition library) to process this voice data in real time.

[1265] Step 2:

[1266] The server converts the voice data into text data using a voice recognition library. The input is voice data, and the output is text data. Through this conversion process, the contents of the call are obtained as text information.

[1267] Step 3:

[1268] The server analyzes the converted text data to identify specific keywords or phrases. The input is the text data, and the output is a list of identified keywords or phrases. For example, it checks whether the text data contains keywords such as "return" or "claim."

[1269] Step 4:

[1270] The server analyzes the text data using an emotion engine (e.g., the TextBlob library) to identify the user's emotional state. The input is the text data, and the output is the emotional state (e.g., happy, anger, sadness, etc.). Through this analysis, the user's emotional state is understood.

[1271] Step 5:

[1272] The server suggests appropriate responses based on the identified keywords and emotional state. The input is a list of keywords and an emotional state, and the output is a list of suggested responses. For example, if a user uses the keyword "return" and indicates an angry emotion, the server automatically notifies customer support to promptly respond.

[1273] Step 6:

[1274] The server uses BigData and AI technology to identify call patterns. The input is past call data, and the output is the identified call patterns. For example, it can identify call patterns that are concentrated in a certain time period and use that information to optimize services for users.

[1275] Step 7:

[1276] The server uses a generative AI model to generate an optimal response for the user. The inputs are the prompt text and the call data, and the output is the generated suggested response. For example, using the following prompt text:

[1277] If a user calls customer support and frequently uses the keyword "returns," what would you suggest?

[1278] Based on this prompt, the generative AI model suggests an appropriate response.

[1279] Example 3

[1280] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1281] In conventional telephone services, it was difficult to effectively utilize the contents of phone calls to optimize the provision of services to users. In addition, there was a lack of means to accurately grasp the user's emotions and needs from the contents of phone calls and provide appropriate services based on that. This meant that improvements in user satisfaction and personalization of services were not fully realized.

[1282] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1283] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using large-scale data and artificial intelligence, a means for collecting the contents of the call in real time and converting the voice into text using voice recognition software, a means for storing the converted text data in a database, a means for extracting specific keywords from the text data using a natural language processing model, a means for recognizing the user's emotions from the contents of the call using an emotion engine, and a means for providing the optimal service to the user based on the extracted keywords and the emotion recognition result. This makes it possible to effectively utilize the contents of the call and provide the optimal service based on the user's needs and emotions.

[1284] "SIP communication" is a communications technology for making voice and video calls using the Session Initiation Protocol.

[1285] "Telephone service" refers to a communication service for providing voice calls.

[1286] "Intangible information assets" are information or data stored in digital form that have value despite not having a physical form.

[1287] The Telecommunications Business Act is a law concerning the operation and regulation of telecommunications businesses.

[1288] "Service provision infrastructure" means the technical and operational infrastructure for providing a service.

[1289] "Large-scale data" refers to a very large amount of data set, which is difficult to process using conventional data processing techniques.

[1290] "Artificial intelligence" is the technology that enables computer systems to mimic human intelligence by learning, reasoning, and self-correcting.

[1291] "Speech recognition software" means software for converting speech into text.

[1292] A "database" is a system for efficiently storing, managing, and searching data.

[1293] A "natural language processing model" is an algorithm or technology for understanding and generating human language.

[1294] "Keywords" are important words or phrases used to represent specific information.

[1295] An "emotion engine" is a technology for recognizing emotions from text and voice.

[1296] "User" means an individual or organization that uses the system or services.

[1297] An "optimal service" is one that best suits the user's needs and circumstances.

[1298] This invention is a system that provides telephone services using SIP communication, treats the contents of calls as intangible information assets, and builds a new service provision infrastructure within the scope of the Telecommunications Business Law. Furthermore, it utilizes large-scale data and artificial intelligence to provide advanced services to users.

[1299] Hardware and Software Used

[1300] The server uses the following hardware and software:

[1301] SIP communication module: Manages voice calls using SIP communication.

[1302] Speech recognition software: Converts voice data into text (e.g., Google Cloud Speech-to-Text API).

[1303] Database system: Stores call content and metadata (e.g. MySQL, PostgreSQL).

[1304] Natural Language Processing models: Extracting keywords from text data (e.g. SpaCy, NLTK).

[1305] Emotion engine: Recognize user emotions from call content (e.g. Microsoft Azure Emotion API).

[1306] Artificial intelligence models: Analyze large-scale data and provide optimal services to users (e.g. TensorFlow, PyTorch).

[1307] Data processing and calculation

[1308] The server collects the call contents in real time and converts the voice to text using speech recognition software. The converted text data is stored in a database. It then uses a natural language processing model to extract specific keywords from the text data. It also uses an emotion engine to recognize the user's emotions from the call contents. Finally, it uses an artificial intelligence model to provide optimal services to the user based on the extracted keywords and emotion recognition results.

[1309] Examples

[1310] Example 1: Service provision by keyword extraction

[1311] If a user frequently mentions "I want a new smartphone" during a call, the server will extract that keyword and provide the user with information about the latest smartphones and links to purchase them.

[1312] Example prompt:

[1313] "Please extract frequently used keywords from the following call: Call: 'I've been thinking about going on a trip lately. Any recommendations on where to go?'"

[1314] Example 2: Service provision using emotion recognition

[1315] If the user expresses a happy emotion during the call, the server can use that information to recommend special discount coupons or related products to the user.

[1316] Example prompt:

[1317] "Recognize the user's emotions in the following call: 'Yay! Finally, the project is successful!'"

[1318] In this way, the server effectively utilizes the contents of the call to provide the most suitable service based on the needs and feelings of the user. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1319] Step 1:

[1320] Call data collection

[1321] The server uses SIP communication to monitor the call between the user and the terminal in real time. When the call is started, the server captures the voice data. The input is the voice call between the user and the terminal, and the output is the captured voice data. In concrete terms, the server uses the SIP communication module to monitor the call and collect the voice data.

[1322] Step 2:

[1323] Converting audio data to text

[1324] The server converts the captured voice data into text using voice recognition software (e.g., Google Cloud Speech-to-Text API). The input is the voice data, and the output is the converted text data. In concrete terms, the server calls the voice recognition software and converts the voice data into text.

[1325] Step 3:

[1326] Data storage

[1327] The server stores the converted text data in a database. The input is the text data and call metadata (date and time, call recipient, etc.), and the output is the data stored in the database. In concrete terms, the server uses a database system to store the text data and metadata.

[1328] Step 4:

[1329] Keyword Extraction

[1330] The server analyzes the stored text data and extracts specific keywords using a natural language processing model (e.g., SpaCy, NLTK). The input is the text data, and the output is the extracted keywords. In concrete terms, the server invokes the natural language processing model to extract keywords from the text data.

[1331] Step 5:

[1332] emotion recognition

[1333] The server uses an emotion engine (e.g. Microsoft Azure Emotion API) to recognize the user's emotion from the call content. The input is text data, and the output is the emotion recognition result. In concrete terms, the server calls the emotion engine, analyzes the text data, and recognizes the user's emotion.

[1334] Step 6:

[1335] Optimizing service delivery

[1336] The server uses an artificial intelligence model to provide the optimal service to the user based on the extracted keywords and emotion recognition results. The input is the keywords and emotion recognition results, and the output is the optimal service provided to the user. In specific operations, the server uses the artificial intelligence model to analyze the data, identify the optimal service or product for the user, and send a notification.

[1337] In this way, the server can effectively utilize the contents of the call and provide optimal services based on the user's needs and emotions.

[1338] (Application example 3)

[1339] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1340] In conventional telephone services, it was difficult to optimize the service provided to users by utilizing the contents of the call. In addition, services were not provided that took into account the user's emotional state, so user satisfaction could not be improved. Furthermore, there was a lack of technology to treat the contents of the call as an intangible information asset and to properly analyze and utilize it. In order to solve these issues, a system is needed that can efficiently analyze the contents of the call and provide optimal services based on the user's emotional state.

[1341] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1342] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, a means for converting the contents of the call into text, a means for analyzing the emotions of the text, and a means for recommending products and services based on the results of the emotion analysis. This makes it possible to efficiently analyze the contents of the call and provide optimal services based on the emotional state of the user.

[1343] "SIP communication" is a communications technology that uses the Session Initiation Protocol to make voice and video calls.

[1344] "Telephone service" refers to a communication service for providing voice calls.

[1345] "Intangible information assets" are information or data stored in digital form, and are assets that have value despite not having a physical form.

[1346] The Telecommunications Business Act is a law concerning the operation and regulation of telecommunications businesses.

[1347] "Service provision infrastructure" means the technical and operational infrastructure for providing a service.

[1348] "Big Data" refers to large volumes of diverse data sets, and is a technology that creates new knowledge and value by analyzing them.

[1349] "AI" is an abbreviation for artificial intelligence, a technology that allows machines to imitate human intelligence to learn and make inferences.

[1350] "Means for converting the contents of a voice call into text" refers to technology for converting the contents of a voice call into text information.

[1351] A "means for analyzing text sentiment" is a technique for extracting sentiment from text data and assessing the emotional state.

[1352] "Means for recommending products and services based on the results of sentiment analysis" refers to technology that suggests products and services suitable for users based on the results of sentiment analysis.

[1353] The following system configuration and processing procedure will be described as an embodiment of the present invention.

[1354] System Configuration

[1355] This system uses the following hardware and software:

[1356] Hardware: Smartphones, servers

[1357] Software: Python, SpeechRecognition library, TextBlob library, Requests library

[1358] Processing Steps

[1359] 1. Voice Recognition:

[1360] A user calls customer support using a smartphone. The contents of the call are recorded on the smartphone and saved as an audio file. The server converts the audio file into text using the SpeechRecognition library. Through this process, the contents of the call are obtained as text information.

[1361] 2. Sentiment analysis:

[1362] The server analyzes the sentiment of the text data using the TextBlob library. Sentiment analysis evaluates the emotional state of the text, such as positive, negative, or neutral, and obtains the user's emotional state as numerical data.

[1363] 3. Product recommendation:

[1364] The server recommends appropriate products and services based on the results of sentiment analysis. Using the Requests library, it retrieves product information from external APIs and suggests the best products and services to the user. This process makes it possible to provide personalized services according to the user's emotional state.

[1365] Examples

[1366] For example, if a user calls customer support and says, "I've been busy at work lately and I'm feeling stressed," the server converts the call content into text and performs sentiment analysis. If the result of the sentiment analysis indicates a negative emotion, the server recommends a product that will help the user relax (e.g., an aroma candle or a massage machine).

[1367] Examples of prompt statements

[1368] Please write a Python program that converts the conversation of the user who called customer support into text, analyzes the sentiment of the text, and recommends appropriate products. The libraries used are SpeechRecognition, TextBlob, and Requests.

[1369] The above is a specific embodiment for carrying out the present invention.

[1370] The flow of the specific process in the application example 3 will be described with reference to FIG.

[1371] Step 1:

[1372] A user calls customer support using a smartphone. The contents of the call are recorded on the smartphone and saved as an audio file.

[1373] Input: User's call audio

[1374] Output: Audio file (e.g. .wav format)

[1375] What happens: A user calls customer support and the conversation is saved using the smartphone's recording function.

[1376] Step 2:

[1377] The server converts the audio file into text using the SpeechRecognition library.

[1378] Input: Audio file

[1379] Output: Text data

[1380] How it works: The server receives the audio file and converts it into text using the SpeechRecognition library. Specifically, it reads the audio file and converts it into text using Google's speech recognition API.

[1381] Step 3:

[1382] The server analyzes the sentiment of the text data using the TextBlob library.

[1383] Input: Text data

[1384] Output: Sentiment score (positive, negative, neutral)

[1385] What it does: The server passes the text data to the TextBlob library for sentiment analysis. TextBlob calculates the text's positive, negative, or neutral sentiment score and returns the result.

[1386] Step 4:

[1387] The server recommends appropriate products and services based on the results of the sentiment analysis.

[1388] Input: Sentiment score

[1389] Output: A list of recommended products

[1390] Specific operation: The server sends a request to an external API based on the emotion score to obtain appropriate product information. For example, if the emotion score is negative, the server sends an API request to recommend relaxing products and presents the obtained product information to the user.

[1391] The above are the specific process steps for carrying out the present invention.

[1392] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1393] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1394] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1395] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1396] [Third embodiment]

[1397] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1398] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1399] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1400] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1401] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1402] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1403] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1404] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1405] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1406] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1407] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1408] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1409] "Example 1"

[1410] As an example of the embodiment of the present invention, there is a system that provides telephone services using SIP communication. This system has a function to handle the contents of calls as intangible information assets. Specifically, the contents of calls are digitized, and the data is analyzed and stored to make effective use of the contents of calls. In addition, the system has a function to build a new service provision infrastructure within the scope that does not violate the Telecommunications Business Act. Specifically, new services are provided to users based on data on the contents of calls. Furthermore, the system has a function to provide advanced services that utilize BigData and AI. Specifically, data on the contents of calls is accumulated as BigData, and the data is analyzed using AI to optimize the provision of services to users.

[1411] "Example 2"

[1412] Another embodiment of the present invention is a telephone service using SIP communication with a function for identifying specific call patterns. This system digitizes the contents of calls and identifies specific call patterns by analyzing the data. For example, it identifies call patterns such as calls where a specific keyword is frequently used or calls concentrated in a specific time period, and provides services to users based on the identified call patterns.

[1413] "Example 3"

[1414] Another embodiment of the present invention is a telephone service using SIP communication that has a function of optimizing the provision of services to users based on the content of the calls. This system optimizes the provision of services to users by accumulating data on the content of the calls as BigData and analyzing the data using AI. For example, the system optimizes the provision of services based on the content of the calls by analyzing calls in which a specific keyword is frequently used and providing the user with services related to the keyword.

[1415] The process flow of each embodiment will be described below.

[1416] "Example 1"

[1417] Step 1: Make a telephone call using SIP communication.

[1418] Step 2: The contents of the calls are digitized and treated as an intangible information asset.

[1419] Step 3: Analyze and store the call content data to make effective use of the call content.

[1420] Step 4: Build a new service provision infrastructure within the scope of the Telecommunications Business Act. Step 5: Provide new services to users based on the data on call content.

[1421] Step 6: Accumulate the call content data as BigData and analyze the data using AI.

[1422] Step 7: Optimize the provision of services to users based on the results of AI analysis.

[1423] "Example 2"

[1424] Step 1: Make a telephone call using SIP communication.

[1425] Step 2: Record the call contents and analyze the data.

[1426] Step 3: Identify specific call patterns from the analysis results.

[1427] Step 4: Provide services to the user based on the identified call patterns.

[1428] Step 1: Make a telephone call using SIP communication.

[1429] Step 2: Accumulate the call content data as BigData.

[1430] Step 3: Use AI to analyze the accumulated data.

[1431] Step 4: Optimize the provision of services to users based on the results of AI analysis.

[1432] Example 1

[1433] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1434] Conventional telephone service systems had limited means of effectively utilizing call content, making it difficult to digitize call content for analysis and storage. It was also difficult to build a new service provision infrastructure without violating the Telecommunications Business Act, and services provided to users were not fully optimized. Furthermore, there was a demand for advanced technology to capture call content in real time, store it as voice data, and convert it into text data for analysis.

[1435] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1436] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the calls as intangible information assets, a means for converting the contents of the calls into digital data and analyzing and storing the data, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing new services to users based on the data of the calls, a means for providing advanced services using BigData and AI, a means for capturing the contents of the calls in real time and storing them as voice data, a means for converting the voice data into text data using voice recognition software, a means for storing the converted text data in a database, a means for analyzing the accumulated data with an AI model, and a means for providing services to users based on the analysis results. This makes it possible to effectively utilize the contents of the calls and optimize the provision of services to users.

[1437] "SIP communication" is a communications technology for making voice and video calls using the Session Initiation Protocol.

[1438] "Telephone services" refers to all services for providing voice calls.

[1439] "Intangible information assets" refers to information data that has value but does not have a physical form.

[1440] "Digitization" refers to the conversion of information such as audio and video into digital data.

[1441] "Analysis" refers to the processing of data to extract useful information.

[1442] "Storage" refers to keeping data for an extended period of time.

[1443] The Telecommunications Business Act is a law concerning the operation of telecommunications businesses.

[1444] "Service Provision Infrastructure" means the technical and physical infrastructure for providing the Services.

[1445] "Big Data" refers to large amounts of data and includes the technologies to collect, store, and analyze that data.

[1446] "AI" refers to artificial intelligence and includes technologies such as machine learning and natural language processing.

[1447] "Real-time" refers to processing that occurs immediately, without delay.

[1448] "Capture" refers to obtaining data.

[1449] "Audio data" refers to data that represents audio in digital form.

[1450] "Speech recognition software" means software for converting speech into text.

[1451] "Text data" refers to data that represents character information in digital form.

[1452] "Database" refers to a system for efficiently storing, managing, and retrieving data.

[1453] "AI model" refers to a model for analyzing data using artificial intelligence algorithms.

[1454] "Analysis Results" refers to information obtained through data analysis.

[1455] "Service provision" refers to providing useful functions and information to users.

[1456] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1457] The present invention is a system that provides telephone services using SIP communication and handles call content as intangible information assets. A specific embodiment of this system will be described below.

[1458] 1. Generate the system program

[1459] The server generates a program for providing telephone services using SIP communication. This program has the function of converting the contents of the call into data and analyzing and saving that data.

[1460] 2. Explain the program's processing in natural language

[1461] The server captures the call in real time and stores it as voice data using SIP server software such as Asterisk, which is then converted to text data using voice recognition software (e.g., Google Cloud Speech-to-Text API).

[1462] The terminal receives the converted text data and stores it in a database (e.g., MySQL). The stored data is accumulated using BigData technology (e.g., Hadoop).

[1463] The server analyzes the accumulated data using AI models (for example, natural language processing models using TensorFlow) to provide new services to users, such as customized suggestions based on the content of the call, call summaries, and sentiment analysis.

[1464] 3. Add examples to your description

[1465] As a concrete example, consider a case where a user calls customer support and asks, "How do I return an item?"

[1466] The server captures the contents of the call and stores them as voice data. It then uses voice recognition software to convert the voice data into text data such as "Please tell me how to return the product." The converted text data is stored in a database and accumulated using BigData technology.

[1467] The AI ​​model analyzes this text data and provides users with detailed guidelines on how to return items. It can also predict frequently asked questions based on past call data and provide FAQs in advance.

[1468] Example of a prompt to be input to a generative AI model:

[1469] "When a user calls customer support to ask how to return an item, digitize the conversation, analyze it, and explain what kind of service you would provide to that user."

[1470] In this way, a system is constructed in which the server, terminals, and users work together to effectively utilize call contents and provide new services.

[1471] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1472] Step 1:

[1473] The server captures calls between users in real time using SIP communication. The input is the call audio between user A and user B, and the output is the captured audio data. Specifically, the server uses SIP server software such as Asterisk to obtain the audio data of the call. For example, when user A says, "Please tell me how to return a product," the audio is captured.

[1474] Step 2:

[1475] The server temporarily stores the captured voice data. The input is the voice data captured in step 1, and the output is the stored voice data. The data is stored in storage within the server (e.g., cloud storage). Specifically, the data is stored as an audio file, and the call date and time and caller information are included in the file name.

[1476] Step 3:

[1477] The server converts the stored voice data into text data using voice recognition software. The input is the voice data stored in step 2, and the output is the converted text data. Specifically, the voice data is converted into text using Google Cloud Speech-to-Text API. For example, the voice saying "Please tell me how to return the product" is converted into text "Please tell me how to return the product."

[1478] Step 4:

[1479] The terminal receives the converted text data and stores it in a database. The input is the text data converted in step 3, and the output is the text data stored in the database. Specifically, it is stored in a MySQL database, and when it is stored, the call date and time and caller information are also stored.

[1480] Step 5:

[1481] The server accumulates and manages the saved text data using BigData technology. The input is the text data saved in step 4, and the output is the accumulated data. Specifically, the data is saved in distributed storage using Hadoop and indexed for efficient access.

[1482] Step 6:

[1483] The server analyzes the accumulated data using an AI model. The input is the data accumulated in step 5, and the output is the analysis results. Specifically, the data is analyzed using a natural language processing model that uses TensorFlow, and sentiment analysis, keyword extraction, and trend analysis are performed.

[1484] Step 7:

[1485] The user receives new services based on the analysis results. The input is the analysis results obtained in step 6, and the output is the service provided to the user. Specifically, this includes customized suggestions based on the content of the call, a summary of the call, and the provision of FAQs. For example, if a user says, "Please tell me how to return an item," the server will provide "detailed guidelines on how to return the item." In addition, based on past call data, the server can predict what questions users will frequently ask and provide FAQs in advance.

[1486] (Application example 1)

[1487] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1488] In the conventional telephone service system, the means to effectively utilize the call contents were limited, and the service provision to users was not sufficiently optimized. In addition, the function to analyze the call contents in real time and recommend the content related to the user was lacking, so an improvement of the user experience was required.

[1489] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1490] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the calls as intangible information assets, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, a means for converting the contents of the calls into digital data in real time and analyzing them using AI, and a means for recommending content related to the user based on the analysis results. This makes it possible to provide advanced services based on the contents of the calls and recommend optimal content to the user.

[1491] "SIP communication" is a communications technology for making voice and video calls using the Session Initiation Protocol.

[1492] "Telephone service" means a system or means for providing voice communications.

[1493] "Call Content" refers to voice data and information exchanged through telephone services.

[1494] "Intangible information assets" are information or data stored in digital form that have value despite not having a physical form.

[1495] The Telecommunications Business Act is a law concerning the operation and regulation of telecommunications businesses.

[1496] "Service Provisioning Infrastructure" means the technical and operational infrastructure for providing the Services.

[1497] "Big Data" refers to large volumes of diverse data sets, and is a technology that creates new knowledge and value by analyzing them.

[1498] "AI" is an abbreviation for artificial intelligence, a technology that allows machines to imitate human intelligence to learn and make inferences.

[1499] "Digitization in real time" refers to instantly converting the contents of a call into digital data.

[1500] "Analysis" is the act of processing data to extract useful information.

[1501] "Content" refers collectively to information and entertainment provided to Users.

[1502] A "recommendation" is the act of presenting the best option based on certain criteria.

[1503] The system for carrying out this invention has the following configuration: The server includes means for providing telephone services using SIP communication, means for treating the contents of the calls as intangible information assets, means for building a new service provision infrastructure within the scope of the Telecommunications Business Act, means for providing advanced services using BigData and AI, means for converting the contents of the calls into digital data in real time and analyzing them using AI, and means for recommending content relevant to the user based on the analysis results.

[1504] Specifically, when a user makes a call using a smartphone, the contents of the call are digitized in real time. This data is sent to a server and analyzed using an AI model. For the analysis, a voice recognition library (e.g., the speech_recognition library) is used to convert the contents of the call into text. The converted text is sent to the API endpoint of the AI ​​model for detailed analysis.

[1505] The server recommends content relevant to the user based on the analysis results. For example, if a user says, "I want to talk about recent movies," the AI ​​model analyzes this text and recommends the latest movie information and reviews. This recommendation is notified to the user's smartphone.

[1506] The hardware used is the smartphone's microphone and speaker, and the server's processing power. The software uses the speech_recognition library and the API of the AI ​​model.

[1507] As a concrete example, if a user says, "I want to talk about recent movies," the AI ​​model analyzes this text and recommends the latest movie information and reviews. The following is an example of a prompt sentence to be input to the generative AI model.

[1508] "If a user says, 'I want to talk about recent movies,' recommend the latest movie information and reviews."

[1509] In this way, advanced services based on the content of the call and optimal content recommendations for the user can be realized.

[1510] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1511] Step 1:

[1512] A user starts a call using a smartphone. The smartphone's microphone captures the voice and digitizes the call content in real time. The input is the user's voice, and the output is digital voice data.

[1513] Step 2:

[1514] The smartphone transmits digital voice data to the server. The server converts the voice data into text using a voice recognition library (e.g., speech_recognition library). The input is the digital voice data and the output is the text data.

[1515] Step 3:

[1516] The server sends the converted text data to the API endpoint of the AI ​​model. The AI ​​model analyzes the text data and extracts the user's intent and interests. The input is the text data, and the output is the analysis result.

[1517] Step 4:

[1518] The server recommends content relevant to the user based on the analysis results. For example, if a user says, "I want to talk about recent movies," the AI ​​model recommends the latest movie information and reviews. The input is the analysis results, and the output is the recommended content.

[1519] Step 5:

[1520] The server notifies the user of the recommended content to the smartphone. The smartphone receives the notification and displays it to the user. The input is the recommended content, and the output is the notification to the user.

[1521] In this way, advanced services based on the contents of the call and optimal content recommendations to the user are realized.

[1522] Example 2

[1523] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1524] In conventional telephone services using SIP communications, it was difficult to efficiently analyze call content and identify specific call patterns. In addition, there was a lack of means to digitize call content in real time and use that data to optimize services for users. This made it difficult to respond quickly and accurately to user needs, resulting in a decline in the quality of service.

[1525] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1526] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using large amounts of data and artificial intelligence, a means for capturing the contents of the call in real time and converting the voice data into text data, a means for storing the converted text data, a means for analyzing the stored text data and extracting patterns such as the frequency of appearance of specific keywords and the time period of the call, and a means for transmitting the analysis results to a user terminal and visually displaying them. This makes it possible to efficiently analyze the contents of the call and identify specific call patterns, thereby optimizing the provision of services to users.

[1527] "SIP communication" is a technology that uses the Session Initiation Protocol to manage multimedia communications such as voice and video.

[1528] "Telephone service" refers to a communication service for providing voice calls.

[1529] "Intangible information assets" are information or data stored in digital form, and are assets that have value despite not having a physical form.

[1530] The Telecommunications Business Act is a law concerning the operation and regulation of telecommunications businesses.

[1531] "Service Provisioning Platform" means the technical and operational infrastructure for providing the Services.

[1532] "Massive data", also known as big data, refers to a very large data set, the amount of data that cannot be handled by conventional data processing techniques.

[1533] "Artificial intelligence" is a technology that enables machines to mimic human intelligence and learn, reason, and solve problems.

[1534] "Real-time" refers to data processing and communication occurring immediately and without delay.

[1535] "Audio data" refers to data in which audio is recorded in digital format.

[1536] "Text data" refers to data in which character information is recorded in digital format.

[1537] A "database" is a system for efficiently storing, searching, and managing data.

[1538] "Natural language processing" is a technology that enables computers to understand and generate human language.

[1539] A "keyword" is a word or phrase that is important when searching for or analyzing specific information.

[1540] A "user terminal" is a device such as a computer or smartphone that is directly operated by a user.

[1541] "Visual display" refers to presenting data or information to a user in an easily viewable form, such as in a graph or chart.

[1542] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1543] This invention is a system for telephone services using SIP communication that digitizes call contents in real time, analyzes the data to identify specific call patterns, and optimizes services for users. A specific embodiment of this system is described below.

[1544] Hardware and Software Used

[1545] The server uses SIP server software (e.g., Asterisk or FreeSWITCH) to capture calls made through SIP communication in real time, and uses a speech recognition API (e.g., Google Cloud Speech-to-Text API) to convert the captured voice data into text data, which is then stored in a relational database (e.g., MySQL or PostgreSQL).

[1546] The server uses Python natural language processing libraries (e.g., NLTK and spaCy) to analyze the stored text data, extracting patterns such as the frequency of occurrence of certain keywords or the time of day of calls.

[1547] The terminal uses JavaScript libraries (e.g. Chart.js and D3.js) to receive the analysis results sent from the server and visually display them to the user.

[1548] Examples

[1549] Consider the case where the user is an administrator of a customer support center. The user wants to identify calls that frequently use certain keywords (e.g., "complaint" or "returned goods"). The server automatically extracts calls that contain these keywords and sends the analysis results to the terminal. The terminal displays a graph to the user showing which time periods these calls are concentrated.

[1550] Example of a prompt to be input to a generative AI model:

[1551] "Analyze call data from your customer support center and identify calls that frequently use certain keywords (e.g., 'claims' or 'returns'). Also, create a graph to show which times of day these calls are concentrated."

[1552] In this way, users can easily keep track of calling patterns and take appropriate measures.

[1553] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1554] Step 1:

[1555] The server captures calls made through SIP communication in real time. Specifically, it uses SIP server software (e.g., Asterisk or FreeSWITCH) to obtain the voice data of calls. The input is the call data from SIP communication, and the output is the captured voice data.

[1556] Step 2:

[1557] The server converts the captured voice data into text data. This conversion is done using a voice recognition API (e.g., Google Cloud Speech-to-Text API). The server sends the voice data to the API and receives the returned text data. The input is the captured voice data, and the output is the converted text data.

[1558] Step 3:

[1559] The server stores the converted text data in a database. Specifically, a relational database (e.g., MySQL or PostgreSQL) is used. The text data is stored together with the call timestamp and caller information. The input is the converted text data, and the output is the text data stored in the database.

[1560] Step 4:

[1561] The server analyzes the stored text data. It uses Python natural language processing libraries (e.g., NLTK and spaCy) to extract patterns such as the frequency of occurrence of specific keywords or the time of day of calls. The input is the text data stored in the database, and the output is the analysis results.

[1562] Step 5:

[1563] The server sends the analysis results to the user terminal. Specifically, it uses a RESTful API to send data in JSON format. The input is the analysis results, and the output is the data sent to the user terminal.

[1564] Step 6:

[1565] The terminal visually displays the received analysis results to the user. It uses JavaScript libraries (e.g. Chart.js and D3.js) to display graphs and lists of calls where specific keywords are frequently used or where calls are concentrated in a specific time period. The input is the analysis results sent from the server, and the output is the visually displayed data.

[1566] (Application example 2)

[1567] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1568] With conventional telephone services, it was difficult to analyze call content in real time and identify specific keywords or patterns. In addition, there was a lack of functionality to detect dangerous call patterns and send warnings to users, so sufficient measures could not be taken against risks such as fraud and blackmail. Furthermore, there was a lack of functionality to analyze call history and provide reports on dangerous call patterns, making it difficult to ensure the safety of users.

[1569] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1570] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, a means for converting the contents of the call into data in real time and identifying specific keywords or patterns, a means for sending a warning to the user based on the identified keywords or patterns, and a means for analyzing the call history and providing a report of dangerous call patterns. This makes it possible to detect specific keywords or patterns in real time during a call and immediately send a warning to the user. In addition, by analyzing the call history and providing a report of dangerous call patterns, the safety of the user can be ensured.

[1571] "SIP communication" is a communication method that uses the Session Initiation Protocol to make voice and video calls.

[1572] "Telephone service" refers to a communication service for providing voice calls.

[1573] An "intangible information asset" is information stored as digital data; it is an asset that has value despite not having a physical form.

[1574] The Telecommunications Business Act is a law concerning the operation of telecommunications businesses, and sets out regulations concerning the provision of telecommunications services.

[1575] "Service provision infrastructure" means the technical and operational infrastructure for providing a service.

[1576] "Big Data" refers to large amounts of data, and is a concept that includes technologies and methods for collecting, storing, and analyzing that data.

[1577] "AI" is an abbreviation for artificial intelligence, a technology that allows machines to imitate human intelligence to learn and make inferences.

[1578] An "advanced service" is a service that has more advanced functions and performance than conventional services.

[1579] "Real-time" refers to immediate processing or response without delay.

[1580] "Digitization" refers to converting information into digital data.

[1581] A "keyword" is a word that has a particular meaning or importance.

[1582] A "pattern" is a collection of data that has a particular regularity or tendency.

[1583] A "warning" is a notification that warns of a possible danger or problem.

[1584] A "call history" is a record of past calls.

[1585] A "report" is a report that summarizes analysis results and information.

[1586] A system for carrying out this invention is configured as follows: The server includes means for providing telephone services using SIP communication, means for treating call contents as intangible information assets, means for building a new service provision infrastructure within the scope of the Telecommunications Business Act, means for providing advanced services using BigData and AI, means for converting call contents into data in real time and identifying specific keywords or patterns, means for sending a warning to the user based on the identified keywords or patterns, and means for analyzing call history and providing a report on risky call patterns.

[1587] Program processing explanation

[1588] Hardware used

[1589] Smartphone (microphone, speaker)

[1590] Software used

[1591] Python

[1592] SpeechRecognition Library

[1593] Google Speech Recognition API

[1594] Data processing and calculation

[1595] The server first captures the call audio through the smartphone's microphone. The captured audio data is converted to text data using the SpeechRecognition library. The converted text data is then analyzed to identify specific keywords and patterns. This analysis includes matching keywords using regular expressions.

[1596] Based on the identified keywords and patterns, the server sends a warning to the user in real time via the smartphone's notification function, and the call history is stored in a database for later analysis. The analysis results are then provided to the user as a report of risky calling patterns.

[1597] Examples

[1598] For example, if the keyword "scam" is detected during a user's call, the server will immediately display a warning and record the call content and the detected keyword. This warning will be sent to the user through the smartphone's notification function. The call history will also be stored in a database for later analysis. The analysis results will be provided to the user as a report of risky calling patterns.

[1599] Examples of prompt statements

[1600] "Write a Python program that will alert the user and record the call if certain keywords (e.g. scam, blackmail) are detected during the call."

[1601] In this way, it is possible to detect certain keywords or patterns during calls in real time and send immediate warnings to users, and also analyze call history to provide reports on risky calling patterns to ensure user safety.

[1602] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1603] Step 1:

[1604] The server captures the call audio through the smartphone's microphone. The input is the call audio data, and the output is an audio data file. This audio data is stored for analysis in subsequent processing steps.

[1605] Step 2:

[1606] The server converts the acquired voice data into text data using the SpeechRecognition library. The input is a voice data file, and the output is text data. This conversion process converts the voice data into an analyzable text format.

[1607] Step 3:

[1608] The server analyzes the converted text data to identify specific keywords or patterns. The input is the text data, and the output is a list of identified keywords or patterns. This analysis includes matching keywords using regular expressions.

[1609] Step 4:

[1610] The server sends alerts to users based on the identified keywords and patterns. The input is a list of identified keywords and patterns, and the output is an alert notification to the user. This notification is done in real time through the smartphone notification feature.

[1611] Step 5:

[1612] The server stores the call history in a database. The input is the text data of the call and a list of identified keywords or patterns, and the output is the call history stored in the database. This storage process accumulates data for later analysis of the call history.

[1613] Step 6:

[1614] The server periodically analyzes the stored call history and provides a report on risky calling patterns. The input is the call history stored in the database, and the output is a report of the analysis results. This report is provided to the user and is used to understand risky calling patterns.

[1615] Example 3

[1616] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1617] With conventional telephone services, it was difficult to efficiently analyze the contents of phone calls and optimize the services provided to users. In addition, there was a lack of appropriate means to treat the contents of phone calls as intangible information assets, and there was a need to build a new service provision infrastructure that did not violate the Telecommunications Business Act. Furthermore, no specific method had been established for providing advanced services that utilize large amounts of data and artificial intelligence.

[1618] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1619] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using large amounts of data and artificial intelligence, a means for capturing the contents of the call in real time and converting the voice data into text data, a means for storing and analyzing the converted text data, and a means for optimizing the provision of services to users based on the analysis results. This makes it possible to efficiently analyze the contents of the call and optimize the provision of services to users.

[1620] "SIP communication" is a communications technology for making voice and video calls using the Session Initiation Protocol.

[1621] "Telephone service" means a system or means for providing voice communications.

[1622] "Intangible information assets" are information or data stored in digital form, and are assets that have value despite not having a physical form.

[1623] The Telecommunications Business Act is a law concerning the operation and regulation of telecommunications businesses.

[1624] "Service Provisioning Platform" means the technical and operational infrastructure for providing the Services.

[1625] "Massive data" refers to very large data sets, usually referred to as big data.

[1626] "Artificial intelligence" is a technology that allows computers to imitate human intelligence to learn and make inferences.

[1627] "Real-time" refers to immediate processing or response without delay.

[1628] "Audio data" refers to data in which audio is recorded in digital format.

[1629] "Text data" refers to data in which character information is recorded in digital format.

[1630] "Storage" refers to storing data and making it available for later use.

[1631] "Analysis" is the process of examining data in detail to find meaning and patterns.

[1632] "Optimization" refers to finding the most efficient way to achieve a particular goal.

[1633] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1634] This invention is a system that provides telephone services using SIP communication, treats the contents of calls as intangible information assets, and builds a new service provision infrastructure within the scope of the Telecommunications Business Law. Furthermore, it provides advanced services by utilizing large amounts of data and artificial intelligence, captures the contents of calls in real time, converts the voice data into text data, and accumulates and analyzes the converted text data, optimizing the provision of services to users.

[1635] Hardware and Software Used

[1636] The server uses the following hardware and software:

[1637] SIP communication module: A communication technology for making voice calls using SIP communication.

[1638] Speech recognition API: An API for converting voice data into text data. Specifically, we use the Google Cloud Speech-to-Text API.

[1639] Data storage system: A distributed file system for efficiently storing and managing large amounts of text data. Specifically, it uses a Hadoop cluster.

[1640] Generative AI model: An AI model for analyzing text data and extracting frequently used keywords and phrases. Specifically, we use OpenAI's GPT-4.

[1641] Program processing explanation

[1642] The server captures the conversation between the user and the device in real time. When the conversation starts, the server receives the voice data in streaming format and converts it into text data using the Google Cloud Speech-to-Text API. The converted text data is stored in a Hadoop cluster.

[1643] Next, the server analyzes the accumulated text data using a generative AI model (GPT-4). The AI ​​model extracts frequently used keywords and phrases from the text data and identifies the user's needs and problems. Based on the analysis results, the server optimizes the service provision to the user. For example, if a certain keyword is frequently used, it automatically suggests services related to that keyword.

[1644] Examples

[1645] As a concrete example, consider the case where a user calls customer support and says, "The product was broken and I want to return it. What should I do?" The contents of this call are captured in real time and converted into text data using the Google Cloud Speech-to-Text API. The data is then stored in a Hadoop cluster and analyzed by a generative AI model (GPT-4).

[1646] As a result of the analysis, it is found that keywords such as "broken" and "return" are frequently used. Based on this information, the server can provide the user with information on the return procedure and a service to exchange the broken product.

[1647] Examples of prompt statements

[1648] Example prompt:

[1649] Please enter the user's call details in the following format:

[1650] "The item was broken and I want to return it. What should I do?"

[1651] By inputting this prompt sentence into the generative AI model, the AI ​​proposes an appropriate service provision method. For example, it instructs the server to take specific actions such as "providing detailed information on the return procedure" or "arranging for a replacement product." The flow of the specific process in the third embodiment will be explained with reference to FIG. 15.

[1652] Program processing flow

[1653] Step 1: Capture the call

[1654] The server captures the contents of the call between the user and the terminal in real time. When the call starts, the server receives the voice data in streaming format through SIP communication. The input is the user's voice data, and the output is the voice data sent to the server.

[1655] Specific behavior:

[1656] A user calls customer support.

[1657] The server captures the contents of the call using SIP communication.

[1658] Step 2: Convert audio to text

[1659] The server converts the received voice data into text data using the Google Cloud Speech-to-Text API. The input is the voice data sent to the server, and the output is the text data.

[1660] Specific behavior:

[1661] The server sends the audio data to the Google Cloud Speech-to-Text API.

[1662] The API converts the audio data into text data.

[1663] The server receives the converted text data.

[1664] Step 3: Accumulating text data

[1665] The server accumulates the converted text data in the Hadoop cluster. The input is the text data, and the output is the text data stored in the Hadoop cluster.

[1666] Specific behavior:

[1667] The server sends the text data to the Hadoop cluster.

[1668] The Hadoop cluster stores the text data.

[1669] Step 4: Analyze the text data

[1670] The server analyzes the accumulated text data using a generative AI model (GPT-4). The input is the text data stored in the Hadoop cluster, and the output is keywords and phrases as the analysis results.

[1671] Specific behavior:

[1672] The server retrieves the text data from the Hadoop cluster.

[1673] The server inputs the text data into the generative AI model.

[1674] The AI ​​model analyzes the text data and extracts frequently used keywords and phrases.

[1675] The server receives the analysis results.

[1676] Step 5: Optimize service delivery

[1677] The server optimizes the service provided to the user based on the analysis results of the AI ​​model. The input is keywords or phrases as the analysis results, and the output is the optimized service provided.

[1678] Specific behavior:

[1679] The server identifies the user's needs based on the analysis results.

[1680] The server suggests appropriate services based on the identified needs.

[1681] For example, if keywords such as "broken" and "return" are frequently used, the server will provide the user with guidance on return procedures and a service for replacing broken products.

[1682] (Application example 3)

[1683] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1684] With conventional telephone services, it was difficult to effectively utilize the contents of phone calls to optimize the provision of services to users. In addition, it was not possible to propose content in real time based on the contents of phone calls, making it difficult to provide services that meet the needs of users.

[1685] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means. In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope not infringing the Telecommunications Business Act, a means for providing advanced services using BigData and AI, and a means for analyzing the contents of the call in real time and automatically proposing content based on the contents after the call ends. This makes it possible to propose content in real time based on the contents of the call.

[1686] "SIP communication" is a communication protocol for making voice and video calls using the Internet Protocol.

[1687] "Telephone service" refers to a communication service for providing voice calls.

[1688] "Call content" refers to the audio information spoken during a telephone call.

[1689] "Intangible information assets" refers to information or data stored in digital form.

[1690] The Telecommunications Business Act is a law concerning the operation of telecommunications businesses.

[1691] "Service Delivery Infrastructure" means the technical and physical infrastructure for delivering a service.

[1692] "Big Data" refers to large amounts of data and the technology to extract useful information by analyzing it.

[1693] "AI" stands for artificial intelligence, a technology that allows machines to imitate human intelligence and learn and reason.

[1694] An "advanced service" is a service that provides higher added value than conventional services.

[1695] "Real-time analysis" means processing data instantly and providing results immediately.

[1696] "Content" means the information and entertainment provided to Users.

[1697] "Automatic suggestion" means that the system provides appropriate information or services without user interaction.

[1698] The system for carrying out this invention is configured as follows: The server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, and a means for analyzing the contents of the call in real time and automatically proposing content based on the contents after the call is ended.

[1699] First, when a user starts a call, the terminal makes the call using SIP communication. Voice data during the call is sent to the server in real time, and the server stores this voice data as an intangible information asset. Next, the server converts the voice data into text using voice recognition software (for example, Google's voice recognition API). This text data is stored as BigData.

[1700] The server analyzes the stored text data using AI (e.g., natural language processing models). As a result of the analysis, certain keywords and call patterns are extracted. Based on this, the server generates content to optimize the provision of services to users. For example, if a user talks about movies during a call, the server will suggest reviews, trailers, viewing links, etc. related to the movie.

[1701] As a concrete example, if a user talks about "recent movies" during a call, the server can extract keywords such as "movies," "reviews," and "trailers," and suggest related content to the user based on those keywords.

[1702] Here is an example of a prompt to be input to the generative AI model:

[1703] "When a user is on a call and they're talking about 'recent movies,' extract keywords to suggest related content."

[1704] In this way, real-time content suggestions based on the contents of the call are possible, and services that meet the needs of the user are provided.

[1705] The flow of the specific process in the application example 3 will be described with reference to FIG.

[1706] Step 1:

[1707] When a user initiates a call, the terminal uses SIP communication to make the call. The input is the user's voice, and the output is the voice data sent to the server in real time. The terminal uses a microphone to capture the voice and sends it to the server via the SIP protocol.

[1708] Step 2:

[1709] The server stores the received voice data as an intangible information asset. The input is the voice data sent from the terminal, and the output is the voice data stored on the server. The server uses a database to store the voice data.

[1710] Step 3:

[1711] The server uses speech recognition software (e.g. Google's speech recognition API) to convert the voice data to text. The input is the stored voice data, and the output is the text data. The server sends the voice data to the speech recognition API and receives the text data in return.

[1712] Step 4:

[1713] The server stores the text data as BigData. The input is the text data obtained from the speech recognition software, and the output is the text data stored on the server. The server uses a database to store the text data.

[1714] Step 5:

[1715] The server analyzes the stored text data using AI (e.g., a natural language processing model). The input is the stored text data, and the output is the extracted keywords and call patterns. The server inputs the text data into the natural language processing model and extracts keywords and call patterns.

[1716] Step 6:

[1717] The server generates content to optimize service provision to users based on the extracted keywords and call patterns. The input is the extracted keywords and call patterns, and the output is the generated content. The server uses a generative AI model to generate relevant content.

[1718] Step 7:

[1719] The server proposes the generated content to the user. The input is the generated content and the output is the content provided to the user. The server transmits the content to the user's terminal and the user receives the content through the terminal.

[1720] In this way, real-time content suggestions based on the contents of the call are possible, and services that meet the needs of the user are provided.

[1721] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1722] "Example 1"

[1723] One embodiment of the present invention is a system that combines an emotion engine with a telephone service using SIP communication. This system recognizes emotions from the user's voice during a call and treats the information as an intangible information asset. Specifically, emotions are analyzed from the user's tone of voice, speaking style, choice of words, etc., and the results are stored in a database. This emotion data is used to increase user satisfaction and improve services.

[1724] "Example 2"

[1725] Another embodiment of the present invention is a system that uses an emotion engine to analyze the emotional state of a user and uses the results as reference information for providing services. Specifically, when a user expresses an emotion such as joy, anger, or sadness, a service is provided based on that information. For example, when a user expresses anger, a response such as contacting customer support is made based on that information.

[1726] "Example 3"

[1727] Furthermore, as another embodiment of the present invention, there is a system that optimizes the provision of services to a user based on the emotion recognition result by the emotion engine. Specifically, the system provides optimal services according to the user's emotional state. For example, if the user expresses an emotion of joy, the system responds by recommending related services or products based on that information.

[1728] The process flow of each embodiment will be described below.

[1729] "Example 1"

[1730] Step 1: A user initiates a telephone call using SIP communication.

[1731] Step 2: The emotion engine recognizes emotions from the user's voice during the call.

[1732] Step 3: The emotion information recognized by the emotion engine is stored in a database.

[1733] "Example 2"

[1734] Step 1: A user initiates a telephone call using SIP communication.

[1735] Step 2: The emotion engine analyzes the user's emotional state during the call.

[1736] Step 3: Use the analysis results as reference information for service provision.

[1737] "Example 3"

[1738] Step 1: A user initiates a telephone call using SIP communication.

[1739] Step 2: The emotion engine analyzes the user's emotional state during the call.

[1740] Step 3: Optimize service provision to users based on the analysis results.

[1741] Example 1

[1742] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1743] Conventional telephone service systems have limited means of effectively utilizing call content, making it difficult to digitize call content for analysis and storage. It is also difficult to recognize user emotions in real time and provide services based on them. Another issue is building a new service provision infrastructure without violating the Telecommunications Business Act. To solve these issues, a system is needed that treats call content as an intangible information asset and provides advanced services that utilize BigData and AI.

[1744] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1745] In this invention, the server includes a means for providing telephone services using SIP communication, a means for treating the contents of the call as an intangible information asset, a means for converting the contents of the call into digital data and analyzing and storing the data, a means for constructing a new service provision infrastructure within the scope of the Telecommunications Business Act, a means for providing advanced services using BigData and AI, a means for recognizing emotions from the voice of a user during a call and treating the information as an intangible information asset, and a means for analyzing emotions from the tone of the user's voice, speaking style, choice of words, etc., and storing the results in a database. This makes it possible to effectively utilize the contents of the call and optimize service provision by recognizing the user's emotions in real time.

[1746] "SIP communication" is a communications technology for establishing and managing voice and video calls using the Session Initiation Protocol.

[1747] "Telephone service" refers to a communication service for providing voice calls.

[1748] "Intangible information assets" are information data that have value even though they do not have a physical form.

[1749] "Digitization" is the process of converting information such as audio and video into digital data.

[1750] "Analysis" is the process of examining data in detail to find meaning and patterns.

[1751] "Preservation" is the process of recording and storing data for later use.

[1752] The Telecommunications Business Act is a law concerning the operation and regulation of telecommunications businesses.

[1753] "Service provision infrastructure" means the technical and operational infrastructure for providing a service.

[1754] "Big Data" refers to large amounts of data, as well as the technologies and methods for collecting, storing, and analyzing that data.

[1755] "AI" is an abbreviation for artificial intelligence, a technology that allows machines to imitate human intelligence to learn and make inferences.

[1756] An "emotion engine" is software that analyzes and recognizes emotions from voice and text.

[1757] "Tone" refers to characteristics of voice such as pitch, intensity, and sound quality.

[1758] A "database" is a system for systematically storing and managing data.

[1759] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1760] This invention is a system that provides telephone services using SIP communications and treats the contents of phone calls as intangible information assets. This system digitizes the contents of phone calls, analyzes and stores the data, and makes effective use of the contents of phone calls. It also has the function of building a new service provision infrastructure within the scope of the Telecommunications Business Act. It also has the function of providing advanced services that utilize BigData and AI.

[1761] Hardware and Software Used

[1762] Hardware: Servers, terminals

[1763] Software: SIP communication protocol, database management systems (e.g. MySQL), AI analysis tools (e.g. TensorFlow), emotion engines

[1764] Specific operation of the system

[1765] 1. Start a call:

[1766] A user uses a terminal to make a call via SIP communication.

[1767] The terminal sends a call request to the server using the SIP protocol.

[1768] The server accepts the call request and establishes the call.

[1769] Specific behavior:

[1770] The user enters a phone number into the dial pad of the smartphone and presses the call button.

[1771] The terminal sends a SIP INVITE message to the server.

[1772] The server responds with a SIP 200 OK message and the call is established.

[1773] 2. Digitization of call contents:

[1774] The server captures the contents of the call as audio data in real time.

[1775] The server uses speech recognition software (e.g. Google Speech-to-Text API) to convert the voice data into text data.

[1776] Specific behavior:

[1777] The server captures the audio stream of the call and sends it to the speech recognition software.

[1778] Speech recognition software converts the speech "Hello, how can I help you today?" into text.

[1779] 3. Data storage:

[1780] The server stores the converted text data in a database.

[1781] The server manages the data using a database management system (e.g. MySQL).

[1782] Specific behavior:

[1783] The server uses the INSERT statement to save the text data "Hello, how are you today?" into the MySQL database.

[1784] 4. Sentiment analysis:

[1785] The server uses an emotion engine to analyze emotions based on the user's tone of voice, speaking style, choice of words, etc.

[1786] The server stores the analysis results in a database.

[1787] Specific behavior:

[1788] The server sends the text data "Hello, how are you today?" to the emotion engine.

[1789] The emotion engine analyzes it as a "neutral emotion" and stores the result in a database.

[1790] 5. Leverage data:

[1791] The server provides new services to users based on the analysis results.

[1792] The server refers to the user's emotional data and takes appropriate action.

[1793] Specific behavior:

[1794] The server will provide standard customer support to users whose sentiment is analyzed as "neutral."

[1795] If a server is analyzed as "unhappy," an alert is sent to customer support to promptly respond.

[1796] 6. Advanced service provision:

[1797] The server uses BigData and AI to accumulate and analyze data on call content to optimize service provision.

[1798] The server uses AI analytics tools (e.g. TensorFlow) to analyze the data and identify patterns and trends.

[1799] Specific behavior:

[1800] The server feeds a large amount of call data into TensorFlow and trains a model to identify common issues.

[1801] The server uses the trained model to analyze new call data in real time to help improve the service.

[1802] Examples of prompt statements

[1803] Examples of prompt sentences to be input to a generative AI model include the following:

[1804] "Create a program that saves the contents of a user's phone call as text data and analyzes the emotions using an emotion engine."

[1805] "Please design a system that will optimize the services provided to users by accumulating call content as BigData and analyzing it using AI."

[1806] In this way, a system is realized that provides advanced telephone services in cooperation with users, servers, and terminals.

[1807] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1808] Step 1:

[1809] Starting a call

[1810] Input: A user uses a terminal to enter a phone number and presses call.

[1811] Specific operation: The user enters a phone number on the dial pad of the smartphone and presses the call button. The device sends a SIP INVITE message to the server.

[1812] Data processing: The terminal sends a call request to the server using the SIP protocol.

[1813] Output: The server returns a SIP 200 OK message and the call is established.

[1814] Step 2:

[1815] Digitization of call contents

[1816] Input: The server receives the audio data of the call.

[1817] What happens: The server captures the audio stream of the call and sends it to speech recognition software.

[1818] Data processing: The server uses voice recognition software (e.g. Google Speech-to-Text API) to convert the voice data into text data.

[1819] Output: Speech recognition software converts the speech "Hello, how is your day?" into text.

[1820] Step 3:

[1821] Data storage

[1822] Input: The server receives the converted text data.

[1823] Specific operation: The server uses the INSERT statement to save the text data "Hello, how are you today?" into the MySQL database.

[1824] Data processing: The server manages the data using a database management system (e.g. MySQL).

[1825] Output: The text data is stored in a database.

[1826] Step 4:

[1827] sentiment analysis

[1828] Input: The server receives the stored text data.

[1829] Specific operation: The server sends text data "Hello, how are you today?" to the emotion engine.

[1830] Data processing: The emotion engine analyzes emotions from the user's tone of voice, speaking style, choice of words, etc.

[1831] Output: The sentiment engine analyzes it as “neutral sentiment” and stores the result in the database.

[1832] Step 5:

[1833] Leveraging Data

[1834] Input: The server receives the sentiment analysis results.

[1835] Specific Action: Provide standard customer support to users whose server has analyzed the user as having a "neutral sentiment".

[1836] Data processing: The server refers to the user's emotional data and takes appropriate action.

[1837] Output: The server sends an alert to customer support to promptly respond.

[1838] Step 6:

[1839] Advanced service provision

[1840] Input: The server receives a bunch of call data.

[1841] How it works: A server feeds a lot of call data into TensorFlow and trains a model to identify common issues.

[1842] Data processing: The server uses AI analysis tools (e.g. TensorFlow) to analyze the data and identify patterns and trends.

[1843] Output: The server uses the trained model to analyze new call data in real time and help improve the service.

[1844] (Application example 1)

[1845] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type termina...

Claims

1. A means for providing a telephone service using SIP communication; A means of treating the contents of telephone calls as intangible information assets; A means to digitize the contents of phone calls and analyze and store that data; A means to build a new service provision infrastructure without violating the Telecommunications Business Act; and A means for providing new services to users based on call content data; A means to provide advanced services utilizing BigData and AI; A means for capturing the contents of a call in real time and storing it as audio data; A means for converting the voice data into text data using voice recognition software; A means for storing the converted text data in a database; A means for analyzing the accumulated data using an AI model; A means for providing services to users based on the analysis results; A system including:

2. 2. The system according to claim 1, wherein the telephone service using the SIP communication has a function for identifying a specific call pattern.

3. The system according to claim 1, wherein the advanced service utilizing BigData and AI has a function of optimizing service provision to a user based on call content.

Images