Remote service support device, system, method, and program

Abstract

To solve a problem where a user go to a site and has to explain a thing the user is embarrassed about or does not wish another person to know about, or worse, has to explain the thing to a plurality of persons over and over again, but still cannot find right service, cannot make a reservation, is often dissatisfied with a service result, and answers the user gets through a GPT (chat) are often unreliable and unsatisfactory, when receiving services to achieve a wish of the user or solve a problem.SOLUTION: A remote service support device or a system supports a user in solving a wish or a concern in order to resolve an issue. The remote service support device or a system includes a conversation service unit including a counseling input / output unit and a counseling determination storage unit, and a matching unit including a matching concierge.SELECTED DRAWING: Figure 1a

Description

Technical Field

[0001] The present invention relates to a remote service support device, system, method, and program for assisting in solving users' wishes and troubles.

Background Art

[0002] Communication network technologies such as 5G, AI (artificial intelligence) technologies, and medical technologies have been developing. As applications thereof, EC (e-commerce) / online store technologies famous for Amazon, IOT (Internet of Things) technologies, VR (Virtual Reality) technologies, and AR (Augmented Reality) technologies have also been developing. The practical application of the metaverse, which is a virtual space using these, and the practical application research of telemedicine technologies are also progressing. On the other hand, global diseases such as COVID-19 have been spreading, and worries and anxieties have been rapidly increasing. Going out, large-scale meetings / events, and face-to-face conversations are restricted. To receive a problem-solving service (including service care), one has to go to a place that is embarrassing or not wanting to be known in public, which takes time, costs money, and may even be dangerous. At worst, one has to repeat the same thing many times. Often, an appropriate service cannot be found, or even if it is found, a reservation cannot be made, or the service result is not satisfactory. It takes time and is inefficient. We have been developing AI technologies that listen with empathy and love on the Internet, and as one talks and confesses without covering up, one will naturally notice and solve one's worries and problems. Regarding EC (electronic commerce, electronic market / shopping street), collaborative learning technologies that provide purchase information of other users with similar preferences have also been developed. Google Search, which displays related sites when a keyword is entered, and especially recently, ChatGPT, which returns an answer when a question sentence is entered, have also been developed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Non-Patent Literature

[0004]

Non-Patent Literature 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] For service care ranging from mental trouble solving to physical treatment, and even for general problem-solving and wish-fulfillment services, people often have to go out in person to a place where they feel ashamed or do not want to be known, and at worst, be told things repeatedly, unable to find a service that seems appropriate, unable to make a reservation even if they find one, or be dissatisfied with the service result. Thinking about it carefully, there are even cases where people receive or provide unnecessary services. To provide high-quality services at low cost and succeed in business without having to go to the actual location, it is necessary to provide high-quality and inexpensive remote services on a virtual space that is integrated with the real space such as the metaverse. However, that's not all. It is also important to accurately listen to users' requests (needs) without leaving out any personal troubles or wishes, to be able to meet those needs with only the current situation and existing services or to notice new requests on one's own, and to efficiently find high-quality remote services on the metaverse that match those needs. This is crucial for customer satisfaction and is the basis for business success and marketing. Also, it is important to establish, maintain, and improve the matching and service methods based on the evaluation results. Although Google Search, which displays related websites when keywords are entered, and ChatGPT, which returns answers when a question sentence is entered, have been developed, users need to come up with the keywords or question sentences that should be entered to solve problems. It is necessary to repeatedly perform so-called surfing until satisfactory answer sentences or data are obtained from the websites of the output URLs, which is extremely time-consuming and laborious and inefficient. It is often difficult to obtain a sufficiently satisfactory, appropriate, or useful solution. In many cases, the necessary information cannot be obtained. It is necessary to arrange answer sentences such as those of ChatGPT, a recently popular generative AI that automatically generates answers, for personal use or at least evaluate and correct whether they are appropriate for oneself. Since ChatGPT learns from a large amount of data, it is difficult to provide information after product release. In addition to the need to specify various conditions such as roles and fields, it is not easy to make specifications to obtain a satisfactory solution. It is also not easy to evaluate whether the solution is satisfactory, optimal, or appropriate for the surrounding area, region, or society for the individual. Evaluation is required not only by the individual but also by humans, including sales representatives, local stores, and service experts. Regarding social soundness, local or individual evaluation and correction by administrative experts, including sensible people and regions, and a database of knowledge such as laws are required. In addition, it is necessary to achieve efficient high-reliability by automating the evaluation and correction of these questions and answers using an inference machine or AI that memorizes (successful cases of obtaining highly evaluated answers) past cases or learns and trains based on these cases to compensate for the shortage of personnel. That is, an object of the present invention is to provide a highly reliable and highly efficient remote service support device, system, method, and program that use a web searcher such as Google Search or a generative AI such as ChatGPT to achieve high user satisfaction and social soundness.

Means for Solving the Problems

[0006] The parts that humans should consider are promoted by VICA's listening and in-depth exploration of the listening-type conversation service, enabling users to become aware of them through self-reflection, thereby enhancing efficiency and reliability. This aims to solve problems related to the selection, creation, output editing, evaluation, improvement, and verification of the input language (search keywords for exploration and search) and input sentences (question sentences and their auxiliary sentences, such as sentences specifying the field of questions and the role in the answer) of the above-mentioned Google Search and ChatGPT.

[0007] To solve problems, it is important to provide a listening-type conversation AI counseling system in a virtual space, allowing users to open up about their personal worries and wishes to a machine rather than a human without going out, accurately capturing user requirements (needs) without omission, and efficiently finding high-quality remote services on the metaverse that match those needs, including the user's own awareness of whether the current situation and existing services can meet those needs. To efficiently find remote services on the metaverse, generative AIs such as ChatGPT and web search engines such as Google Search can be used. To improve the quality of search results, it is necessary to evaluate multiple search results and select the ones with high evaluation values, but it is important to accurately grasp user requirements (needs). As a listening-type conversation AI counseling system for this purpose, for example, there is the listening-type conversation service (AI) system: VICA, which is a counseling AI that can conduct listening-type conversations or dialogues in natural language through an avatar. Install this in a virtual space called the metaverse, listen to user needs and problems in achieving them through listening-type conversations, record (log) listening-type conversation data, and support users' spontaneous awareness through self-reflection to clarify and specify their wishes, problems, and worries and solve them. When spontaneous awareness through listening-based conversations cannot solve problems, such as when satisfaction cannot be achieved with only existing or previously answered services, questions are generated from the logs of listening-based conversations (data) and sent to generative AIs like ChatGPT or web search engines to remotely match with services in the virtual space on the web. When sufficient answers cannot be obtained, an expert may assist with matching at the human-machine dialogue section 4, but basically, the matching service can be automatically searched for. Users do not need to go out locally and repeatedly talk about their wishes, needs, and problems in achieving them. Once the answer of the matching service is returned, awareness support is obtained through VICA's listening-based conversation to clarify and specify whether there are still problems with the service in question. If new problems that cannot be solved independently are noticed, the automatic search for other services in the virtual space for the new problems can be repeated until a satisfactory answer is obtained.

[0008] VICA does not ask or answer questions about 5W1H (who, what, where, when, why, how), but listens for worries and wishes by offering unconditional positive interest and empathetic statements based on parroting and in-depth exploration. To adhere to such listening (dialogue), one can talk with an open heart. Since AI (artificial intelligence) is not human, it is easy to talk to even shy people.

[0009] VICA creates a dialogue-based counseling response that adheres to the above listening, outputs a listening-based conversation response (dialogue output) to users or patients via clients, and creates and saves a listening-based conversation log (dialogue log) in a file.

[0010] If the user's wishes are not fulfilled by VICA, match the customer (user / patient) requests that summarize the conversation log with VICA and the services on the metaverse (service shops including hospitals and pharmacies in the case of medical care). Repeatedly search for, implement, and evaluate more optimal and available services, including those involving humans, until customer satisfaction is achieved. Improve the services (implementation) and the matching method based on the evaluation results. In particular, regarding the establishment, maintenance, and improvement of the matching and service methods, consider the iterative integration method of a listening-type conversation service with the following characteristics and a matching concierge that utilizes generative AI and an evolutionary digital twin.

[0011] That is, highly reliable search and matching of products and services for wish fulfillment and problem-solving are made highly reliable through the following iterative loop. 1) Listening-type conversation input / output for the concretization, clarification, self-confirmation, and awareness support of the user's wishes and troubles by the listening-type conversation AI counseling system VICA, and 2) When self-resolution is not possible, conversion of the user's wishes and troubles into questions for generative AI (including ChatGPT) and acquisition of answers, 3) Evaluation and correction of those questions and answers, including human-machine cooperation, 4) Learning of the evaluation and correction methods of questions and answers by a large language model (especially local, that is, localized to the services or shops within the scope of use and exploration including the user's wish / trouble field, that time period, region, and preferences), and evaluation and correction of questions and answers by inference using the learning results, 5) Confirmation, clarification, and concretization of problems by returning to the conversation rally with VICA in 1) to check whether there are any problems with the answer that meets the evaluation criteria, that is, whether the user can be satisfied, and if self-resolution is not possible, repetition from 2).

[0012] Based on the above, the present invention includes the following. [Item 1] A remote service support device for assisting in solving the user's wishes or troubles, The remote service support device includes a conversation service unit including a counseling input / output unit and a counseling judgment memory unit, and a matching unit including a matching concierge. The counseling input / output unit receives user information including voice, image, or text information regarding the wishes or worries of the user, converts it to text in the case of voice or image, and passes the user text information with all the user information converted to text to the counseling judgment memory unit. transmits the conversation response information from the counseling judgment memory unit to the user using a speaker, a display, or an external device. The counseling judgment memory unit generates a natural language text response for promoting self-awareness and problem-solving necessary for the user to solve the problem with respect to the user text information from the counseling input / output unit. stores a conversation log including the user text information. judges whether the wishes or worries of the user have been solved based on the words and emotional change information of the stored conversation log. When it is determined that it is difficult to solve the wishes or worries of the user, the conversation log regarding these wishes or worries is passed to the matching concierge of the matching unit. The matching concierge asks the generation AI or the web searcher about the conversation log received from the counseling judgment memory unit. In the counseling judgment memory unit, a confirmation sentence is added to the answer returned from the generation AI or the web searcher and returned to the counseling input / output unit, a remote service support device. [Item 2] A remote service support system including a counseling conversation client including a counseling input / output unit, a counseling server including a counseling judgment memory unit, and a matching server including a matching concierge, The counseling input / output unit In the case of voice or images related to the user's wishes or troubles, convert them into text, pass the user text information with all the user information in text form to the counseling judgment storage unit, and transmit the conversation response information from the counseling judgment storage unit to the user using a speaker, a display, or an external output device. The counseling judgment storage unit Generates a natural language text response for promoting self-awareness and solution to help the user solve problems for the user text information from the counseling input / output unit. Stores a conversation log including user text information. Judges whether the user's wishes or troubles have been resolved based on the words and emotional change information in the stored conversation log. When it is determined that the user's wishes or troubles are difficult to solve, pass the conversation log related to these wishes or troubles to the matching concierge of the matching unit. The matching concierge Asks the generated AI or web searcher questions about the conversation log received from the counseling judgment storage unit. In the counseling judgment storage unit, add a confirmation sentence to the answer returned from the generated AI or the web searcher and return it to the counseling input / output unit, a remote service support system. [Item 3] Further includes a matching intelligent server including an evaluation and improvement intelligent unit, and the evaluation and improvement intelligent unit evaluates or corrects the answers of the generated AI and the web searcher. The evaluation and improvement intelligent unit is provided with knowledge or a database used for evaluating or correcting the answer, the remote service support system according to [Item 2]. [Item 4] Further includes a matching dialogue client equipped with a man-machine dialogue unit, and the man-machine dialogue unit supports the evaluation or correction by an expert of the conversation log passed to the generated AI or the web searcher, or the answer returned from the generated AI or the web searcher. The remote service support system according to [Item 2]. [Item 5] The evaluation and improvement intelligent unit has a digital twin that simulates human judgment or actions as an avatar. In the digital twin, perform learning or training related to the method and memory of evaluation or modification by the expert, Automatically extract and evaluate the response of the generation AI or the web searcher using the results of learning or training, and correct it if the evaluation is poor. The remote service support system according to claim 3. [Item 6] The remote service support system according to [Item 5], wherein the digital twin has a learning machine including a large language model LLM, and the learning or training is performed by the learning machine. [Item 7] When the counseling judgment memory unit determines that a wish or worry can be resolved from the user's conversation response information for the response and the confirmation text, reserve, execute, or evaluate the service of the response. The remote service support system according to [Item 2]. [Item 8] A remote service support method using the remote service support device according to [Item 1] or the remote service support system according to any one of [Item 2] to [Item 7], In the counseling input / output unit, receive a user utterance including voice, image, or text information, convert it to text in the case of voice or image, and pass the user text information with all the user utterances converted to text to the counseling judgment memory unit; Determine by the counseling judgment memory unit whether the user has a problem including a wish or worry from the user text information, and whether the wish or worry can be solved; When it is determined that the problem can be solved, generate a natural language text response including paraphrasing, concretization promotion, or relationship inquiry with the previously uttered words for a listening-type conversation for promoting self-awareness or self-discovery, and pass the natural language text response to the counseling input / output unit; Transmit the natural language text response to the user by the counseling input / output unit using a speaker, a display, or an external device; In the counseling judgment memory unit, store the conversation history including the wish or worry as a conversation log; Determining whether the wish or worry has been resolved based on the words and emotional change information in the conversation log, and ending the conversation service if it has been resolved; In the step of determining whether the wish or worry can be resolved, if it is determined that it has not been resolved, the counseling judgment storage unit passes the conversation log to the matching concierge and requests problem-solving support; The step of the matching concierge creating a question including the phrases of the conversation log; The step of the matching concierge asking the generative AI or web searcher the question and retrieving the answer; The step of the evaluation and improvement intelligence unit evaluating the answer; In the evaluation and improvement intelligence unit, when the evaluation value of the answer is equal to or higher than the specified value, returning the answer to the user together with the confirmation text via the counseling judgment storage unit and the counseling input / output unit; In the evaluation and improvement intelligence unit, when the evaluation value of the answer is less than the specified value, repeating the evaluation or correction within the specified number of times until the evaluation value becomes equal to or higher than the specified value, or passing it to the man-machine dialogue unit and having an expert evaluate or correct the question or the answer so that the evaluation value becomes equal to or higher than the specified value; If, despite performing the evaluation or correction the specified number of times, the evaluation value is still less than the specified value, in the evaluation and improvement intelligence unit, determining that the one with the highest evaluation value or the one selected by the expert is the answer; In the evaluation and improvement intelligence unit, storing or learning the history of the question or the answer in order to automatically create, evaluate, or correct the question and the answer; If the counseling judgment storage unit determines that the user who has received the answer cannot solve it on their own, continuing the conversation aimed at self-resolution; If the user who has received the answer can solve it on their own, assisting with the reservation, execution, or evaluation of the corresponding service; If the result of the reservation or execution is obtained, returning to the user the answer including the comment on this result and the confirmation text via the counseling judgment storage unit and the counseling input / output unit; If there are any concerns about the answer during the problem confirmation by the user, the conversation service department will resume the conversation to clarify and specify the problem points for self - resolution. If self - resolution is not possible, the matching concierge will be restarted repeatedly until the problem points are eliminated. If self - resolution is achieved, after requesting the matching concierge to reserve, execute, or evaluate the answer and store or learn the history, end the conversation. A remote service support method characterized by including the above. [Item 9] A program for executing the remote service support device described in [Item 1] or the remote service support system described in any one of [Item 2] to [Item 7], In the counseling input / output unit, receive user statements including voice, image, or text information, convert them to text in the case of voice or image, and pass the user text information with all user statements converted to text to the counseling judgment memory unit. The counseling judgment memory unit determines whether the user has a problem including wishes or worries from the user text information, and whether the wishes or worries can be solved. If it is determined that the problem can be solved, generate a natural language text response including paraphrasing, promoting concretization, or asking about the relationship with previously spoken phrases for a listening - type conversation to promote self - awareness or self - discovery, and pass the generated natural language text response to the counseling input / output unit. The counseling input / output unit transmits the natural language text response to the user using a speaker, display, or external device. In the counseling judgment memory unit, store the conversation history including the wishes or worries as a conversation log. Judge whether the wishes or worries have been resolved based on the words and emotional change information in the conversation log, and end the conversation service if they have been resolved. In the step of determining whether the wish or worry can be solved, if it is determined that it has not been solved, the counseling judgment memory unit passes the conversation log to the matching concierge and requests problem-solving support, and the step of the matching concierge creating a question including the phrases of the conversation log, and the step of the matching concierge asking the generative AI or web searcher the question and retrieving the answer, and the step of the evaluation and improvement intelligence unit evaluating the answer, and in the evaluation and improvement intelligence unit, when the evaluation value of the answer is equal to or greater than the specified value, the answer is returned to the user via the counseling judgment memory unit and the counseling input / output unit together with the confirmation text, and in the evaluation and improvement intelligence unit, when the evaluation value of the answer is less than the specified value, evaluation or correction is repeatedly performed within the specified number of times until the value becomes equal to or greater than the specified value, or it is passed to the man-machine dialogue unit and an expert performs evaluation or correction of the question or the answer so that the value becomes equal to or greater than the specified value, and even if the evaluation or correction has been performed the specified number of times and the evaluation value is still less than the specified value, in the evaluation and improvement intelligence unit, it is determined that the one with the maximum evaluation value or the one selected by the expert is the answer, and in the evaluation and improvement intelligence unit, for automatically creating, evaluating, or correcting the question and the answer, the history of the question or the answer is stored or learned, and if the counseling judgment memory unit determines that the user who received the answer cannot solve it by themselves, the step of continuing the conversation for self-solving, and if the user who received the answer can solve it by themselves, the step of assisting in reserving, executing, or evaluating the corresponding service, and if the result of the reservation or execution is obtained, the counseling judgment memory unit generates an answer text including comments on this result and a confirmation text, and returns them to the user via the counseling input / output unit, and If there are any concerns about the answer during the problem confirmation by the user, the conversation service department will reopen the conversation to clarify and specify the problem points for self - resolution. If self - resolution is not possible, the process of restarting the matching concierge will be repeated until the problem points are eliminated, and If self - resolution is successful, after requesting the matching concierge to reserve, execute, or evaluate the answer and store or learn the history, end the conversation, and A program to execute. [Item 10] A remote service support method executed using a client - server system, wherein the client - server system includes, in the client or the server, a counseling input / output unit, a counseling judgment storage unit, and a matching concierge, In the counseling input / output unit, receiving a user's statement including voice, image, or text information, converting it to text in the case of voice or image, and passing the user text information with all the user's statements converted to text to the counseling storage unit, and In the counseling judgment storage unit, determining whether the user has a problem including a wish or a worry from the conversation log of the user text information, and whether the wish or worry is solvable, and When it is determined that the wish or worry is unsolvable, passing the conversation log to the matching concierge and requesting problem - solving support, and The matching concierge creating a question including the phrases of the conversation log, and The matching concierge sending the question to a generative AI or a web searcher and retrieving the answer from the generative AI or the web searcher, and The matching concierge passing the answer to the counseling judgment storage unit. In the counseling judgment storage unit, generating a confirmation statement regarding the answer, and returning the answer and the confirmation statement to the user via the counseling input / output unit. A remote service support method including these steps. [Item 11] The remote service support system further includes an evaluation and improvement intelligence unit, The remote service support method according to [Item 10], including the step of providing an evaluation value to the answer from the generative AI or web searcher in the evaluation improvement intelligent unit. [Item 12] When the evaluation value of the answer is less than the specified value, the matching concierge repeats the evaluation or correction within the specified number of times until it reaches the specified value or more, or includes the step of assisting the expert to evaluate or correct the question or the answer so that it reaches the specified value or more. The remote service support method according to [Item 11]. [Item 13] The remote service support method according to [Item 12], further including the step of storing or learning the history of the question or the answer in the evaluation improvement intelligent unit for automatically creating, evaluating, or correcting the question and the answer. [Item 14] Based on the user's response to the answer and the confirmation text, the counseling judgment storage unit further includes the step of assisting in the reservation, execution, or evaluation of the corresponding service. The remote service support method according to any one of [Item 11] to [Item 13].

[0013] Optionally, it has the following features. Item [a] The remote service support system uses a comprehensive loop to solve the user's problem. The loop includes the awareness support for confirming, specifying, clarifying, and solving problems through the conversation in the conversation service unit, extracting questions and answers from the generative AI or web searcher in the matching unit when self - solution is not possible, man - machine cooperation in the man - machine dialogue unit, and / or simulation of things and human judgment through the digital twin or avatar, evaluation and correction of answers and search results within each part of the simulation, returning to the conversation service unit to check if there is a problem with the answer, and repeating the answer search in the matching unit again when self - solution is not possible. Item [a] The remote service support system has a local learning machine localized for each individual, region, field, and service, for example, including the large language model or DB of the above digital twin, and creates, evaluates, corrects, and stores the answer by using the conversation of the conversation service unit, the dialogue of the matching unit, the simulation evaluation, and the comprehensive iteration / loop within and between the execution / evaluation of the other services, so as to perform the learning / training for the above high reliability always online or offline in advance / regularly to grow, and perform the evaluation / correction of the question / answer by integrating the inference results of the above local learning machine.

Effect of the Invention

[0014] Error! The link is incorrect. As described in [1], 1) the conversation service unit generates a response for promoting awareness necessary for clarifying and specifying the user's wishes and troubles, 2) when it is determined that it is difficult to become aware by oneself based on the user's words and emotional change information, the matching unit is requested for a solution, and 3) the above two are repeated until satisfaction is obtained with the answer obtained by becoming aware by oneself or by requesting the matching unit. As described above, in the sense that the user becomes aware by oneself and obtains satisfaction (thoroughly thinks and is convinced), a highly reliable solution to wishes and troubles becomes possible by the present patented technology. That is, for information obtained from the collective knowledge (Wikipedia) in the virtual space by a web searcher, Google searcher, etc., and answers of AI such as generative AI and ChatGPT that are being regarded as dangerous, in order to promote the awareness of each person by a listening-type conversation, its reliability is enhanced. As a result, upon careful consideration, it reduces the situation where a purchaser buys a product or service with various problems that are unsatisfactory and useless, and in the worst case, socially unhealthy, and becomes disappointed. It contributes to the achievement of the SDGs, and meaningful utilization of AI including this system becomes possible. In addition, when using a server, since the server is connected to the network, the service can be enjoyed without going to the local area, and it has the following merits. Services including counseling and medical treatment can be enjoyed without going to the local area. A listening-type dialogue system can obtain information that is not usually talked about. Because it is AI, even shy people like the Japanese can talk to it about their wishes and seek advice. From the dialogue log / treatment log and its summary, it is possible to efficiently search, reserve, and utilize appropriate service institutions such as (advanced / other types of AI / human) medical care, housing, clothing, food, and grocery stores, including their composite and collaborative bodies, without having to go to the local area. By accumulating dialogue / treatment logs and their summary data, the AI can be automatically created and evolved. For advanced medical care, IoT devices, robots, and avatars can be placed on the patient side client, enabling remote diagnosis / treatment using AR / VR on the metaverse without having to go to an actual hospital, etc. For the information obtained from the collective wisdom (Wikipedia) in the virtual space by web search engines, Google search engines, etc., and the answers of AI such as generative AI and ChatGPT that are being regarded as dangerous, in order to promote each person's discovery through listening-type conversations and enhance its reliability. As a result, upon reflection, it reduces the likelihood of purchasers buying various problematic products and services that are unsatisfactory and useless to them, and in the worst case, socially unhealthy, and being disappointed. It contributes to the achievement of the SDGs and enables the meaningful utilization of AI, including this system. Assuming it is provided as a one-dollar app, a business with a scale of 100 billion yen per month, 1 trillion yen per year, and 100 billion yen per year in Japan alone (10% of the total) is possible with 1 billion contracts.

Brief Explanation of Drawings

[0015]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiments for Carrying Out the Invention

[0016] Hereinafter, a remote service device, system, method, and program according to the first embodiment of the present invention will be described with reference to the drawings.

[0017] [First Embodiment] (High-Efficiency Remote Service Support Device / System) FIG. 1(a) is a functional block diagram showing an example of a remote service support device according to the first embodiment for general wish fulfillment not limited to medical care including counseling. As shown in FIG. 1(a), the high-efficiency remote service support device includes a counseling input / output unit 1 of the conversation service unit, a counseling judgment storage unit 2, a matching concierge unit 3 of the matching unit, a man-machine dialogue unit 4, a (meta) service 5 (service server / client group) on a virtual space including the meta (verse) related to the matching service of this device, an evaluation improvement intelligent unit 6 possessed by the matching concierge 3, a storage device 7 for storing conversation logs, dictionaries, etc. by the conversation service unit, a conversation (voice) recognition server 8 related to the counseling input / output unit 1 of the conversation service unit, a facial expression / image / sensor information recognition server 9, and an IOT input / output device / robot / avatar 10. Note that the IOT input / output device / robot / avatar 10 (these are referred to as external devices) may be provided in the counseling input / output unit 1, or may be provided externally through wired or wireless communication or the Internet. The remote service support device includes control means for controlling a conversation service unit including a counseling input / output unit 1 and a counseling judgment storage unit, and a matching unit including a matching concierge 3. Further, the remote service support device can include at least data transmission / reception means capable of transmitting and receiving data with a generation AI 12 or a Web searcher 11, and further includes storage means for storing conversation logs. Here, the control means is composed of, for example, a CPU or the like. The data transmission / reception means can be any device that can communicate with a server or client computer located remotely. In addition to a wired LAN or a telephone line, it can be a device for wireless communication such as WIFI. It may be a volatile memory that temporarily stores data, a non-volatile memory or a hard disk that stores data long-term, or a storage medium on a network server. In this specification, electronic devices such as various devices and servers all preferably include a control means, a data transmission / reception means, and a storage means. The control means, communication means, and storage means may all be the above specific examples. FIG. 1(b) is a diagram showing that the remote service support device is connected to the Internet and can transmit and receive data with other servers or the like. In FIG. 1(b), an IOT input device, a robot, and an avatar are connected to the remote service support device. However, the IOT input device, the robot, and the avatar do not have to be directly connected to the Internet or directly connected to a computer. Also, the remote service support system of the present invention can be realized by a client-server system.

[0018] The counseling input / output unit 1 (counseling conversation client) inputs the user's speech from a keyboard or a microphone and converts it into a text sentence. If the speech is text data from the keyboard, it remains as it is. If the speech is input from the microphone, the voice data is sent to a conversation recognition server such as Google's voice recognition server and a request is made to convert the voice data into text data or a text sentence, and the obtained text data is converted into a text sentence (if a text sentence is obtained, this is verified, that is, evaluated and corrected). The verified text sentence (user text information) is sent to the counseling judgment storage unit 2. The image recognition server recognizes the expression and body language in the image, converts them into a scale (numerical value, text sentence, or phrase) representing the quality of the emotion, and sends it to the counseling judgment storage unit 2. The data for reservation and execution is only the text information, which is intercepted as history information for evaluation and passed to the counseling judgment storage unit 2, while other media information is directly exchanged with the relevant sites.

[0019] The counseling input / output unit 1 and the man-machine dialogue unit 4, which are components of the remote service support device shown in Fig. 1(a), can be clients on a user's computer with an interface including a web browser. The counseling judgment storage unit 2, the matching concierge 3, and the evaluation and improvement intelligent unit 6 can be servers on a computer that can be remotely connected via the Internet, thereby forming a remote service support system (see Fig. 2(a)). Note that the counseling judgment storage unit 2, the matching concierge 3, and the evaluation and improvement intelligent unit 6 may be provided by one server each, or two or three servers may be equipped with one or more of each unit. Fig. 2(b) is an example showing the connection status of the remote service support system to the Internet, and is a diagram showing that data can be transmitted and received with other servers and the like. The client-server model, which is a model of a typical distributed system in recent years using the Internet, is used for load countermeasures, fault countermeasures, and improvement of expansion and change. It is for highly efficient utilization support of services in a virtual space composed of a computer and a communication network including the metaverse. That is, as shown in Figs. 1 and 2, the remote service support device or system is roughly divided into a conversation service unit and a matching unit. The conversation service unit is composed of a counseling input / output unit 1 (counseling conversation client) and a counseling judgment storage unit 2 (counseling server). The matching unit includes a matching concierge 3 (matching server), a man-machine dialogue unit 4 (matching dialogue client), and an evaluation and improvement intelligent unit 6 of the matching intelligent server. Note that the counseling judgment storage unit 2 and the evaluation and improvement intelligent unit 6 can have a storage unit 7 (which is also a storage server) for storing dialogue logs, dictionaries, and the like. The counseling input / output unit 1, the counseling judgment storage unit 2, the matching concierge 3, the man-machine dialogue unit 4, and the evaluation and improvement intelligent unit 6 can all be executed, for example, by an information terminal or a server equipped with a storage unit such as a CPU and a memory, and communication equipment. In addition, the system can utilize a conversation recognition server 8 for speech / voice recognition, a facial expression / image / sensor information recognition server 9, an IOT input / output device / robot 10 including a microphone for voice input, a camera for image input, a speaker for voice output, and a display for displaying image data such as an avatar, which are directly or network-connected, a web searcher 11, and a generative AI 12 (ChatGPT, GPT4). It is network-connected to a (meta) service 5 (service server / client group) on a virtual space including a metaverse and becomes a target for matching to provide services to users. Each of these servers (computers dedicated to services) can be any of a cloud server, an edge (cloud) server, or a local server. Considering response performance and price, the SDN slicing technology is used to selectively connect these servers, enabling the efficient configuration of a virtual space with good cost performance by a computer (computer) and a network (communication network). SDN is an abbreviation for Software Design Network and is a method of efficiently configuring a network (communication network) of computers (computers) by a program. The following description of the embodiment uses the configuration of FIG. 2 in the client-server model, but it may also be implemented using the configuration as the device in FIG. 1.

[0020] The counseling input / output unit 1 (counseling conversation client) inputs voice / image data, etc. from the user via the IOT input / output device / robot 10, uses the conversation recognition server 8 and the facial expression / image / sensor information recognition server 9 to convert it into text of the uttered sentence and emotional expression, and sends it to the counseling judgment storage unit 2 (counseling server). A client such as a counseling conversation client is an information terminal capable of communication and equipped with a control and storage device, and means, for example, a personal computer, a mobile terminal, etc. Each of the above servers may be any of a cloud, edge, or local server. Using SDN slicing technology (such as core / dedicated / best-effort Internet lines), it is possible to configure and switch the connection between these servers (computers) and the network (communication network). This is done to satisfy the response quality performance measured in real time according to the user's contract conditions (service level). This quality assurance shall also apply to the services of Matching Concierge 3 and Service 5. When the conversation recognition quality deteriorates due to network congestion or the conversation recognition server 8, it is also possible to continue the conversation by entering text from the keyboard of the IOT input / output device / robot 10. That is, the remote service support system of the present invention can be realized by a client-server system. A client-server system is a system in which a computer (client) that uses a service and a computer (server) that provides a service and data operate while communicating via a network.

[0021] The counseling judgment memory unit 2 (counseling server) receives the text sent from the counseling input / output unit 1 (counseling conversation client), and makes unconditional positive interest or empathetic statements that do not depend on context or situation, such as nodding, paraphrasing, etc., dig deeper, such as specifically / more / detailed, summarize feelings and their changes, and when the same words are repeated, ask about the relationship with wishes and problems. That is, it creates a dialogue response text that specifically asks about the user's troubles and wishes and is dedicated to spontaneously noticing solutions. This is returned to the user via the counseling input / output unit 1 (counseling conversation client). Even when asking, a non-invasive basic counseling conversation is carried out with consideration not to directly use expressions such as 5W1H (When, Where, etc.) for the user. This conversation is logged in the dialogue log / dictionary storage device 7. However, in the user satisfaction not reached mode of 0013 below, the text received from the counseling input / output unit 1 (counseling conversation client) is directly sent to the Matching Concierge 3 (matching server).

[0022] When the counseling judgment memory unit 2 (counseling server) determines that the counseling is unsuccessful (in basic counseling where spontaneous problem-solving through listening fails to achieve the resolution of worries, problems, or the fulfillment of wishes, that is, it is unable to solve the problem on its own or has difficulty doing so), it enters the user dissatisfaction (unfulfilled user requirements, unable to solve problems on its own) mode. Next, this mode (symbol) is additionally set in the log and notified to the matching concierge 3 (matching server). In this mode, the counseling judgment memory unit 2 (counseling server) conveys the proposals, explanations, and instructions from the matching concierge 3 (matching server) to the user and conveys the user's responses and data to the matching concierge 3 (matching server). The counseling judgment memory unit 2 (counseling server) determines that the counseling is successful when the user responds with "feeling refreshed", "solved", "got motivated", "seems good" in natural language text or voice conversation, or when the degree of emotion obtained from images, voice, or other sensors changes in the positive direction by a certain value or more. When the conversation continues with responses like "boring" or joking around and then stops, or when there is silence and the response time exceeds a certain time limit, or when the number of conversations exceeds a certain value, or when explicit questions or requests such as interrogative sentences including 5W1H, "want to know XXX", "want to buy XXX" are given, it is determined that the counseling is unsuccessful (that is, unable to solve the problem on its own or has difficulty doing so). This is because it is a listening-based counseling method that does not provide solutions or recommendations (service plans) and instead focuses on deeply listening and allowing the user to realize on their own. The emotional changes are identified from 1) an emotion dictionary in voice conversation responses ("feeling refreshed", "solved" -> success), 2) the recognition and analysis of body language such as facial expressions, gestures, and head movements in images, 3) the intonation, sound quality, and volume of voice, and in other sensors, from pulse, blood pressure, body temperature, etc.

[0023] The matching concierge 3 (matching server) retrieves the log of this conversation from the conversation log / dictionary storage unit 7 and summarizes the user requests and questions. The user requests can include basic information such as the user's ID, URL, and language used, as well as an estimate and upper limit of the costs that can be used in addition to the issues / wishes / hopes.

[0024] When it is determined (or found out) that the above self-reliant (problem) cannot be solved or the user satisfaction is not achieved, the matching concierge 3 (matching server) matches each service in the service 5 on the metaverse with this user request.

[0025] In the matching of the service and the user request, a generative AI 12 (including ChatGPT and GPT), a web searcher 11, or an autonomous decentralized evolutionary intelligence method with the following characteristics is used. The matching is considered as a service trading market. The service is composed of a business level, a service function level, a resource level, etc., and can be modeled so that there is a matching of requests and offers as a market between each level. In this case, the autonomous decentralized evolutionary intelligence method can also be applied to the matching at each of these levels. When a service that can be matched by the generative AI 12 (including ChatGPT and GPT) or the web searcher 11 can be explored, the autonomous decentralized evolutionary intelligence method may not be used (including the following mass transmission). By using both of them, evaluating alternative solutions, repeatedly applying and improving the three methods, or finally selecting through a rally of (counseling) conversation services, a more satisfactory matching for the user becomes possible.

[0026] The autonomous decentralized evolutionary intelligence method is a matching method with the following characteristics. 1) Broadcast (simultaneous transmission) and autonomous / independent response without relying on others; The user request is broadcast with the intention to the services 5 distributed on virtual spaces such as the metaverse. The broadcast is a transmission to all the services (nodes) registered as service 5. When the user request is sent together with the intention data described in 2) below, each service (node) returns this service as a response if it can content-wise respond and is within the allowable range in terms of service type, method, quality, time, cost, etc., and has an available service at its location. This determination is made independently without relation to other services (nodes). This simultaneous transmission (Publishing) and independent registration-based response can be realized by the known PubSub model.

[0027] 2) Integration by intention data; The intention is a business intention such as budget and fee, or a service intention such as service quality and desired time. The matching concierge 3 (matching server) selects the one with the optimal quality, time, cost (weighted maximum value of high quality level and low time and cost) from among the services that have returned responses. Details will be described in the next section.

[0028] 3) Integration of virtual space and physical space by digital twin; Advanced and diverse services exist distributed as service 5 on virtual spaces such as the metaverse. This service includes digital twins of real (spatial) humans and objects (devices, robots) that access (collaboratively) the service nodes on the virtual space to provide services. The digital twin is a digitized object or avatar on the virtual space. It has a simulation program for simulating reality with structured data representing the doubles of real objects on virtual spaces such as the metaverse. When the user request is sent together with the intention data, each service (node) autonomously determines whether it can provide a service that matches the request and intention using the information including the simulator of the digital twin within its own node, without depending on other nodes, and responds with the possible service content. More specifically, a digital twin is a computer object (a combination of data and programs) of a real object such as a building, machine, or clothing that is equipped with simulators for strength against earthquakes, wind, and heavy rain, responsiveness and accommodation during overload (for a container and its contents, or for clothing, the fitting of the product to the customer's height, shoulder width, and waist circumference). In the case of humans and living organisms, it is an avatar. Services include avatars of shops, government agencies, businesses, hospitals, pharmacies, and doctors within them. The availability of services such as land on the metaverse and its tourism can also be determined through the avatars of real estate shops and travel agencies. That is, using the data and simulators, i.e., the intelligence, of these avatars, each service node (shop) can autonomously determine whether it can provide a service that matches the request and intention.

[0029] 4) Utilization of redundancy / encryption technology; Use blockchain and DLT (distributed ledger) to spatially and temporally (horizontally and vertically) disperse, redundantly store, encrypt, and back up the data of services including digital twins. Details will be described in the following sections.

[0030] 5) Utilization of evolutionary intelligence technology; For efficient matching and services, an intelligence method that involves acquiring knowledge from experience and using the acquired knowledge is used. For optimal service matching (searching and selection), intelligence is achieved by using knowledge (expertise) obtained by humans organizing and structuring empirical data and knowledge automatically obtained by well-known data mining methods. Furthermore, to automatically improve the matching and service methods, an evolutionary intelligence method is realized that constructs a learning-based search (inference) device using data analysis AI including neurocomputing. In the evaluation and improvement intelligence unit 6, an evolutionary intelligence method for automatically constructing a search (inference) device through the memory and learning of empirical data is shown in the third embodiment.

[0031] Each service of Service 5 may be vertically, i.e., functionally and hierarchically, distributed. However, the lower-level service(s) will not respond directly to the Matching Concierge 3 (matching server) by skipping the immediate upper level. The immediate upper-level service checks the autonomous (independently judgmental) but individual responses of the lower-level service(s), adjusts, integrates, and summarizes them, and then repeats the process of responding to the upper-level service. Finally, the Matching Concierge 3 (matching server) summarizes each response of the topmost service(s) directly below it, i.e., the topmost service of Service 5. If there is no response, the conditions are relaxed within the allowable range (margin) indicated by the intention, etc. The margin, etc. is the subject of learning from the experience data of the Evaluation and Improvement Intelligence Unit 6. When further relaxing the conditions, the Matching Concierge 3 (matching server) consults with the user via the Counseling Judgment Memory Unit 2 (counseling server). Finally, only the service(s) directly below the Matching Concierge 3 summarize the above and respond to the Matching Concierge 3. The upper-level service adjusts and integrates the individual responses of the lower-level service. To improve the integration efficiency, the service functions to be shared by the lower-level service(s) and the information on the allocated upper limit price for each are added to the message as intention (Intention: service / business intention / information), i.e., intention information, and broadcast (sent all at once, Publish). The intention (information) is basically generated by the higher-level ones in the hierarchy in the Matching Concierge 3 (matching server) and the service, and is sent to the lower-level service. In this way, the intention functions as glue-like control information for efficiently integrating the user requests of the Matching Concierge 3 (matching server), the provided contents of each service of Service 5, the requests of the upper-level service, and the provided contents of the lower-level service. Each service of Service 5 stores and backs up its data in a spatially and temporally (horizontally and vertically) distributed, redundant, and encrypted manner using blockchain or DLT. This enhances security and traceability. Furthermore, each service of Service 5 in Figure 1 that has been contracted and registered is wrapped as a pseudo (simulated) server using blockchain or DLT so as to be subject to the above autonomous distributed method.

[0032] When the counseling judgment memory unit 2 (counseling server) determines that the counseling is unsuccessful (i.e., in this counseling that spontaneously solves problems through listening-type conversations, it has not been successful in solving worries, problems, or achieving wishes), it enters a mode where the user cannot solve the problem on their own, that is, the user satisfaction is not achieved (requirement not met: the user's requirements are not met). This mode (symbol) is additionally set in the log and notified to the matching concierge 3 (matching server). In this mode, the counseling judgment memory unit 2 (counseling server) conveys the proposals, explanations, and instructions from the matching concierge 3 (matching server) to the user, and conveys the user's responses and data to the matching concierge 3 (matching server). In this embodiment, the counseling is determined to be successful, that is, the problem is solved, the worry is solved, or the satisfaction or wish is achieved when the user responds with "feeling refreshed", "solved", "getting motivated", "feeling good" in natural language text or voice conversation, or when the degree of emotion obtained from images, voices, or other sensors changes in the positive direction by a certain value or more. If the conversation or dialogue cannot continue due to "boring" or joking around, or if there is silence and the response time exceeds a certain time limit, or the number of dialogues exceeds a certain value, or if explicit questions or requests such as "want to know XXX" or "want to buy XXX" are given, it is determined to be unsuccessful, that is, the user cannot solve the problem on their own. This is because it is a counseling (consultation) method that does not provide solutions or recommended service plans, but instead focuses on deeply listening and allowing the user to realize it on their own. The emotional changes are identified from 1) an emotion dictionary in voice conversation responses ("feeling refreshed", "solved" -> success), 2) recognition and analysis of body language such as facial expressions, gestures, and head movements in images, 3) intonation, tone quality, and volume of voice, and other sensors such as pulse, blood pressure, and body temperature.

[0033] The matching concierge 3 (matching server) extracts the logs of this conversation or dialogue from the listening-type conversation log (dialogue log) / dictionary storage device 7 and summarizes the user requirements. The user requirements can include basic information such as the user's ID, URL, and language used, as well as an estimate and upper limit of the costs that can be used in addition to the issues / wishes / hopes.

[0034] When the user's satisfaction is not met (or it is found that the user's satisfaction is not met), that is, when the user cannot solve the problem on their own, the matching concierge 3 (matching server) in the matching section uses each service among the services on the virtual space (Web) or the services 5 on the metaverse, as well as Web search engines 11 including Google Search and generative AIs 12 including GPT4 and ChatGPT, to match this user request. Here, since the above self-organizing distributed evolutionary intelligence method is realized by the PubSub model, only services with contracts or registrations can be searched. On the other hand, due to the utilization of redundancy / encryption technologies of blockchain and DLT (distributed ledger), it is safe and secure. The providers / producers of services with contracts or registrations are the subscription-based stores, enterprises, and institutions of this system that pay the contract fees or registration fees. That is, they are the revenue sources of this system. It is important to securely manage information on what services were provided to which customers (users as consumers), at what price, and whether they could be provided. Therefore, in the first embodiment, first, the self-organizing distributed evolutionary intelligence method is used to preferentially search for services with contracts or registrations, that is, subscription-based and important stores and enterprises. If it cannot be solved thereby, that is, if the service to be matched is not found, search using a Web search engine or ChatGPT. Also, the proposal uses the service with the best evaluation (evaluating including whether it is a contracted store, enterprise, or institution and the contract amount) among the searched services as the answer. Although it is less than the specified number, multiple answers are made possible.

[0035] The web searcher 11 extracts a URL (the address of a site in virtual space) where there is text related to the answer to the user's question, that is, the means of achieving / resolving the user's desire / problem. The web searcher is, for example, Google Search or the like. The generative AI 12 extracts text related to the answer to the user's question, that is, the means of achieving / resolving the user's desire / problem, from virtual spaces including the metaverse. A specific example of the generative AI 12 is ChatGPT (registered trademark). The matching unit 3 uses the evaluation and improvement intelligence unit 6 to evaluate and verify the answers of the web searcher (Google Searcher) 11 and the generative AI 12 (ChatGPT) and the matching results of the above-mentioned self-organizing and decentralized intelligence method using a simulator of the evaluation and improvement intelligence unit 6 and a knowledge database including rule bases. The evaluation and verification results are sent to the human-machine dialogue unit 4 (including remote locations connected via the network), and further evaluation and verification are performed through human-machine interaction. The answer that has obtained an evaluation above the specified value (even if there is only one answer with the highest evaluation value) is sent to the conversation service unit as an answer together with comments including the evaluation value and qualitative opinions. The conversation service unit creates a response with an inquiry about whether there are any further problems for the answer and comments, and sends it to the user via its input / output unit 1. Whether the answer can solve the desire or worry will ultimately be evaluated by the user.

[0036] The service (including shops: government agencies, businesses, hospitals, pharmacies, and avatars of doctors therein) corresponding to the answer selected by the user, judging that they are satisfied, that is, there are no problems or concerns, is sent to the human-machine dialogue unit 4 (matching dialogue client) for display and is corrected as necessary. The correction result is transmitted (sent back) to the matching concierge 3 (matching server). The matching results, including corrections, are conveyed to the user from the matching concierge 3 (matching server) via the counseling judgment storage unit 2 (counseling server) and the counseling input / output unit 1 (counseling conversation client). If an approval response is obtained from the user, the matching concierge 3 (matching server) reserves the execution (service execution: commitment) of the selected and approved service.

[0037] When the service is being implemented (service execution: commitment), information such as each instruction and explanation from the server of the service (store) among the services 5 that have been matched, selected, and reserved is successively transmitted to the user via the counseling judgment memory unit 2 (counseling server) and further via the counseling input / output unit 1 (counseling conversation client), and a response is obtained. This is repeated until the service ends. The server of the service (store) may be configured by any of a cloud, edge, or local server. Furthermore, considering the user's contract (service price / level) included in the reservation conditions as a parameter at the time of matching and the response quality performance measured in real time, the network configuration (such as core / dedicated / best-effort type Internet lines) including the above server and its switching are possible using SDN and slicing technologies.

[0038] According to each instruction transmitted from the service, it is also possible to control the IOT devices including the robot connected to the counseling input / output unit 1 (counseling conversation client) to implement the service. The following are considered as IOT devices and can be connected wirelessly. That is, 1) a microphone for dialogue voice input, a camera for image input, especially a camera capable of 3D (three-dimensional) video acquisition as needed for VR (virtual reality) to identify facial expressions and body language, a thermometer, a blood pressure monitor, and other sensor input devices, 2) a speaker for voice output, a display for 3D moving image display effective for realizing the metaverse, and a display for image output including VR goggles also used in game machines, various devices, and actuators for moving them, etc., output devices, and further, 3) general-purpose robots, dedicated robots for nursing / care / guard, etc., 4) water / land / air moving transportation devices, air purifiers, ozone suction masks, sterilizers, 5) amulets / mascot dolls, kits for avatars (avatars or doubles, mementos), and (3D) display machines, etc.

[0039] These are connected to each service client of the service 5 (service server / client group).

[0040] The avatar is displayed on a (ultra) small 3D (three-dimensional) display such as a VR goggle or a normal image display. The avatar is used to realize a virtual space such as a metaverse by being displayed in three dimensions and being a digital twin (a twin object / double in a virtual space that can simulate its operation on the web using a computer). For example, the customer (user, consumer) and the tactile (touch feeling such as texture and fitting like tightness / too tight) simulation model / program obtained by the 3D camera of the counseling input / output unit 1 (counseling conversation client) (user, consumer side) or the service (store side) client are attached to the clothes in that store, and it is possible to try them on in a so-called metaverse virtual space in a VR (virtual reality) manner to see if they match.

[0041] The matching concierge 3 (matching server) takes logs of information such as each instruction and explanation from the service during service implementation and the user's reaction (response) to it, that is, the implementation log (treatment log in the case of medical treatment).

[0042] Using the log of the above instructions including the final result and the user's (patient's in the case of medical treatment) response, the matched service (including the patient's reservation and medical record information in the case of medical treatment), and the information of the user's (patient's) request for evaluation.

[0043] The evaluation result is displayed on the man-machine dialogue unit 4 (matching dialogue client). If the evaluation is poor and it is considered that other services are needed, this is conveyed to the matching concierge 3 (matching server), and matching is performed again, and the execution of other services is repeated.

[0044] The evaluation improvement intelligence unit 6 accumulates the above (matching and service implementation) experience data, associates and acquires and holds it as knowledge, and uses it for the improvement and evolution of services and stores (including AI). Using data analysis AI such as big data analysis from matching and these experience data, an automatic learning / exploration (inference) device for the method of matching services and user requests and the method of the service itself on virtual spaces such as the metaverse is constructed.

[0045] Each service of Service 5 consists of a server (service server) and a client (service client). An IOT device is connected to the service client, similar to the counseling input / output unit 1 (counseling conversation client). For example, as the IOT device, a camera for VR (virtual reality) is connected for input, and a (3D) display or goggles are connected for output. The moving images of a real human counselor, store clerk, or doctor are input as 3D (three-dimensional) avatars by the camera for VR (virtual reality), sent to the service server, where they are made into a digital twin (a computer simulation / double of objects or people in a virtual space), sent to the counseling input / output unit 1 (counseling conversation client), and can be displayed as avatars on the connected (3D) display or goggles. Conversely, a camera for VR (virtual reality) connected to the counseling input / output unit 1 (counseling conversation client) can display the moving images of a real human counselor or doctor as 3D (three-dimensional) avatars on the connected (3D) display or VR goggles connected to the counseling input / output unit 1 (counseling conversation client).

[0046] Hereinafter, a highly reliable and highly efficient remote service system according to a second embodiment of the present invention (peripheral invention) will be described with reference to the drawings.

[0047] [Second Embodiment] (Remote Service Support Device / System) FIG. 2 is a functional block diagram showing an example of a highly reliable and efficient remote service support system according to a second embodiment that particularly uses a generative AI 12 (including ChatGPT and GPT4) and a web searcher 11 (including Google Search) for wish fulfillment. That is, it relates to supporting the efficient use of services 5 within a virtual space / metaverse consisting of a computer including a generative AI 12 and a web searcher 11 and a communication network. Of course, this embodiment is also a more general embodiment for wish fulfillment not limited to medical care (including counseling). However, it mainly describes the implementation of each loop and its integrated loop, including problem discovery / solution and matching through a conversation rally for listening-type counseling in the conversation service department, answer search using the generative AI 12 / web searcher 11 in the matching department, evaluation and correction through a man-machine dialogue rally, acquisition of intelligence for its automation, and its application. Therefore, different from Embodiment 1, services (groups) do not have to be registered. That is, it can be a general service within the virtual space or a site of a shop. It is an example of an integrated loop that emphasizes the independence of the problem discovery / specification / solution loop in the conversation service department, the matching department when self-resolution is not possible, and each of these loops. Details regarding reservation / execution and its evaluation after matching (transmission and reception of service reservation data and instruction / control data during execution, and input / output in the conversation service department) follow the description in Example 1, that is, Embodiment 1.

[0048] As shown in FIGS. 1 and 2, the remote service support device or system is roughly divided into a conversation service department and a matching department. The conversation service department is composed of a counseling input / output unit 1 (counseling conversation client) and a counseling judgment storage unit 2 (counseling server). The matching department includes a matching concierge 3, a man-machine dialogue matching unit 4, an evaluation improvement intelligence unit 6, and a storage unit 7 such as a conversation log / dictionary unit that stores conversation logs, sentiment dictionaries, morphological analysis dictionaries, etc., that is, a listening-type conversation log (dialogue log) / dictionary server.

[0049] It is possible to use the conversation recognition server 8 for speech and voice recognition, either directly or connected via a network, the facial expression / image / sensor information recognition server 9, the IOT input / output device / robot 10 including a microphone for voice input, a camera for image input, a speaker for voice output, and a display for displaying image data such as an avatar, the web searcher 11, and the generative AI 12 (ChatGPT, GPT4). It is network-connected to the (meta) service 5 (service server / client group) on the virtual space including the metaverse and becomes the target of matching to provide services to users.

[0050] Each of these servers (computers dedicated to services) can be any of a cloud server, an edge (cloud) server, or a local server. Considering response performance and price, use SDN slicing technology to selectively connect these servers. In this way, a virtual space with good cost performance is efficiently realized by a computer network, that is, a computer (computer) and a network (communication network). SDN is an abbreviation for Software Design Network and is a method of efficiently configuring a computer (computer) network (communication network) by a program. The services of the (meta) service 5 that are the target of matching include all services such as those related to food, clothing, housing, wholesale, agency sales, administration, medical care, education, management, manufacturing, construction, energy, transportation, etc. It is not only an actual store that provides or sells products and services themselves for users to achieve their wishes and solve problems, but also a web service site including online stores on Amazon and Rakuten Ichiba that introduce, reserve, and implement them. That is, it includes physical and mental services including product offerings on the real space and virtual space (i.e., meta space) other than the conversation service department and the matching concierge 3 of the present application, and the organizations, departments, stores, people, and robots (including AI avatars) that perform them. It is the target (products and services) that users find it difficult to solve on their own only through introspection and self-awareness by conversations and counseling in the conversation service department and need to obtain or purchase from external people and services.

[0051] The counseling input / output unit 1 (counseling conversation client) receives voice / image data, etc. from the user via the IoT input / output device / robot 10, uses the conversation recognition server 8 and the facial expression / image / sensor information recognition server 9 to convert it into a spoken sentence and an emotion expression text, and sends it to the counseling judgment memory unit 2 (counseling server). Each of these servers may be any of a cloud / edge / local server. Using SDN / slicing technology (such as core / dedicated / best effort type Internet lines), it is possible to configure and switch the network connection configuration of these servers and the network (communication network). It is performed so as to satisfy the response quality performance in which the user's contract conditions (service level) are measured in real time. This quality assurance is also performed for the services of the matching concierge 3 and the service 5. When the conversation recognition quality deteriorates due to network congestion or the conversation recognition server 8, it is also possible to continue the conversation by entering text from the keyboard of the IoT input / output device / robot 10. The counseling judgment memory unit 2 (counseling server) receives this text and basically makes unconditional positive interest and empathetic statements that do not depend on context / situation such as nodding and paraphrasing, and moreover, delves deeper such as specifically / more / detailed, summarizes emotions and their changes, and when the same words are repeated, asks about the relationship with wishes and problems, and creates a dialogue response text that focuses on specifically eliciting the user's worries and wishes and spontaneously noticing solutions. This is returned to the user via the counseling input / output unit 1 (counseling conversation client). Even when eliciting information, a non-invasive basic counseling-oriented dialogue is conducted without directly using expressions such as 5W1H (when: When, where: Where, etc.) for the user. This dialogue is logged in the dialogue log / dictionary storage device 7. However, in the user satisfaction unmet mode of 0013 below, the text received from the counseling input / output unit 1 (counseling conversation client) is directly sent to the matching concierge 3 (matching server). In this user satisfaction unmet mode, data is also sent from the matching concierge 3 (matching server), but this is also directly output to the counseling input / output unit 1 (counseling conversation client).Furthermore, in this mode, large-capacity data such as images is directly sent from Service 5 to the Counseling Input / Output Unit 1 (Counseling Conversation Client) and outputted.

[0052] When the Counseling Judgment Memory Unit 2 (Counseling Server) determines that the counseling is unsuccessful (i.e., in this counseling where problems are spontaneously solved through listening, no success has been achieved in resolving worries, problems, or fulfilling wishes), it enters a mode where the user cannot solve the problem on their own, that is, the user satisfaction is not achieved (the user's requirements are not met). In this mode, this mode (symbol) is additionally set in the log and notified to the Matching Concierge 3 (Matching Server). In this mode, the Counseling Judgment Memory Unit 2 (Counseling Server) transmits the history of situation data including the user's wishes, worries, questions, requests, etc., that is, the log of the conversation in the counseling section, to the Matching Concierge 3 (Matching Server). The user is informed of the explanations from the Matching Concierge 3 (Matching Server), that is, the commented answers, proposals, or instructions to execute the matched service. In the counseling of this embodiment, when the user responds with "feeling refreshed", "solved", "got motivated", "feeling good" in natural language text or voice conversation, or when the degree of emotion obtained from images, voices, or other sensors changes in the positive direction by a certain value or more, it is determined as successful, that is, the problem is solved, the worry is resolved, or satisfaction or wish fulfillment is achieved. When the conversation (dialogue) cannot continue because of "boring" or joking around, or when there is silence and the response time exceeds a certain time, or when the number of dialogues exceeds a certain value, or when explicit questions or requests such as "want to know XXX" or "want to buy XXX" are given, it is determined as unsuccessful, that is, the user cannot solve the problem on their own. This is because of the counseling (consultation) method that does not provide solutions or recommended service plans but instead focuses on deeply listening and allowing the user to realize on their own. The emotional changes are identified from 1) in voice conversation responses, using an emotion dictionary ("feeling refreshed", "solved" -> success), 2) in images, through the recognition and analysis of body language such as facial expressions, gestures, and head movements, 3) in voices, through the intonation, tone quality, and volume of the sound, and in other sensors, through pulse, blood pressure, body temperature, etc.

[0053] The matching concierge 3 (matching server) summarizes the (listening-type) conversation log retrieved from the dialogue log / dictionary storage device 7 into user requests (including questions and keywords). The user requests can include basic information such as the user's ID, URL, and language used, as well as an estimate and upper limit of the costs that can be used in addition to the issues / wishes / hopes.

[0054] When user satisfaction is not achieved (or it is found that it has been notified), that is, when the user cannot solve the problem by themselves, the matching concierge 3 (matching server) in the matching section uses services on the virtual space (Web) or services 5 on the metaverse, including web searchers 11 including Google Searcher and generative AI 12 including ChatGPT and its core GPT4, etc., to match this user request.

[0055] The web searcher 11 extracts a URL (the address of a site in virtual space) where there is text related to the answer to the user's question, that is, the means of achieving / solving the user's desire / problem. The generative AI 12 extracts text related to the answer to the user's question, that is, the means of achieving the user's desire and solving the problem, from virtual spaces including the metaverse. The matching part (question-answering part) accumulates, organizes, and utilizes the matching results including the answers of the web searcher (Google searcher) and the generative AI 12 (ChatGPT) and their utilization results using the evaluation and improvement intelligence part 6. Thereby, the influence given by inputs from objects and humans across time and space is simulated (emulated) in the virtual world, including from universal physical laws for predicting the distant, past, and even future to more local causal laws and rules. That is, not only universal physical simulations but also qualitative simulations by artificial intelligence AI (local simulations and inferences based on the rules of thumb of human experience including the past sales success experiences of salespersons) are used to evaluate and verify these answers and matching results. The evaluation and verification results are sent to the man-machine dialogue part 4, and evaluation and verification in man-machine cooperation (including cooperation with distant people and organizations connected via the Internet such as SNS) are also carried out. Answers that have obtained an evaluation above the specified value are sent to the counseling judgment memory part 2 (counseling server) of the conversation service part (VICA) together with comments including the evaluation value and qualitative opinions. The counseling judgment memory part 2 (counseling server) of the conversation service part sends, in addition to the answer and comments, a response with an inquiry about whether there are any concerns or problems with the answer to the user of the counseling input / output part 1 (counseling conversation client). Whether it can solve the desire or worry is finally evaluated by the user in the counseling (conversation rally) in the conversation service part. Since it is counseling by a listening-type conversation that listens to the user's speech without giving suggestions, etc., it is a conversation rally rather than a dialogue rally where one confronts the conversation partner.

[0056] Specifically, 1. The matching concierge 3, which is a matcher, returns the response and comments to the counseling judgment memory unit 2 (counseling server) of VICA, that is, the conversation service department. VICA responds with a response to which a problem confirmation sentence "Is there anything to worry about?" is added. If there is a problem, then afterwards, it enters a listening in-depth conversation rally that asks about the user's previous utterance, a summary of the resolution emotions, and the relationship between the current and past utterances again. It concretizes and clarifies the user's abstract and unclear wishes and problems. If it becomes difficult to solve independently, the conversation history in the counseling department including the more specific / clear wishes and problems is passed to the matching concierge 3 in the matcher / matching department again. It asks ChatGPT the question again. Through such a repetitive loop including the conversation service department (counseling server: VICA), the matching concierge 3 (concierge means guidance) is further highly reliable and made into something that the user can be satisfied with and have no anxiety.

[0057] 2. The user's wishes and problems are often incomplete / fragmentary / ambiguous / advanced. The matching concierge 3, which is a matcher, converts this wish and problem into a question for which the generative AI 12 including ChatGPT can give a reasonable answer, taking into account the above dialogue history. A trial-and-error loop is carried out in the man-machine dialogue department 4, which is separate from the conversation service department (counseling judgment memory unit 2 and counseling input / output unit 1: VICA), to change the question and inquire of ChatGPT again until the generative AI (ChatGPT or its new version GPT4, etc.) gives a reasonable answer, that is, an answer with an evaluation value or higher. In the man-machine dialogue department 4, a knowledgeable person, a consultant, a (former) salesperson, a receptionist, and other experts specialized in the service field, and furthermore, a counseling expert evaluate whether the answer from the web searcher, especially ChatGPT, makes the user feel any problems or anxiety. If the evaluation is below the specified value, the question and answer are corrected. Construct a function of storing the history of valid questions and answers obtained through loops including these man-machine interfaces and VICA, converting user requests and problems into valid questions, and creating a knowledge database containing dictionaries and rules for obtaining valid answers as one of the functions of an (evolved) digital twin, that is, an (AI) avatar. For the automatic construction of this AI, that is, the inference mechanism, use a learning and training model based on large-scale data (big data), and in the case of natural language processing, a large language model (LLM), and use generative AI 12 when generating multimedia, that is, text, images, and voices. For digital twins (computer twins) and avatars (avatars), set up a knowledge and database including a large language model, and classify and store question and answer histories (including evaluation values) for each factor such as different fields, regions, and times. Use the above historical data, especially the pairs of questions and answers with high evaluations or those of human experts, as teacher data, further increase the evaluation values of those with a high total degree of consistency with each of the above factors, and learn by weighting neurons and rules so as to integratively and automatically select them. In this way, realize an AI (artificial intelligence) that simulates or emulates human intelligence. Specifically, store and remember the history of questions and answers with high evaluations or those that contributed to problem-solving obtained through loops including (dialogue) rallies in the man-machine dialogue section 4 and (dialogue) rallies in VICA (conversation service section), and construct the function of dictionary formation and rule formation for converting user requests and problems into valid questions and answers as the function of an (evolved) digital twin or (AI) avatar in the evaluation and improvement intelligence section 6. For this AI, that is, the inference and learning mechanism, use a large language model including generative AI, that is, a neural network-based model or a knowledge base model of a DB (database) system represented by a rule base. These inference learning models are localized for each consultation field, region, time, and even user and stored in the (evolved) digital twin or (AI) avatar of the matching intelligence server.

[0058] To summarize the above, the matching results (services that meet user requirements) obtained using generative AI including Google Search and ChatGPT, as well as the creation and utilization results of these answers, are accumulated and organized to build a knowledge and database. Through this utilization, the impact of inputs from distant objects or people is simulated, inferred, and verified on a computer on the network, which is a virtual world. That is, these answers are evaluated, corrected (or improved), and output to the real world on the client side (ultimately the user) using the interface, which is the input / output section of the conversation service department (including extended interfaces such as VR and Google in Figures 1 and 2).

[0059] The matching part's schreiter (simulating physical causal relationships and human intelligence), that is, the evaluation and improvement intelligence department 6 (evolved digital twin, AI avatar) and the man-machine dialogue department 4 (dialogue matcher) verify (evaluate and correct). The answers, that is, the matching results, which are understood (no problem / satisfied) through the counseling judgment memory department 2 and the counseling input / output department 1 of the conversation service department, are used for reservation and execution of services such as shops. This reservation and execution, as well as their evaluation results, are also accumulated, organized, and utilized by the schreiter (digital twin, especially the AI avatar simulating human intelligence) as the utilization results of the above services. On the other hand, the digital twin and AI avatar are learning machines including large language models. The above answers are verified and corrected using all or some of the above-mentioned individual parts and the iteration and loop integrating these. Some parts include, for example, the conversation service department including the user's personal information, and further, the part excluding the accumulation and knowledgeization of information in the matching part that some experts dislike the leakage of top-secret know-how to other companies in the same industry. Through this, learning for high reliability is carried out constantly or regularly to train and grow the above AI avatar. An avatar means an agent, that is, a representative. In this way, a high-level and highly reliable man-machine cooperation that can contact and immerse in distant objects or people (from a distance) becomes possible in matching and services, enabling the enjoyment and provision of appropriate and satisfactory high-reliability and high-quality services without having to travel far.

[0060] In this embodiment, the search, i.e., matching, for products and services for wish fulfillment and problem solving is made highly reliable by the following loop. 1) Listening-type conversation input / output for the concretization, clarification, self-confirmation, and awareness support for solving the user's wishes and troubles by the listening-type counseling VICA of the conversation service department, and 2) when self-solving is not possible, conversion of the user's wishes and troubles into questions for the generative AI 12 (including ChatGPT) and acquisition of answers, 3) evaluation and correction of those questions and answers, including human-machine cooperation, 4) learning of the method for evaluating and correcting questions and answers by the large language model (especially local, i.e., localized to the services and shops in the user's wish and trouble fields, that period, region, and even preferences, including the utilization and search scope), and evaluation and correction (or improvement) of questions and answers by inference using the learning results, 5) confirmation, clarification, and concretization of whether there are any problems with the answer that meets the evaluation criteria, i.e., whether the user can be satisfied, by the listening-type conversation input / output of VICA in 1), and repetition of the loop back to 2) when self-solving is not possible.

[0061] In this way, instead of a global learning and inference machine for the whole world and all of humanity like ChatGPT, a dedicated local learning and inference machine specialized for individual users, local stores, institutions, and service fields is constructed. The counseling AI (VICA) of the conversation service department reports to the matching department whether the user is satisfied with the answer of the matching concierge 3. In addition to the local evaluation and correction loop of the answers of the web searcher 11 and ChatGPT in the matching department for each user, region, and field, the trial-and-error loop of checking the user's satisfaction by VICA of the conversation service department, discovering problems when dissatisfied, creating a new answer in the matching department when self-solving is difficult, and evaluating and correcting it is repeated many times. In this way, when combined with VICA, as a whole, a localized / local generative AI or its system with high user satisfaction in each region and each field is obtained. Here, "localized" means "suitable for each individual or individual user", and it means suitable for the user's attributes, such as region, field, role (e.g., salesperson in the clothing, food, housing, and hotel industry, education service seller, government office, employment field, etc.), period, personal attributes, etc.

[0062] Implementation device conversation rally example 1: Employment problem 1) S: Is there anything you're concerned about? U: I want to get into Company A, but I'm worried because I can't program. S:..... You're worried, aren't you? Specifically? U: ···· S: ···· U: I always end up failing in the software practice. S:... You always end up failing. And then? U: They tell me it's a good program even though I fail. S: They tell you it's a good program. What's the relationship with the program you mentioned at first? U: That's right. If I can create a good program, I should be able to get into the research institute. (A flash of inspiration!) S: If you can create a good program, you should be able to get into the research institute! So! U: My hopes are rising. (Emotions improve, the counseling VICA conversation service (temporarily) ends, the matching concierge 3 (matching server) that was passed the counseling history information and control asks GPT a question and gets an answer. If the human (a person with common sense, or a (quasi)-expert including a former salesperson, former guide, former counselor) in the man-machine dialogue section 4 (matching dialogue client) determines that the answer from the web searcher 11 (ChatGPT) is not (professionally) common-sensibly appropriate, it is possible to remake the question within the scope of the user's wishes, problems, and past speech history and ask GPT again to get an answer, repeating this until a proper answer is obtained. Remember this proper answer in the digital twin including GPT and large language models, and automate the process of making the above answer including GPT appropriate by constituting an AI avatar instead of a human in the man-machine dialogue section 4 (matching dialogue client), and self-propagate the scope.) S: Company A has a software research institute. Is there anything you're worried about? The Matching Concierge 3 (matching server) asks GPT a question and returns the verified result of the answer obtained to VICA. VICA returns the answer to the user and also adds and returns 'Is there anything you're worried about?', or 'Are there any problems or things you're worried about?', or 'Are there any points of concern?', or 'Are there any problem points ( / points of concern) or things you're worried about?' and returns control at the start of the counseling. U: I'm concerned that although the quality of the development program is high, the development speed is slow. S: So you're concerned (worried) that although the quality of the development program is high, the development speed is slow. Specifically? U: Can you ignore the development speed? S: Can you ignore the development speed, right? (The Matching Concierge 3 (matching server) asks GPT if Company A's software research institute can ignore the development speed, and gets a highly reliable answer with the help of the above people or avatars as needed, such as 'If the development speed can be ignored, there was a basic research institute before, but now there are only national research institutes.')

[0063] Example of the implementation device conversation rally 2: Shopping problem 1 S: Is there anything you're worried about? U: It's hot and I'm afraid of getting heatstroke! S: So it's hot and you're afraid of getting heatstroke. More details! U: I want an air conditioner, but I'm concerned about dust. S: So you want an air conditioner, but you're concerned about dust. Then U: Isn't there an air conditioner that doesn't let dust fly? [Ask GPT via the Matching Concierge 3 (AI). Exclude parts such as 'I'm an AI with a 2019 cutoff and don't know about new ones...' from the answers of Web Searcher 11 and ChatGPT and answer as follows, and return to the beginning of VICA] S: Company A has air conditioners with HEPA filters. Is there anything you're worried about? U: How much is the price? [When Matching Concierge 3 asks the generative AI (ChatGPT), it gets a response with an evaluation value of zero saying "Since I'm an AI with a cutoff in 2019, I don't know about new things...", so it enters the question loop of Matching Concierge 3. That is, it asks the web searcher again with keywords for Company A air conditioners and HEPA filters. Since the web searcher has replied with a URL, it passes that to the conversation service department such as the Counseling Judgment Memory Unit 2 and returns to the beginning of the conversation rally] S: There is an answer at this URL, that is, http:xxxxxxxx. Is there anything you're concerned about? U: http:xxxxxxxx has a lot written about it. It's very difficult to find the price of this type. S: http:xxxxxxxx has a lot written about it. It is very difficult to find the price of this type. So? U: Specifically, how much is the price? [At this point, since the user's statement includes 5W1H (Who, when, where, what, why, how; and How much for the price?), it is determined that it is difficult to solve independently. The counseling judgment memory unit 2 of the conversation service department determines this by collating whether the natural language text of the user's statement contains "how much" using regular matching technology. As a result of this judgment, the matching concierge 3, that is, the concierge, passes the user's request including the above URL in the conversation rally history. The concierge extracts the content of this URL, asks the generative AI (ChatGPT) a question, and requests it to summarize and report only the price of Company A's air conditioner. Although an answer (text) is obtained, the text is still long and the evaluation value of the answer is below the specified value, so it enters a question loop from the concierge to the generative AI. That is, using the conversation history log from the conversation service department, not only the price of Company A's air conditioner but also the keyword with a HEPA filter is added, and the generative AI is requested to re-summarize the answer (improve the summary of the page content of the URL by the generative AI). To improve efficiency, a (local / localized) learning database of product and service attributes (price, color, accessories including HEPA filter) (in Figure 6) is created in advance by humans including page searches by web searchers and summaries of the searched pages. The concierge can also sequentially add and expand (update) the content of this URL and its summary to the above (local / localized) learning database online. This time, an answer with a length below the appropriate specified value and an evaluation value above the specified value has been given, so it is passed to the conversation service and returns to the beginning of the conversation rally]

[0064] S: The price of Company A's air conditioner with a HEPA filter is XXX. Is there anything you're concerned about? U: I feel like the one with a HEPA filter is expensive. S: You feel like the one with a HEPA filter is expensive, right? Specifically! U: How much is the price of other companies' air conditioners with HEPA filters? [Since it's 5W1H again and it's judged that it's difficult to solve on one's own, a request is passed to the matching concierge 3. The matching concierge 3 extracts the content of this URL and requests the generative AI (ChatGPT) to summarize and report on the prices of air conditioners with HEPA filters. Although an answer is obtained, the text is long (judged to be difficult to understand as an answer) and the evaluation value is below the specified level, so it enters the concierge's question loop. That is, it requests the generative AI to further summarize the answer by adding the keywords of Company B's air conditioner with HEPA filter. This time, an answer of an appropriate length and with an evaluation above the specified value is given, so that and the previous price range of air conditioners with HEPA filters (the AI avatar of the matching concierge 3, that is, the evolved digital twin learned from the logs of these matching conversation histories) are passed to the conversation service, and it returns to the beginning of the conversation rally]

[0065] S: The price of Company B's air conditioner with HEPA filter is XXX. For the previous ones, the price range of air conditioners with HEPA filters is ZZZ1 - ZZZ2. Is there anything you're worried about? U: Company B's air conditioner with HEPA filter is a bit cheaper. I wonder if this is okay. In the matching of services and user requirements, the methods to achieve the user's wishes or solve their problems are efficiently retrieved, compared, evaluated, and verified using web search engines including Google Search and generative AI including ChatGPT. As a result, the user's requirements can be achieved and problems can be solved with high efficiency and high reliability as follows: 1) When the user inputs a question containing 5W1H (when, where, who, what, why, how) to the VICA (Counseling AI during conversation service) client shown in Fig. 6, the question is summarized into user requirements. The matcher shown in Fig. 6, i.e., the matching concierge 3 (matching server), uses a web search engine or ChatGPT to create an answer to this question. Through VICA, questions such as "The answer is... Is there anything you're concerned about?" or "The answer is... Is there any problem?" or "The answer is... Is there any problem or concern?" are output for the answer and points of concern. That is, when an explicit question beyond the scope of VICA's listening-based counseling is asked, a solution is explored, verified, and answered using a web search engine or ChatGPT. After this answer, VICA asks the user again about things that worry them, such as anxiety or wishes, and starts counseling. As a result, the mirroring (repetition and paraphrasing) and in-depth exploration functions of VICA promote realizations and inspirations for further concretization and verification of the answer. 2) Also, in the human-machine dialogue section 4 (matching dialogue client), the level of verification of the answer is improved through human-machine interaction with experts. That is, it comprehensively improves beyond the level of mechanical learning and training using a large amount of data, i.e., beyond the level of the accumulated information and the AI for its processing. 3) The history of concretization and verification of the answers by these counseling AIs and human-machine interaction with experts, as well as the actual usage results and problems encountered during use, are also saved in the evaluation and improvement intelligence section 6 of the matching intelligence server, which provides further verification of the answer and comments for that purpose. In this way, the accuracy of the answers from the web search engine (ChatGPT) and the matching using them is further enhanced.

[0066] The following is the specific implementation method. 1) In the method of using the web searcher 11, the matching concierge 3 (matching server) gives the user request to the web searcher 11 as a service search key. The web searcher 11 returns the address (URL) of the page in the virtual space corresponding to this search key (for example, a site like a Q&A site by a portal site operator) to the matching concierge 3 (matching server) as a response. The matching concierge 3 (matching server) returns this URL or the edited content thereof to the counseling judgment storage unit 2 (counseling server) as a response. In the case of a URL, it can be a summary of the content of the page into a text sentence including the extraction keyword, or a response requested to summarize the content (including the keyword if the result is bad) to the generative AI 12 including ChatGPT. The counseling judgment storage unit 2 (counseling server) returns this response to the counseling input / output unit 1 (counseling conversation client). The counseling input / output unit 1 (counseling conversation client) outputs the content of this response to the screen as the output of the conversation service unit VICA and outputs "Is there anything you are concerned about?". 2) In the method of using the generative AI 12 including ChatGPT, the matching concierge 3 (matching server) asks the user's request as a question and gives it to the generative AI 12. The generative AI returns an answer to this to the matching concierge 3 (matching server). The matching concierge 3 (matching server) sends this answer to the evaluation and improvement intelligence unit 6 of the matching intelligence server. The evaluation and improvement intelligence unit 6 of the matching intelligence server evaluates this answer. The evaluation result is returned to the matching concierge 3 (matching server) as a comment. At the same time, these answers and evaluations are saved as know-how specific to the questioner or question category as a user in user correspondence / user request correspondence. The matching concierge 3 (matching server) returns the edited answer, evaluation, and content to the counseling judgment memory unit 2 (counseling server) as an answer. The counseling judgment memory unit 2 (counseling server) passes this answer to the counseling input / output unit 1 (counseling conversation client). The counseling input / output unit 1 (counseling conversation client) outputs the content of this answer as text output under the avatar image of VICA (Visual Counseling Agent) on the screen and outputs "Is there anything to be concerned about?". This text sentence can be read aloud as voice. 3) Regarding the evaluation of the answer, it is performed by the evolutionary digital twin, the AI avatar, within the evaluation and improvement intelligence unit 6 of the matching intelligence server. This AI is a twin in the virtual space, that is, a doppelganger like a look-alike, an avatar if it is a person, a simulation program of the shape, structure, and operation if it is an object, and further a simulation system (simulation system) doppelganger that integrates these avatars of people and objects in the virtual space. For example, it is a simulation system (simulation system) doppelganger that simulates the influence on the user avatar when the avatar of a user who is heat-intolerant and has dust allergies lives in a room avatar where an air conditioner avatar that the user avatar is about to purchase is installed. As a specific example, this doppelganger simulates the effects and problems of the air conditioner by changing the adjustment function and sensor function of the air conditioner, and the flow of air and dust and the cooling of each part of the body.

[0067] Verification, evaluation, and correction (or improvement) of questions and answers by a matcher using generative AI12 (GPT) or a web searcher11, along with the implementation and evaluation results of the matched (searched) services, are carried out in an integrated dialogue loop (rally) that combines these three types. 1) The matching concierge 3 asks questions to GPT, obtains answers, and sends them to the conversation service department (VICA: counseling AI, CA). After the answer (and comment), VICA responds to the user by adding "Is there anything you're concerned about?", or "Do you have any problems or concerns?", or "Are you feeling uneasy about anything?", or "Do you have any problems ( / uneasy points) or concerns?". The question and answer are evaluated and corrected through a (conversation) rally via CA's mirroring, sentiment summarization, and relationship evocation. 2) The questions (and comments) are not displayed. Whether through the input and output of CA or not, it is not mirroring but a rally of normal man-machine answer checking and question modification to optimize questions and answers and acquire knowledge for improving matching accuracy. 3) Integrate the cases where the changes, evaluations, and corrections of questions and answers are carried out not only by CA users or man-machines (both humans) but also through an evolving digital twin AI (initially, it only changes to simple and mechanical different questions including "Is the answer of ChatGPT correct?", but it learns, evolves, and self-propagates through changes, evaluations, and corrections).

[0068] In summary, aiming to remotely provide, with high efficiency, the services necessary and sufficient to solve wishes and troubles without leaving dissatisfaction using a computer, the conversation service department uses an input / output unit that inputs from a microphone or mouse or keyboard and outputs to a speaker or display, repeatedly confirms the input speech and its details, etc., listens out for wishes and troubles, and supports self-resolution. The matching department searches for other services (other than conversation services and matching services) that can support high-efficiency remote service resolution, and supports reservation, execution, and evaluation. A) Evaluate and improve (confirm and specify the concerned points) the questions and answers through a (dialogue) rally with the responses of mirroring, sentiment summarization, and relationship evocation in the conversation service department. That is, it is confirmed whether the answer (including the questions, prompts, and comments when obtaining the answer) can resolve the user's concerns, that is, whether the wishes can be fulfilled or the worries and problems can be solved, or the defects of the answer are confirmed and specified. Then, another answer is requested from the matching concierge 3. B) For this solution, the matching concierge 3 repeats the loop of presenting web (virtual) pages or answers from generative AI to solve the user's problem. That is, when it is difficult to solve the problem independently in the above-mentioned conversation service section, the heard problem is summarized and converted into questions or user requests. 1. The summary of the web page (page in the virtual space) including the Yahoo! Knowledge Bag of Google Search for the problem (question or user request), or 2. Utilize the answers of generative AI (including ChatGPT) for the above questions. In particular, 3. Present this answer to the user. If there are any concerns, further present web (virtual) pages or answers of generative AI (including ChatGPT) for this solution (to improve the reliability of the answer including the conversation with the user), and repeat the loop to solve the user's problem. Furthermore, 4. Before presenting web (virtual) pages or answers of generative AI (including ChatGPT), these are checked and evaluated by ordinary people or experts with common sense, and questions are changed within the scope of the user's problem or wish until they are considered reasonable (above the specified evaluation value), and the loop of obtaining answers by retrial is repeated (to improve the reliability of the answer including the conversation with the expert or its AI, that is, the AI avatar). 5. Remember the user's problems and wishes in both loops, the content of the conversion of the questions, and the history of the answers, and use this to automatically obtain, verify, and correct the answers of web (virtual) pages, web search engines 11, and generative AI 12 without the above-mentioned people or experts. Now, when exploring the matching service using the generative AI (including ChatGPT) and web search engine 11 described in the second embodiment, the self-organizing decentralized evolutionary intelligence method described in the first embodiment (including batch sending) may not be used. However, by using the latter in combination, a more user-satisfying matching can be achieved.

[0069] In the second embodiment, since the self-organizing distributed evolutionary intelligence method is not used, services and contracts that are the targets of matching do not have to be registered. As a simpler / alternative embodiment, 1) Simplification of the embodiment starts with having the generation AI 12 and the Web searcher 11 answer when there is an inquiry from the user about whether there are any problems with execution. In response to the need for and tolerance of simplification, speed increase, and cost reduction, ii) verification and evaluation are carried out using a digital twin (physical simulator, AI simulator based on memory and learning of past experiences), iii) a trial run, iv) reservation and inquiry on the Web, and further v) verification and evaluation by actual execution are carried out step by step. AI-like feedback through their memory and learning and the utilization thereof is also made possible in this order. 2) For the next simplification of the embodiment, the function of the matching unit is limited to that of a concierge, that is, only matching. The reservation and execution functions are deleted. When there is an inquiry from the user about whether there are any problems with reservation and execution, as in 1) above, questions are asked to the generation AI 12 and the Web searcher 11 to obtain answers. Alternatively, simulation of reservation and execution is carried out using the digital twin (simulator) of the evaluation and improvement intelligence unit 6 for verification and evaluation. This inquiry about reservation and execution or the simulation in the evaluation and improvement intelligence unit 6 is also carried out simultaneously when evaluating the answers (obtained from the generation AI or the Web searcher 11), and the results are stored as comments on the answers or in the evaluation and improvement intelligence unit 6. The above answers during execution anxiety are carried out by the conversation service unit (when using comments) or the matching unit (when using the memory of the evaluation and improvement intelligence unit 6) using these. 3) When no answer comes from the generation AI or the like. Or when an inquiry such as conditions (constraints and roles) necessary for a (better) answer comes together with the answer, that is, when a reverse question comes, it is returned to the counseling judgment memory unit 2 of the conversation service unit. Instead of or in addition to the answer, the counseling judgment memory unit 2 makes this reverse question into a response sentence. It is output via the counseling input / output unit, and the user is asked to answer the reverse question. If no answer is obtained, the question to the generation AI is stopped, and the search by the Web searcher 11 may be switched to. 4) In yet another simple embodiment, the matching dialogue client is deleted in FIG. 2. That function is given to the counseling conversation server in the man-machine dialogue matching mode. In this case, the display and correction instructions of questions, question destinations, answers, and evaluation results through communication with the matching server are performed on the counseling conversation client via the counseling conversation server. However, the mode is set so that the man-machine dialogue matching function can only be used by administrators (including former salespersons, former consultants, and former counselors). Also, in this mode, the content of the private conversation log 7 of each user cannot be accessed. All data can only be accessed through communication with the matching server.

[0070] [Third Embodiment] Next, a third example of the present invention will be described.

[0071] (Remote Service Support Device / System's Matching Intelligent Server) FIG. 3 is a more detailed description of the evaluation and improvement intelligent unit 6 of the matching intelligent server, which is a component for supporting the efficient use of services including remote services. That is, it is a functional block diagram showing a part of an example of a remote service support system according to a third embodiment related to an evolutionary digital twin or an AI avatar. Regarding the third embodiment, in the second embodiment, the case where the matching intelligent server including the evaluation and improvement intelligent unit 6 is used will be described. However, in the first embodiment, the evaluation and improvement intelligent unit 6 can be provided inside or outside the device.

[0072] The evaluation and improvement intelligence unit 6 of the matching intelligent server in FIG. 3 is an example that utilizes a large language model (LLM) obtained by scaling up a neurocomputing model for natural language learning prediction using big data in FIGS. 1 and 2. As shown in FIG. 3, AI avatars using large language models and neurocomputing models are prepared for each individual, region, and field, which serve as the roles of the evolved digital twin avatars. The distribution integrator distributes dialogue logs and implementation histories to these avatars and evaluates and integrates inferences and answers. The integration has evaluation criteria / evaluation rules (knowledge base) / simulator. Here, the (conversation / dialogue) log refers to the historical data such as the questions and obtained answers of the matching concierge, service reservations and implementations, and evaluations at that time in this specification. It includes historical data that is a time-series set of pairs of the user's utterance sentences and the listening-type conversation response sentences created by the listening-type counseling judgment memory unit 2 in the listening-type counseling in the conversation service, and historical information of dialogue rallies between service care clients and experts including (former) counselors, (former) consultants / consultants, (former) salespersons, and (former) businesspersons. The logs and questions are distributed to relevant avatars for individual (local) learning and training. The answers to the questions are selected and integrated based on the relevant individuals, regions, stores, organizations, and fields (for example, the buyer who is the user, their residential area, familiar counselors or salespersons, and the fields are home appliance purchases, job hunting, ···). In this way, instead of a global learning and inference machine like ChatGPT for the whole world, a learning and inference machine specialized for individual users, regional stores / organizations, and service fields is constructed. When combined with a counseling AI (VICA) that provides the answer and the result that it is adopted without problems, as a result, a more user-satisfying local generation AI is obtained. The evaluation and improvement intelligence unit 6 of the matching intelligent server constructs an automatic learning and exploration mechanism for the above-mentioned matching and medical service methods in virtual spaces such as the metaverse to improve and evolve the services including the matching in the matching concierge 3 (matching server) and the AI of service 5A. That is, a learning machine (including learning data) shown in Fig. 3 that automatically improves the inference machine using data analysis AI such as big data analysis from the above experience data is constructed. That is, the learning device sequentially inputs experience data such as the planning, implementation, and evaluation of matching and services (accumulated as dialogue / implementation / evaluation logs), converts it into the transmission weights of each nerve of a neuron (cerebral nerve network) model or a neurocomputer, and saves it as learning data. As a result, the sequentially evolved inference machine (explorer) proposes improvement plans (sequentially improved and evolved excellent plans) for the methods and procedures of matching and services for the dialogue log. Since the inference machine (explorer) evolves sequentially, it is saved and managed together with the date and time of evolution (change), user requirements, and evaluation results, and is used to create alternative plans. The created alternative plans are evaluated, and it is checked and recorded whether the evolution is proceeding correctly (not degenerating), and this is utilized for future alternative plan proposals. Such a learning-type inference machine using a neuron (cerebral nerve network) model can also be used for constructing a digital twin, especially its intelligence unit. It enables not only the preservation of the intelligence of matching, services, and their digital twins (structured data and programs for simulating real objects), but also their eternal evolution. Furthermore, the listening-type counseling AI that controls the input of this system can also be regarded as a service. The intelligence of its user (patient) is also saved and evolved in this way as the intelligence of a digital twin (computerized twin) (data and programs for simulating real objects), and the quality of the listening-type counseling AI service for accurately obtaining user requirements can be improved. On the other hand, for stores (including online stores) and organizations (including factories and hospitals), etc., the intelligence of digital twins (data and programs for simulating real objects) that simulate the ways of working and skills of human beings on the service side and the operation of devices is saved, evolved, and grown, that is, the know-how of the service is learned and maintained. When simulating even the appearance and behavior, it is called an avatar.For example, even if a veteran counselor (human) retires, their digital twin remains and serves as an AI counselor. When combined with a 3D image as in the aforementioned VICA, it becomes a counselor or conversation partner as a memorial (counseling AI) avatar even when the veteran counselor has passed away.

[0073] [Fourth Embodiment] Next, a fourth example of the present invention will be described.

[0074] (High-Reliability and High-Efficiency Remote Service Support Method / Program) Figures 4 and 5 show a high-reliability and high-efficiency remote service support program and a high-reliability and high-efficiency remote service support method that can be installed on a computer (client, server, or cloud computer) of a remote service support system according to a first embodiment for the efficient use of services including remote. However, Figure 5 particularly represents the main flow of the second embodiment, with some steps omitted. Also, the dashed line between S11 and S12 in the counseling input / output section in Figure 5 indicates an indirect control transfer via another process and shows the passage of time. Similarly, the dashed line between S109 and S32 (S114) in the matching concierge also indicates an indirect transfer via another process and shows the passage of time. As shown in Figures 4 and 5, the high-reliability and high-efficiency remote service support method / program includes S11, S12 in the computer (such as a mobile or personal computer) of the counseling input / output section 1 (counseling conversation client), S21, S22, S23 in the computer of the counseling judgment memory section 2 (counseling server), S31, S32 in the computer (cloud / server) of the matching concierge 3, S41 in the client computer (mobile / personal computer) of the man-machine dialogue section 4, S51 included in service 5 (in a cloud / server computer in a virtual space or metaverse space), S61 in the evaluation and improvement intelligence section 6 (matching intelligence server), etc., and is a method / program installed and provided in each client, server / cloud computer.

[0075] When the user starts the system 101, the counseling input / output unit 1 (counseling conversation client) (the program counter of a computer including a general name such as a mobile phone or a personal computer or the control of a program) proceeds to step S11. In step S11, each data of voice, image, and sensor is input, including those for virtual reality (VR) and augmented reality (AR), and is converted into data of speech, emotion / physical information (such as a text sentence), and is transmitted to the counseling judgment storage unit 2 (counseling server). If conversation response data (text) is received from the counseling judgment storage unit 2 (counseling server), it proceeds to step S12, and the response sentence for counseling created by the counseling judgment storage unit 2 (counseling server) and also data at the time of reservation or execution of the meta-service from the (meta) service 5 including the instruction / control data are output to a display unit such as a monitor of the counseling conversation client or an external device (device, robot, etc.) 10 connected to the counseling conversation client. In step S12, image data of virtual reality (VR), digital twin / avatar (3D / three-dimensional or video), etc., including those sent from the (meta) service 5, are output to the display unit of the counseling input / output unit 1 (counseling conversation client) and the external device 10.

[0076] When the counseling input / output unit 1 (counseling conversation client) receives the user's speech, emotion / physical information data, in the program counter of the computer of the counseling judgment storage unit 2 of the counseling server, it proceeds to S22. In S22, the user's requests / troubles are analyzed, a response sentence for listening counseling is created, transmitted to the counseling conversation client, and the dialogue sentence is logged.

[0077] However, if the meta-service is being carried out in the meta-service request (user satisfaction achievement) mode, that is, the reservation / execution mode, the instruction information related to service reservation / execution and the user's speech / affection / physical information data from the counseling input / output unit 1 (counseling conversation client) are directly sent to the matching concierge 3 of the matching unit. At this time, the counseling judgment memory unit 2 (counseling server) does not perform the processing of creating a response sentence for listening counseling, sending it to the user, logging the dialogue sentence, etc., nor does it perform the processing of step S22 below.

[0078] In step S22, the counseling judgment memory unit 2 (counseling server) determines from the user's speech / affection information whether the user satisfaction is achieved by basic listening counseling. If it is impossible to solve independently (hereinafter, it may also be referred to as (user satisfaction) not achieved), it switches to the meta-service request (user satisfaction not achieved) mode and notifies the matching concierge 3 (matching server).

[0079] On the other hand, when data at the time of matching the user request and the service or at the time of service reservation / execution comes from the matching concierge 3 (matching server), the program counter of the computer of the counseling judgment memory unit 2 (counseling server) proceeds to step S23, receives the service reservation data and the instruction / control data at the time of execution, and sends them to the counseling input / output unit 1 (counseling conversation client). If there is a notification from the counseling judgment memory unit 2 (counseling server), the program counter of the matching concierge 3 (matching server) advances to program step S31. In S31, the user request is created / changed, the matching between the request (including questions / problems) and the services in the meta / virtual space is performed, the reservation / execution of the selected service, evaluation, and settlement (payment) are carried out. For matching (selection / evaluation), as shown in the second embodiment, the use of a web searcher 11 and a generative AI 12 is also possible. When using these, the matching (selection) target is extended to general services on the web. The selected general service can also be contracted / registered as a (meta) service. The matching results are logged including the selection / reservation / execution / evaluation status. If the evaluation is below user satisfaction (reached), a repeated process is carried out starting from the matching including changing the user request (also including assisting in noticing and introspecting for problem confirmation / specification / solution in the conversation service section). The above can also be performed semi-automatically with mutual data transmission / reception (information sharing) with the man-machine dialogue unit 4 (matching dialogue client). In the matching concierge 3 (matching server), the service execution / evaluation status including the above user request and user satisfaction evaluation result is logged. The logged information notifies the evaluation improvement intelligence unit 6 of the matching intelligence server of the start of matching and the end of service implementation / evaluation, and receives and utilizes improvement proposals for the methods of matching and services (reservation / execution / evaluation) (including diagnosis / treatment methods) from the evaluation improvement intelligence unit 6 of the matching intelligence server.

[0080] Note that the above meta-services and services are, as shown in FIG. 1 etc., the services (shops) 1 ··· service (shop) service i ··· of the meta-service (metaverse shop server / client) group 5. If there is a notification from the meta-service 5 to, for example, explicitly accept a service, the matching concierge 3 (matching server) (the program counter or program control of the computer) proceeds to S32 and transmits data at the time of service reservation and instruction / control data at the time of service execution. (VR: three-dimensional for virtual reality) Images, etc. can also be directly transmitted and received with the counseling input / output unit 1 (counseling conversation client) according to the load.

[0081] If there is a notification from the matching concierge 3 (matching server), the program counter of the man-machine dialogue unit 4 (matching dialogue client) proceeds to program S41. In S41, the man-machine dialogue unit 4 (matching dialogue client) displays or corrects user requests and matching results, and displays or outputs the service status.

[0082] If there is a notification from the matching concierge 3 (matching server) to inquire about a service that matches the user request, the program counter of each server of the meta-service 5 proceeds to program S31. In S31, if the service is available, it notifies the matching concierge 3 (matching server) of the available reservation date and time, fees, etc. If a notification of successful matching comes, it sets a reservation if necessary (as required). If a notification of the start of service implementation comes, it transmits and receives data for implementation, explanation, and instruction with the matching concierge 3 (matching server). If a notification of the end / evaluation of service implementation comes, it ends the service implementation and notifies the matching concierge 3 (matching server) of the evaluation result. The notification of the inquiry about the possibility of matching from the above-mentioned matching concierge 3 (matching server) is broadcast (or in anycast mode) to the servers of each service (store) of the meta-service 5, and only the matching services respond autonomously without being restricted from the outside. However, it is also possible to inquire individually about the possibility of matching one by one, or to set up a server that oversees the meta-service 5 and manage the inquiries and implementations to each service (store) registered in the meta-service 5 from there, either collectively or individually. On the other hand, in the case where each service including the shop is not registered in Example 2 or the like, the processing of the instruction information related to service reservation and execution of S12 and S23 may be performed in a separate system.

[0083] If there is a notification of the end of service implementation and evaluation from the matching concierge 3 (matching server), the program counter of the evaluation improvement intelligence unit 6 of the matching intelligent server proceeds to program S61. In S61, the matching concierge 3 (matching server) reads the experience data such as the planning (selection / reservation), execution, and evaluation of the service accumulated as the dialogue log (user request log) or service log (service implementation / evaluation log, diagnosis / treatment log in the medical field). These read selection / execution experience data are organized and utilized as knowledge. As a specific example of implementation, in S61 of the matching concierge 3 (matching server), the above cumulative experience data is sequentially input into the neural network program installed as a learning machine, and is converted and stored in the transmission weights of each neuron for organization. As a result, the neural network evolves and becomes an inference device including generative AI. If there is a notification of a new user request from the matching concierge 3 (matching server), it proceeds to program S61 and proposes a service matching for this and an improvement plan for the service method (including diagnosis / treatment methods).

[0084] Hereinafter, as described in the second embodiment, a web searcher 11 and generative AI 12 are used. The matching (selection) target is extended to general services on the web. That is, not only the registered services and the contracted net shops but also the general service sites described on the web pages are set as the matching targets. Here, an implementation method for enhancing the reliability of the selection from this extended matching target including the loop with the dialogue service will be described in detail.

[0085] In the Conversation Service Department, responses to listening-based conversation texts (such as mirroring for input speech listening confirmation, in-depth exploration for concretization promotion, emotional change summary for problem-solving degree confirmation, and related inquiries) for promoting self-awareness for self-resolution are repeated to identify wishes and troubles and support self-resolution (Step S100).

[0086] When self-resolution is difficult, a request is made to the Matching Concierge 3. Difficulty in self-resolution means that, for example, if there are statements including questions or interrogative sentences, especially those containing 5W1H (Who, When, What, Where, Why, How), it is determined that self-awareness (Self-awareness: self-recognition) has limitations and assistance from others or a concierge is required. When making a request, a log of listening-based conversation texts including the user's statement content (wishes, troubles, problems) is passed to the Matching Concierge 3. That is, the Counseling Judgment Memory Unit 2 shares the listening-based conversation log, which is a history of user statements and responses to the above listening-based conversation texts, with the Matching Concierge 3 (Step S101).

[0087] The Matching Concierge 3 summarizes the problems heard as the user's statements, that is, troubles and wishes, into tuples with added categories (such as job hunting, further education, home appliance purchase) according to needs and converts them into questions (Step S102). For this conversion, a conversion dictionary based on keywords or a large language model that utilizes neurocomputing learned from a large amount of data for natural language conversion and prediction is used.

[0088] The Matching Concierge 3 inquiries about this question or user request to the Web Searcher 11 and the Generation AI 12 (Step S103). Then, the Matching Concierge 3 extracts the answers (including URLs and their page contents) to the questions from the Web Searcher 11 and the Generation AI 12 (Step S104).

[0089] The answers from the Web Searcher 11 and the Generation AI 12 are sent to the Evaluation and Improvement Intelligence Unit 6 for evaluation (Step S105). If the evaluation value is below the specified value, change the question and improve it so that the evaluation value becomes equal to or higher than the specified value. For example, if the answer from the generative AI is "I can only generate pre-learned sentences, so I cannot answer questions about events after 2022 when I was released. That is, I cannot give an accurate answer.", then the matching concierge 3 cannot answer, which means the evaluation value is 0. Therefore, the matching concierge 3 instructs the expert AI (avatar) to create a question limited to before 2022 to obtain an answer from the generative AI, and also obtains an answer from the web searcher 11, and evaluates the sum of the latter answer added to the former answer as the answer. (Step S106). If the evaluation value does not reach or exceed the specified value with the expert AI (avatar), in the man-machine dialogue section 4, the expert (human) changes the question and answer (step S107). If the expert is also unable to do so, or if there is no (suitable) expert, etc., even if the answer is corrected by changing the question or the question target in cooperation with the AI (avatar), if the answers with an evaluation below the specified value continue for a predetermined number of times or more, the evaluation value of the answer with the best evaluation value is set as the specified evaluation value. As described above, it becomes possible to correct the evaluation of the answer within a limited time.

[0090] Such an evaluation regarding the execution is performed by the digital twin (including simulators and AI avatars which are qualitative / knowledge-based / evolutionary simulators that utilize particularly accumulated data) through simulation or by using knowledge such as previous implementation results (step S108).

[0091] If the evaluation value is less than the specified value, return to step S103, change the question content and the destination of the question, and improve it so that the evaluation value becomes equal to or greater than the specified value. If the evaluation value does not become equal to or greater than the specified value, an expert changes the question and answer and re-evaluates. Even if that (including the case where there is no expert) is not enough (a response with an evaluation value equal to or greater than the specified value cannot be obtained within a predetermined number of times), adopt the question and answer with the highest evaluation value as the response of the matching concierge 3. The above process of evaluating, correcting, and adopting questions and answers includes memorizing / learning in connection with the ID and image information of the user, expert who evaluated / corrected, required by the (evolving) digital twin or AI avatar, and is used for creating questions and destinations of questions, answer evaluation, determination of experts and AI avatars to be used when the evaluation value does not become equal to or greater than the specified value, and evaluation / correction (or improvement) of their answers in the future. The memorization / learning is performed with different weights for each person, region, and field, that is, locally, and its utilization is integrated by weighting the degree of fitness of the person, region, and field (step S109).

[0092] When a response with an evaluation value equal to or greater than the specified value is obtained in this way, add a comment to the response such as the answer of the generation AI whose evaluation has been corrected and reliability has been improved, and pass it to the counseling judgment memory unit 2 of the conversation service department. (Step S110).

[0093] In the conversation service department, the counseling judgment memory unit 2 adds a confirmation text (text), passes it to the counseling input / output unit 1, presents it to the user, and asks the user as if listening attentively for any concerns (provides a confirmation text). That is, after the answer (and comment), respond by adding "Do you have any problems or concerns?", or "Do you have any points of anxiety?", or "Do you have any problem points ( / points of anxiety) or concerns?". If there are no concerns, that is, no problems, output the answer and the corresponding service as a solution to the user and end, and request the matching concierge 3 to implement (reserve, execute, evaluate) the answer and the corresponding service. The matching concierge 3 simulates the execution of the answered service or executes it (including a trial), and passes the answer that has reached the specified evaluation value to the conversation service department together with the evaluation of the result. The conversation service department adds "Do you have any problems or concerns?" and responds to the user again (step S110).

[0094] If there are problems, the counseling judgment memory unit 2 of the conversation service department through mirroring, digging deeper, emotion summarization, and relationship evocation of the problem points of the answer (instead of confronting and talking, with unconditional positive concern / care (R. Rogers' Unconditional Positive Regards), like a mother wrapping her child, talk in a way that shows closeness, try to open the user's heart and encourage introspection, so rather than a dialogue) confirm and evaluate and correct the answer in a conversation rally (step S112).

[0095] If it becomes difficult to solve independently, request the matching concierge 3 again (step S113) and repeat the above loop. In this way, the answer is until the user (the user) is satisfied. 1) The user on the conversation service side; and 2) The digital twin (AI avatar) in the matching concierge 3; and / or 3) Humans other than the user (including people and organizations connected via the Internet through the portal care server, their tweets, and current / fomer sales representatives and service providers on the service providing side including their tweets, and experts in the service field), that is, humans other than the user; and 4) Real (including trial use), simulated, and remote (digital twin) services provided or equivalent; For the responses of the generation AI 12 at the four levels or the URL responses obtained from the web searcher 11, natural language response sentences are used for individual correspondence (by individual, region, and field), including the case where the AI avatar or experts edit them, in an iterative loop of collaborative conversation / dialogue rally and multiple / iterative checks and improvements in their comprehensive loop. When satisfaction is achieved, this is transmitted to the matching concierge 3 (step S111). The matching concierge 3 that receives this instructs the reservation, execution, and evaluation of the service corresponding to the satisfied answer where the user is Self-Aware and satisfied without problems in the conversation rally in the listening-type counseling in the conversation service department (S114).

[0096] As described above, evaluation is performed regarding both the accuracy of the answer and user satisfaction, enabling highly favorable evaluations, that is, ultra-high reliability in both. Regarding user satisfaction, not only the user's speech (emotion determination using an emotion dictionary) in the conversation service but also image understanding / processing including facial expressions, body temperature, and pulse, and multi-sensor fusion technology that integrates these multiple sensor information are digitized and judged based on a threshold.

[0097] Summarizing the above steps, an example of the implementation method of the interface between the conversation service department and the matching concierge 3 goes through the following steps 1 to 6. 1. Using the input / output unit that inputs from the microphone, mouse, or keyboard and outputs to the speaker or display, the conversation service department repeatedly makes responses for promoting awareness (mirroring for listening confirmation of input speech, in-depth excavation for promoting concretization, summary of emotional changes for confirming problem-solving ability, related inquiries) to assist in self-resolution by listening out for wishes and worries (conversation rally for listening out for wishes and worries); and 2. When it is difficult to solve the problem independently, request the matching concierge 3 to summarize and convert the identified problem into questions or user requirements (question creation) sub-step, and inquire the web searcher 11 and the generative AI 12 about the questions or user requirements, and extract the answers including the URL and the content of the page (answer extraction) sub-step, and repeat the sub-step of the expert or its AI (avatar) for dialogue evaluation and correction, or the digital twin for evaluation and correction by simulation or previous implementation results to improve reliability (answer evaluation and correction), and inquire the web searcher 11 and the generative AI 12 to extract, evaluate, correct, and verify the answers and convey them to the conversation service (answer creation and verification) step, 3. Present the created and verified answer to the user and inquire as if listening attentively to the user's concerns, that is, after the answer (and comments), ask "Do you have any concerns?", or "Do you have any problems?", or "Do you have any problems or concerns?", or "Do you have any worries?", or "Do you have any worries or concerns?" (simulated) step of asking whether conversation verification of the answer after evaluation and correction before execution is required, 4. If there is a problem, use the conversation service again to clarify the problem of the answer through self-awareness by means of (conversation) rally through mirroring, digging deeper, emotion summarization, and relationship evocation, break it down into specific and detailed ones, and try to solve it independently (conversation verification rally of the answer) step, 5. If it becomes difficult to solve the problem independently, repeat the loop (creation and evaluation rally of the answer) of the conversation service department requesting the matching concierge 3 again to create, evaluate, improve, and verify the answer and convey it to the conversation service, 6. The conversation service department recognizes that there is no problem from the response statements such as "There is no problem (point)", "There is nothing to worry about", or "There is no worry", etc. (this recognition uses the summary function of the conversation history, sensor data such as facial expressions and blood pressure, and the digital twin AI trained with the conversation speech history in addition to the above statements), convey this to the matching concierge 3, and end the conversation service for the current user (end of all loops) step.

[0098] The above steps and sub-steps will be described in more detail using examples of each loop (conversation rally) and nested loops, i.e., multiple loops, in the conversation service department and the concierge department. The leading tags of the text in each conversation rally are (S:, U:) for the conversation service department and (Question:, (Generative AI / Web Searcher) Answer:) for the concierge department. The content within the rectangular key brackets [] is the explanation of each rally.

[0099] Example of the implementation method rally 1 (employment problem 2) [Since a good program can be created as if there is speech content in the conversation history dialogue.text, it should be possible to enter a research institute! I want to enter XX Company, but I'm worried because I can't create a program! I failed in the exercise! What's the relationship with the program? I got it! Since a good program can be created, it should be possible to enter a software research institute!] Question: Is there a software research institute in XX Company? [Enter relevant sentences, latest sentences, and related words in the conversation history dialogue.text] Generative AI Answer: XX Company has XX Research Institute and conducts research in a wide range of information and communication fields... [It's not a software research institute, and even if a good program can be created, it's not clear whether one can be hired, so the evaluation is below average, i.e., 2 out of 5 levels] Question (changed): I'm asking you, the hiring staff. Is there a software research institute in XX Company? Generative AI Answer: Please explain about the hiring staff of XX Company. [No answer is obtained, so the evaluation is zero] Question (changed, other sentences in the speech history): Are there any companies with a software research institute? Generative AI Answer: Company B has a software research institute. [An answer is obtained, but it's not about XX Company, so the evaluation is 3 out of 5. Since an evaluation above average is obtained, the answer and the comment including the evaluation value are returned to the conversation service department]

[0100] Example 1A [User answer check loop in the conversation service department, satisfaction evaluation loop by clarifying and specifying wishes] [Answer evaluation correction loop, question and answer correction and improvement loop] S: Company B has a software research institute. Is there any problem or concern? U: I'm concerned that the quality of the development program is high, but the development speed is slow. [If Company B has a software research institute, then Company XX probably has a similar one. At worst, I can go to B, but my real concern is whether I can still do software work even if the development speed of the program is slow.] S: You're concerned that the quality of the development program is high, but the development speed is slow. So? [Digging deeper, that is, concretizing the wish through Digging] U: Can you ignore the development speed? [Slow development speed → Ignore; Concretize the wish through Digging] S: Can you ignore the development speed? That's a good question. [Return to Matching Concierge 3 again and ask the generative AI if the software research institute of Company B can ignore the development speed.], Question for the generative AI: Can the software research institute of Company B ignore the development speed? Answer: For an AI with a cut-off in 2019, it doesn't know about new things... [No expected answer regarding ignoring the development speed is obtained at all, so the evaluation is zero.] Search Google with (Company B, software, research institute, development speed, ignore) Extract the content of the Google search answer URL with the above keywords: Company A doesn't require recent results, so there was probably a basic research institute that could probably ignore the development speed. Now, there are few such institutions, at least only national research institutes (however, since the Google search answer is a URL, summarize it using the knowledge of human experts or avatar AI to obtain a natural language answer). [A response regarding ignoring the development speed was obtained, but since it was not exactly what was desired, the evaluation was medium (3). Since an evaluation of medium or higher was obtained, the response and the comment including the evaluation value were returned to the conversation service department, and we entered the loop of the service for further clarification and concretization of the desire and providing awareness for that purpose by mirroring, delving deeper, and asking about the relationship with the sentences and phrases of past utterances in the conversation service department again (asking about the relationship between past utterances and the current sentence). Whether the desire has been fulfilled or the worry has been alleviated is confirmed by the function of the conversation service to summarize emotions, or by responses such as the user's "feeling refreshed", "resolved", "feeling motivated", "feeling good" (including natural language text and utterances), or by emotion sensing including multi-sensor fusion of the facial expressions, body temperature, and pulse of the image / audio user. When the specified value for the improvement of emotions is exceeded, the loop of high-reliability matching (concierge) is terminated.] "If the development speed could be ignored, there was a basic research institute, but now there are only national research institutes." Depending on the need, get a highly reliable answer with the help of the above-mentioned humans or AI avatars.

[0101] That is, automatic generation by a local (localized) LLM or generative AI of question generation change dictionaries / patterns / rules; the local LLM or generative AI changes the question and its field in order to increase the reliability and satisfaction of the answer of the generative AI. As a specific rally: In the conversation service, "I want to join Company A, but I can't program, so I'm anxious! I failed the programming exercise! What's the relationship with programming? If I can write good programs, I should be able to enter the research institute!" In the matching concierge 3, ask ChatGPT if the recruitment staff of Company A can conduct an aptitude test for the software research institute. Since an answer with an evaluation value above the reference value was not obtained for this question, the question was changed as follows: Does Company A have a program research institute? Does Company A have a software research institute? Does it have a program research institute? Does it have a software research institute? (If I can write good programs, should I be able to enter the research institute?)

[0102] As described above, the questions to the generative AI 12 for the fulfillment of wishes and the resolution of problems, worries, and obstacles, which are the outputs of VICA in the conversation service section, and the answers of the data processors of ChatGPT and large language models (LLMs) to these questions are repeatedly corrected by humans and machines to enhance the reliability of these answers. In the following, we will further explain, using a specific rally example, how an evolved digital twin AI that is individual and local (localized) for the user, service, and field can learn from the human-machine correction history described above and automate it.

[0103] Implementation Method Rally Example 2: Employment Problem 3) [Construction of an AI avatar that learns the history of human-machine evaluation correction by local LLMs; in-depth exploration such as VICA; clarification of the effectiveness of cooperation with counseling AI] U: Since I can create good programs, I should be able to get into a research institute. S: Since you can create good programs, you should be able to get into a research institute, right! So! U: Hope has welled up. (Emotional improvement, temporarily ending the conversation service VICA, the matching concierge 3 that has been passed the conversation history information and control asks questions to the commercial and general-purpose generative AI 12 including ChatGPT and obtains answers. If the human (a person with common sense, or a (former) salesperson, (former) guide, (former) counselor, i.e., a (quasi-)expert) in the human-machine dialogue unit 4 (matching dialogue client) deems the answer from the generative AI 12 (GPT) to be not (specially) commonsensically appropriate, it is possible to remake the question within the scope of the user's wishes, problems, and past speech history, and repeatedly ask the generative AI 12 (GPT) again to obtain an answer until the above appropriate answer is obtained. This appropriate answer is learned by the digital twin including the data processing machine of the local (personal, regional, field, service, store, or their group-specific localized) generative AI 12 or large language model (LLM). That is, the above history of question modification (and answer and its evaluation) by the above humans in the human-machine dialogue unit 4 (matching dialogue client) is memorized, an AI avatar that replaces those humans is constructed, and the automation of making the above answers including the generative AI 12 (such as ChatGPT) appropriate is achieved. In this way, the automation scope of the evaluation and improvement intelligence unit 6 of the matching intelligence server including the (evolved) digital twin (AI) with a distribution and integration processor that individually distributes and integrates the individual AI avatar, which is a local LLM, and its input and output, self-propagates (i.e., learns). To improve the reliability, quality, efficiency, and performance as a commercial product, a large number of simulated wishes and anxieties of individual local users including the group are prepared in advance, and the evaluation and improvement intelligence unit 6 of the matching intelligence server including the (evolved) digital twin (AI) with a distribution and integration processor that individually distributes and integrates the individual AI avatar, which is a local (localized) LLM, and its input and output, is pre-trained and learned offline using the history of pre-evaluating and modifying the simulated questions and answers by relevant experts and people with good sense. Furthermore, to improve the performance (reliability, efficiency), online and offline collaborative evaluation, correction, and learning and training from the history with humans, especially higher-level humans including other local or teacher's teacher connected via the Internet, and AI avatars are carried out.To further improve performance, including loops that involve asking questions and receiving answers from a web searcher and evaluating and modifying based on them, as well as loops for evaluating and modifying based on past implementation results, especially in the conversation service which is a feature of this application, including loops for evaluating and optimizing (improving) the above questions and answers by delving deep, listening, making associations, and tracing back to the desires and problems noticed, conduct a wider range and higher level of online and offline collaborative evaluation, modification, and learning and training from the history with human beings and AI avatars. S: Company A has a software research institute. Is there any problem or concern? Matching Concierge 3 verifies the results of the answers that could be obtained by asking GPT and returns them to VICA. VICA returns the answer to the user and displays 'Is there any concern?' or 'Are there any problems?' or 'Are there any anxieties or concerns?' and returns control at the start of counseling. U: I'm concerned that although the quality of the development program is high, the development speed is slow. S: So you're concerned (anxious) that although the quality of the development program is high, the development speed is slow. Specifically? U: Can you ignore the development speed? S: Can you ignore the development speed, indeed. When asking GPT through Matching Concierge 3 whether Company A's software research institute can ignore the development speed, the evaluation value is zero with the answer 'Depending on the situation and department'. So, ask the web server to get the URL and question whether the text there can be summarized by Generation AI12. That is, request a summary and obtain the summary as an answer. Since keywords related to time such as 'basic' or'research' or 'latest achievements' are not required, and conversely, although there are keywords related to rapid prototyping or POC but the result is too long and the evaluation is low. Therefore, further request Generation AI12 to generate a summary. If the summary is too much and keywords related to time such as 'latest achievements are not required' disappear and the evaluation drops, request the summary from the AI avatar (evolved digital twin) or a human being, and with the help of the above-mentioned human beings and AI avatars, obtain a highly reliable answer and pass it to the conversation service.

[0104] Implementation Method Rally Example 2: Life Problem 2 S: Is there anything you're worried about? U: It's hot and I'm afraid of getting heatstroke! S: It's hot and you're afraid of getting heatstroke. More details! U: I want an air conditioner, but I'm concerned about dust. S: You want an air conditioner, but you're concerned about dust. So U: Isn't there an air conditioner that doesn't let dust fly? [Since it can't be solved on its own, ask the generative AI (ChatGPT) via the matching concierge 3 (AI).] Generative AI answer: "I don't know about new ones with AI cutoff in 2019, but Mitsubishi air conditioners with HEPA filters are available." [Since the first "··· don't know" has a zero evaluation, answer the part excluding this as follows and return to the beginning of the counseling AI (VICA) of the conversation service] S: Mitsubishi air conditioners with HEPA filters are available. Is there any problem or concern? U: How much is the price? [Ask the generative AI (ChatGPT) via the matching concierge 3 (AI) again] Generative AI answer: "I don't know about new ones with AI cutoff in 2019." [Search for Mitsubishi air conditioners and HEPA filters with a web searcher. Obtain a URL as an answer.] Web searcher answer: URL: http··· [Return to the beginning of the counseling AI (VICA) of the conversation service] S: There is an answer at this URL: http···. Is there any problem or concern? U: I understand. Please use this. [Place an order via the Matching Concierge 3 (AI). Record the history for future use, input it into the digital twin's large language model, and let it learn. The digital twin has avatars for each user, each service, each field (including employment, enrollment, food, clothing, housing, education, plants, and medical care), and each region (including countries, states, and prefectures) (Figure 3). The above history is input and learned for each corresponding avatar (multiple avatars are possible). By having multiple avatars cooperate, the digital twin can evaluate, including simulating (mimicking), the answers of general / commercial / standard generative AIs represented by ChatGPT and web searchers. (Figure 6)]

[0105] Next, regarding the highly reliable and efficient remote service system according to the fifth embodiment of the present invention, particularly its application to business AI (automation), an explanation will be given with reference to the drawings.

[0106] [Fifth Embodiment] (Remote Service Support Device / System) Figure 6 is a functional block diagram showing an example of application of a highly reliable and efficient remote service support system according to the fifth embodiment, particularly for business AI (automation), which uses a listening-type counseling AI including (VICA) to determine the satisfaction of search results for highly reliable and efficient search of services such as generative AI 12 (including ChatGPT and its core program GPT4) and web searcher 11 (including Google Searcher) for wish fulfillment. That is, it relates to the support for the efficient use of services in a virtual space / metaverse composed of a computer including generative AI 12 and a web searcher 11 and a communication network, particularly regarding business AI (automation).

[0107] As shown in FIG. 2, this system includes a counseling AI (VICA: counseling input / output unit 1 and counseling judgment memory unit 2), a matching concierge 3 (matching server), a man-machine dialogue unit 4 (matching dialogue client), an evaluation and improvement intelligence unit 6 (matching intelligence server, digital twin, avatar: although the entity is a server (program and DB) on the web, in terms of its (intelligent) behavior and appearance represented by a 3D image, it transforms into a salesperson, a guide, etc., and substitutes for the verification of answers from a generation AI 12, etc. and high-reliability matching via VICA, and automates it, that is, an evolutionary digital twin or AI avatar), a reception / consultation service (AI substitution) 61, a customer service (AI substitution) 62, an order-made support (AI substitution) 63, a substitution ability (of a robot, avatar) 64, a web searcher 11 (Google searcher), a generation AI 12 (ChatGPT). Each of these servers (computers dedicated to services) may be any of a cloud server, an edge (cloud) server, and a local server. Considering response performance and price, SDN and slicing technologies are used to selectively connect these servers, enabling the efficient configuration of a virtual space with good cost performance by a computer (computer) and a network (communication network). SDN is an abbreviation for Software Design Network, which is a method of efficiently configuring a network (communication network) of computers (computers) by a program.

[0108] (1) The counseling input / output unit 1 (counseling conversation client) of the conversation service unit inputs voice / image data, etc. from the user, converts it into an utterance sentence and an emotion expression text, and sends it to the counseling judgment memory unit 2 (counseling server). The counseling judgment memory unit 2 (counseling server) receives this text and basically makes an unconditional positive interest or empathetic statement that does not depend on context or situation, such as nodding, paraphrasing, etc., and moreover, delves deeper with words such as "specifically", "more", "in more detail", etc., summarizes emotions and their changes, and when the same words are repeated, checks the relationship with wishes and problems, and creates a dialogue response text that is dedicated to specifically eliciting the user's worries and wishes and spontaneously making the user notice a solution.

[0109] (2) When it is difficult to solve independently, request the matching concierge 3 to summarize and convert the identified problem into questions or user requirements (question creation). Query the web searcher 11 and the generative AI 12 with the questions or user requirements, and extract answers including the URL and the content of the page (answer extraction). Evaluate and correct this answer to improve reliability, including dialogue with the expert AI (avatar, robot) included in the (evolving) digital twin (AI), simulation by the simulator in the digital twin (simulation), or utilization of the history of previous implementation results in the digital twin (answer evaluation and correction). The matching concierge 3 repeats this process. If an answer with an evaluation value equal to or higher than the specified value is obtained, the evaluated, corrected, and verified answer is transmitted to the conversation service (repeated answer creation, verification, and transmission). This dialogue / (simulation) implementation evaluation and correction history is input and learned locally (for individuals, regions, and fields) into the digital twin including the large language model as a history (log). Initially, since it is difficult to perform without a history (log), human experts including the original sales representatives ask questions with simulated data to the web searcher 11 and the generative AI 12, log the evaluation and correction history for learning and training, and improve the performance. If this is still insufficient, the above-mentioned human experts evaluate and correct including checking the questions, log the history, and improve the performance online until the level where human experts are not required is reached. More specifically, it is as follows.

[0110] Including the dialogue input history, the matching unit learns by storing in the (evolving) digital twin including the large language model (weights) the user's wishes, problems, requests, questions and answers to the generative AI 12 and their histories, and the feedback (evaluation, re-execution, improvement) history of the execution of the matched services. In particular, from the dialogue system unit, including by specialty, service type, and region, generally in advance, for each individual, (special) region, and service, it is always online and also includes learning by checking, verifying, and correcting the ways of asking questions and answers to the generative AI 12 (ChatGPT) from the dialogue system unit through the Web (collective wisdom by web searchers) and SNS, etc. That is, the memory and learning of the digital twin including the large language model include the memory and learning of the history (data) of evaluating and correcting questions and answers to the generative AI 12 in the human-machine dialogue unit 4. This evaluation and correction in the human-machine dialogue unit 4 are also carried out offline in advance by gathering experts in the case of roughly dozens or fewer service types and regions. When it comes to each individual, each individual service, and each individual / special region, it is always online, and includes the memory and learning of the history data of evaluation and correction including cooperation with experts in remote locations including former salespersons via the Web and SNS from the human-machine dialogue unit 4. Based on the learned data, questions to the above generative AI 12 and their answers are generated. High-reliability matching is performed including the iterative loop for comparison verification and improvement.

[0111] (3) Present this answer to the user and ask as if listening attentively to the user's concerns, that is, after the answer (and comment), respond with "Do you have any concerns?", or "Do you have any problems?", or "Do you have any problems or concerns?", or "Do you have any uneasiness?", or "Do you have any uneasiness or concerns?" (answer satisfaction confirmation).

[0112] (4) If there is a problem, clarify the question through self-awareness (self-recognition) by means of (conversation) rallies involving mirroring, digging deeper, emotional summarization, and relationship evocation, break it down into specific and detailed elements, and attempt to solve it independently (conversation verification rally for answers).

[0113] (5) If it becomes difficult to solve independently, request again from Matching Concierge 3 to create an evaluable answer, that is, create the answer by performing the above-mentioned question creation, answer extraction, evaluation, correction, and verification, and convey it to the conversation service. (Answer creation and evaluation rally)

[0114] (6) In the conversation service department, ask the user whether there are any problems or concerns along with the answer, and repeat the above loop until the user is satisfied, that is, until it is recognized that there are no problems or there is an improvement in the emotion. If there are no problems including the above-mentioned independent solution, convey this to Matching Concierge 3, and at this point when ending the conversation service for matching that satisfies the current user, the entire loop before implementation ends.

[0115] However, there may be cases where the conversation service (counseling AI) is restarted after the service is implemented (reservation / execution), including when problems occur such as when a reservation cannot be made or when the ordered product arrives and is used. Listen to the concerns and problems regarding the purchased product or the service provided, conduct listening-type conversations and in-depth exploration to resolve dissatisfaction independently, or if it cannot be resolved independently, convey it to Matching Concierge 3 and repeat the steps of using Generate AI 12, Web Searcher 11, AI Avatar or interacting with experts to explore new services. Repeat these steps until the entire loop including after implementation ends. If the loop does not end even after repeating more than the specified number of times, consider it the limit of the conversation service which is an AI counselor, and a human counselor will be introduced through Matching Concierge 3. The history of these listening-type conversations and matching through Matching Concierge 3 is stored as a log in an evolutionary digital twin AI (AI Avatar, AI Robot) including a large language model and a neurocomputing model for each individual, region, and field, and is used for their learning and training.

[0116] Also, when it is difficult to resolve issues independently in the conversation service department, the determination is made as follows: (1) The user makes statements in the form of questions or interrogative sentences regarding 5W1H (When, Where,...). (2) Even after repeating the conversation rally a specified number of times or more, no improvement in sentiment exceeding the specified value is observed in image recognition including facial expression recognition or sensor fusion including sphygmomanometers and thermometers. And for the determination of the end of all loops in the conversation service department and the matching department, after asking the user if there are any problems or concerns in the conversation service department, (1) Recognize an improvement in the user's sentiment in natural language, including expressions such as "resolved" or "relieved". (2) An improvement in sentiment exceeding the specified value is observed in image recognition including facial expression recognition or sensor fusion including sphygmomanometers and thermometers.

[0117] Google Search, which displays related websites when keywords are entered, and ChatGPT, which returns answers when question sentences are entered, have also been developed. However, the user needs to consider the keywords or question sentences that should be entered to solve problems. It is often the case that satisfactory answer sentences or data cannot be obtained from the websites of the URLs output by Google Search, and answer sentences such as those from ChatGPT need to be customized for personal use or at least correctly evaluated and corrected to determine if they are appropriate for oneself. In most cases, this evaluation and correction can only be done by humans, including the person himself / herself, sales representatives, local stores, and service experts. Regarding social soundness, local or individual evaluation and correction are required using databases of knowledge such as administrative experts and laws, including sensible people and regions. As in the above embodiments, this system can support the search and execution of remote services efficiently and with high reliability, compensating for the shortage of human resources, by automating the evaluation and correction of these questions and answers using an inference machine or AI that memorizes past (successful cases of obtaining highly evaluated answers) or learns and trains based on them.

[0118] As described above, including the local and human expert learning and training in (2), the listening-type conversation service VICA (Counseling AI [awareness, creative support]) + Matching Concierge 3 (Matching AI [search, verification, unique information sharing and inheritance] + Digital Twin AI [simulation verification, accumulation, inheritance, and sharing of unique local information by large language model LLM]) achieves the substitution capabilities of the AI robots and AI avatars declared below.

[0119] That is, the following tasks 1] - 3] are performed by this AI (Artificial Intelligence), which is an AI avatar or IOT robot, to sustain the development of services such as stores and companies. By providing the following reception desks, professional business support, and service improvement information of this AI (Artificial Intelligence), it supports cost reduction and the promotion of in-house improvements. 1] Reception and consultation services: It is possible as a reception AI avatar / IOT robot to support primary reservation and application services for user inquiries (which are costly for humans and insufficient in traditional chats). 2] Customer service: Automate customer service such as product explanations using daily conversations including professional levels by AI avatars / IOT robots. Also, by verifying and achieving user satisfaction exceeding the answers of Generation AI 12 using a listening-type conversation that listens to the user's desires and provides awareness, propose new (especially) products and services that are difficult in traditional chats including Generation AI 12. 3] Made-to-order services: Perform customization (personalization) for the sustainable (inheritance) and development of products (including buildings and plants) / services unique to stores / companies (sellers) and customers, such as salesperson AI characters (avatars, robots). To facilitate practical implementation, the proposed devices, systems, methods, and programs can be selectively implemented step by step from simple to complex and high-quality ones in terms of functions and usage methods, and according to the purpose of use, ease of use, costs, benefits, effects of practical implementation, maintainability, and reliability. (1) Regarding the matching part, the answer is only queried to Generation AI 12 such as ChatGPT. (2) Enable the answer to be queried to multiple sites including web searchers such as Google Searcher. (3) If no answer or good answer can be obtained by the AI 12, make it possible to obtain an answer from a web searcher, etc. (4) Do not evaluate the answer. (5) Automatically evaluate the answer using a physical simulator such as a digital twin. (6) Automatically evaluate and correct (improve) the answer based on the built-in evaluation dictionary / database, evaluation criteria / rules. (7) Support the man-machine dialogue evaluation and correction (improvement) of the answer by experts, etc. (8) Remember and learn the successful cases of evaluation and correction (improvement) by automatic or experts, etc. (including the man-machine dialogue content depending on the system level), and use the evolving AI to automatically evaluate and correct (improve) the answer. (9) Visualize the above AI as a two-dimensional or three-dimensional AI avatar, and use the stored man-machine dialogue data to explain the answer and its evaluation content to enhance the user's satisfaction and trust. (10) Assign the IDs of users, experts, shops, institutions, regions, fields, time zones / periods / seasons related to the successful cases, individualize (localize) them for memory and learning, and perform integrated application by weighting the evaluation according to the ID matching degree during evaluation and correction, etc., to enable the evaluation and inference of the answer by individual (local) or localized inference. (11) Make it possible to explain the answer and its evaluation content using the stored man-machine dialogue data of the expert avatar with a high ID matching degree in individual (local) localized inference. Furthermore, display the explanations / images of the shop / institution, region, field to which it belongs, and the images / avatars of the user's family / ancestors / acquaintances according to the ID, and enable the explanation and conversation / dialogue of the answer and its evaluation content. (12) For the conversation service section, only the input / output function of questions and answers to the matching section by text or voice is provided. Depending on the functions of the matching section, it is possible to only ask questions to the generative AI 12, ask questions to multiple search engines such as web searchers, and obtain evaluations of answers. That is, more than 10 levels of concierge, that is, consulting or service guidance, can be received. However, since only the evaluation of the answer is output, or only whether to approve or reject it, or only ask different questions, even if there are concerns, a listening-type conversation service that concretizes and clarifies the problems cannot be received. (13) Return comments such as answers and evaluation values obtained by asking questions to the generative AI 12 and the web searcher 11 to the conversation service section. The listening-type conversation service can be received only once when clarifying wishes and problems before asking questions that cannot be solved initially. (14) For the answers obtained by asking questions to the generative AI 12 and the web searcher 11, add a confirmation sentence (about problem points) such as "Are there any concerns?" to the listening-type conversation service and return it. Utilize the problem clarification and in-depth exploration functions of the listening-type conversation service, carefully consider the problem points of the answers obtained by asking questions to the generative AI 12 and the web searcher 11, and repeat the re-questioning with the problem itself concretized until a satisfactory answer is obtained. (15) Regarding the reservation, execution, and evaluation of services that are matched (selected and searched) and guided and introduced (concierge), etc., each stage such as i) only matching, ii) only notification, iii) only reservation and execution, iv) up to evaluation, and v) also performing online settlement is selectively implemented. In iii), through the introduction and implementation of conversations, dates, games, trips, and various other aspects such as friends, partners, places, and systems in the metaverse space (a world that combines virtual and real spaces), a chat (conversation, meeting, guidance) system, a dating system, a travel system, etc. can be implemented in the metaverse that includes virtual space (3D avatars), objects (including robots and IoT), services, and actual people, objects, and services (a world where virtual and real spaces are connected and integrated via the internet, etc.). When proceeding to iv) up to evaluation, it becomes possible to improve those evaluations, that is, SDGs, and it becomes possible to construct and evolve the metaverse culture and civilization. At the v) stage, all of these can be enjoyed with net money. Among the above, the stages 1)-11) which are the stages of the matching section, the stages 12)-14) which are the stages of the conversation service section, and the stages i)-v) regarding the execution such as reservation, execution, evaluation, and settlement of the matched services in 15) are independent of each other, and are appropriately combined according to development costs, ease of use, usability, etc., and effectively utilized. For example, in step iii), a metaverse space that fuses reality and virtuality is given IDs related to users, experts (salespersons), shops, seasons, etc. in the successful cases of the service in (10), individualized (localized), and stored and learned. And with the individualized (localized) inference in (11), even without going to the local area, even if the expert salesperson has transferred, or even if the season has changed, answers and explanations of the evaluation content are provided as if the salesperson is having a conversation with a real user. Moreover, with the avatar simulator and knowledge, the situation in different seasons can also be explained. Furthermore, the description / image of the shop to which the user belongs, the images of old acquaintances or ancestors that the user can rely on, and the avatar are also displayed according to the ID registered as user attributes, enabling conversations and consultations like answering and explaining the evaluation content and listening attentively like a mother in a metaverse that fuses the real and virtual spaces. This is not only highly effective for mental distress consultations but also often very effective for simple shopping consultations.

[0120] Example of implementation method rally 3: Life problem 3; Experimental example S(VICA): Please tell me what you are worried about. U(USER): I want to buy an air conditioner, so please tell me. [Since I can't solve it myself, I ask the generative AI12 (ChatGPT) through the matching concierge 3. The following is the conversation in the matching section] Matching concierge 3: I want to buy an air conditioner, so please tell me. Generative AI answer: I can introduce it specifically if I know the size of the room and the manufacturer, etc. If the matching concierge 3 cannot obtain an answer from the generative AI 12 and thus cannot return a matching plan to the user in the conversation service department, but is instead being asked by the generative AI 12, this results in a zero evaluation. Since the generative AI has a learning ability and is likely to change its response the second time, ask the same question again. If that still doesn't work, change the question by adding "unconditionally" or the like. If the generative AI 12 fails, change the question target to a web searcher (keywords: air conditioner, purchase, etc.). The conversation service department and the matching department are independent. Instead of specifically asking the users of the conversation service about the room size, manufacturer, etc. in addition to the requirements, they faithfully ask questions according to the wishes and concerns listened to from the users. After obtaining some kind of answer from the matching concierge 3, ask the user if there are any problems with the air conditioner in the answer. The above response of the generative AI 12, that specific introduction can be made if the manufacturer and room size are known, is added to the answer as an accompanying comment after obtaining an evaluable answer. However, it is only an option for alerting problems, and it is up to VICA, a listening-type and non-intrusive (not an opposing conversation) conversation service, to decide whether to enable it. Whether to attach this option as a comment can be determined by the evolved digital twin or AI avatar in the matching department based on the learning results of the adoption rate of previous options. Even if an air conditioner for a factory is returned as an answer, the user may not think it is a problem. If the user thinks it is a problem, the listening-type counseling of the conversation service department VICA can inquire about the problem, and the things the user really wants to know, wishes, and problems will become clear. This is the advantage of the idea of this patent that combines VICA and the concierge. That is, VICA promotes the discovery, concretization, and clarification of problems through the user's self-awareness, introspection, and inspiration. The concierge uses AI such as evolved digital twins and AI avatars that utilize human-machine cooperation with ChatGPT, web searchers, and experts, as well as previous solution data and its learning results, to solve the problems discovered, concretized, and clarified by the user. In this way, it enables the collaboration between the discovery, concretization, and satisfaction evaluation of solutions in VICA, which can only be solved by humans, especially the user's introspection and awareness, and the creation of automated problem solutions in the concierge.

[0121] An example of the above embodiment will be described below. Matching Concierge 3 (asking questions to the generative AI): I want to buy an air conditioner, so please tell me. (Or, emphasizing that there are no specifications for the room size or manufacturer in the user's question) I want to buy an air conditioner unconditionally, so please tell me. [When it is not so, since questions other than the above answer of the generative AI are directly passed to the user via VICA, the user directly interacts with the generative AI.]

[0122] U (USER): The room size is 16 tatami mats, and the manufacturer is Daikin. Generative AI (ChatGPT) answer: ··· Introduction of Daikin air conditioner series [Since the answer is poor and the evaluation is low, ask the user of the conversation service for more specific requirements.] U (USER): Something with a humidity control function [Generative AI] ··· Corresponding series [Since the answer is still poor and the evaluation is low, the user of the conversation service ends the question.] U (USER): I will end. S (VICA): I will end. Please continue the conversation. U (USER): Please search for a list of air conditioners manufactured by Company A. [Judging that it cannot be solved independently, ask through the matching section. Since the interaction with the generative AI has ended under the direct control of the user, the generative AI stops and an answer is obtained from the web server.]

[0123] S (VICA): There are search results. Is there anything you are concerned about? Company A (https: / / www.○○.co.jp) In the above embodiment, the user of the conversation service directly and short-circuitingly evaluates and controls the generated AI's answer without going through the matching unit, and ends the question to the generated AI. Although the system is simple, it is not a listening-type conversation service where the user interface listens to requests (wishes and worries) according to the user's position and feelings. Instead, the matching unit and even ChatGPT reverse-request the user, so the mental and work burden on the user is large. In the present invention, basically, the matching concierge passes the GPT answer to VICA, and VICA does not ask questions or give suggestions to the user or client / patient. It conveys the answer within the range asked, adding "Is there anything you're concerned about?" As in the above example, if there seems to be no answer from ChatGPT and instead questions come from ChatGPT, the matching concierge, for example, searches on Google Search, finds the URL, passes it to VICA as an answer, and listens to whether there are any problems. Regarding the user or patient, it does not question them from a superior perspective like "I (ChatGPT) would ask specifying the room size and manufacturer too, because such a question can't give a specific good answer." This is contrary to the essence of Rogers' listening-type counseling used in the present invention. With unconditional positive regard, like a mother wrapping her child, in this case too, faithfully answering the user's question opens the user's heart and prompts introspection, (letting the user notice the problems with the questions and answers in this conversation example). This is the feature of the listening-type conversation counseling, and the proposed technology utilizes this.

[0124] On the other hand, in the following embodiment, the independence of the listening-type conversation service unit is high, and the basic evaluation, correction, and control of answers such as those from ChatGPT are entrusted to the matching unit. The following is considered to be highly reliable as a system because the matching unit evaluates and corrects multiple times until the user obtains a satisfactory solution.

[0125] Example of implementation method rally example 3A: Life problem 3; Example S: Is there anything you're worried about? U: It's hot and I'm afraid of getting heatstroke! S: It's hot and you're afraid of getting heatstroke. More details! U: I want an air conditioner, but I'm concerned about dust. S: You want an air conditioner, but you're concerned about dust. So U: Isn't there an air conditioner that doesn't let dust fly? [Since it can't be solved independently, ask the generative AI via the matching concierge 3.]

[0126] Matching Concierge 3: Isn't there an air conditioner that doesn't let dust fly? Generative AI Answer: "I don't know about new ones with an AI cutoff in 2019, but there is an H company air conditioner with a HEPA filter." [The first "··· don't know" has a zero evaluation, but the second half after excluding this can be evaluated, so answer as follows and return to the beginning of the counseling AI (VICA) of the conversation service] S: There is an H company air conditioner with a HEPA filter. Is there any problem or concern? U: How much is the price? [Ask the generative AI such as ChatGPT again via the matching concierge 3] Generative AI Answer: "I don't know about new ones with an AI cutoff in 2019." Matching Concierge 3 (to web searchers such as Google Searcher): Air conditioner manufactured by H Co., Ltd., with HEPA filter, price [Search for H company air conditioners and HEPA filters on the web. Get a URL as the answer] Web Searcher Answer URL: http··· [Return to the beginning of the counseling AI (VICA) of the conversation service] S: There is an answer at this URL: http···. Is there any problem or concern? U: I understand. There is no problem with this.

Explanation of symbols

[0127] In FIG. 1 1 ··· Counseling input / output unit 2 ··· Counseling judgment and memory unit 3 ··· Matching concierge 4 ··· Man-machine dialogue unit 5 ··· Service (meta-service group) 6 ··· Evaluation and improvement intelligence unit 7 ··· Memory unit 8 ··· Conversation recognition server 9 ··· Information recognition server 10 ··· IoT input / output device, robot, avatar 11 ··· Web searcher 12 ··· Generation AI 100 ··· High-efficiency remote service support device

[0128] In FIG. 3, it is as follows 1 ··· Counseling input / output unit 2 ··· Counseling judgment and memory unit 3 ··· Matching concierge 4 ··· Man-machine dialogue unit (human-machine function for high reliability, which is the human interface of the matching concierge that can utilize generation AI, etc.) 5 ··· Service (wrapped as a service server / client group) 6 ··· Evaluation and improvement intelligence unit ((evolved) digital twin, (AI) avatar: materialize (transform into a server (program and DB) on the web) a salesperson or a guide lady, and automate the verification of answers such as generation AI and high-reliability matching via VICA) Matching intelligence server (alias: digital twin, evolved digital twin, avatar, AI avatar): Materialize and transform a salesperson or a guide lady into a server (program and DB) on the web, and perform verification (evaluation and correction) of answers such as generation AI and automation of high-reliability matching via the counseling conversation service VICA 7 ··· Memory unit 8 ··· Conversation recognition server ​9 ··· Information Recognition Server 10 ··· IoT Input / Output Device, Robot, Avatar 11 ··· Web Searcher 12 ··· Generative AI 101 ··· High-Efficiency Remote Service Support System

[0129] In Figure 3, as follows 6 ··· Evaluation and Improvement Intelligence Unit (Digital Twin, AI Avatar)

[0130] In Figure 6, as follows. 3 ··· Matching Concierge 6 ··· Evaluation and Improvement Intelligence Unit (Also known as: Digital Twin, Evolving Digital Twin, Avatar, AI Avatar): Transforms sales representatives and guides into servers (programs and databases) on the web and materializes them, verifies answers from generative AI, etc., and automates highly reliable matching via VICA 61 ··· Reception and Consultation Services (AI Proxy) 62 ··· Customer Service (AI Proxy) 63 ··· Customized Support (AI Proxy) 64 ··· Substitution Capabilities of Robots and Avatars

Claims

1. A remote service support device for supporting a user in solving a wish or a problem, comprising: The remote service support device includes a conversation service unit including a counseling input / output unit and a counseling judgment storage unit, and a matching unit including a matching concierge, The counseling input / output unit includes: accepting user information including voice, image or text information relating to the user's wishes or worries, converting voice or image information into text, and transferring the user text information in which all of the user information has been converted into text to a counseling judgment storage unit; Transmitting the conversation response information from the counseling judgment memory unit to a user using a speaker, a display, or an external device; The counseling judgment storage unit includes: generating a natural language text response for encouraging the user to realize and solve the problem by themselves, which is necessary for solving the problem, in response to the user text information from the counseling input / output unit; storing a conversation log including user text information; Judging whether the user's wish or concern has been resolved based on the words and emotional change information in the stored conversation log; When it is determined that the wishes or concerns of the user are difficult to solve, a conversation log regarding these wishes or concerns is provided to the matching concierge of the matching section; The matching concierge: The conversation log received from the counseling judgment storage unit is queried to a generation AI or a web searcher; A remote service support device in which, in the counseling judgment memory unit, a confirmation message is added to the proposed answer returned from the generating AI or the Web searcher, and the confirmation message is returned to the counseling input / output unit.

2. A remote service support system comprising a counseling conversation client including a counseling input / output unit, a counseling server including a counseling judgment storage unit, and a matching server including a matching concierge, The counseling input / output unit includes: converting voice or images relating to the user's wishes or worries into text, passing the user text information in which all of the user information has been converted into text to a counseling judgment storage unit, and transmitting conversation response information from the counseling judgment storage unit to the user using a speaker, a display, or an external output device; The counseling judgment storage unit includes: generating a natural language text response for encouraging the user to realize and solve the problem by themselves, which is necessary for solving the problem, in response to the user text information from the counseling input / output unit; storing a conversation log including user text information; Judging whether the user's wish or concern has been resolved based on the words and emotional change information in the stored conversation log; When it is determined that the wishes or concerns of the user are difficult to solve, a conversation log regarding these wishes or concerns is provided to the matching concierge of the matching section; The matching concierge: The conversation log received from the counseling judgment storage unit is queried to a generation AI or a web searcher; A remote service support system in which the counseling judgment memory unit adds a confirmation message to the proposed answer returned from the generation AI or the Web searcher and returns it to the counseling input / output unit.

3. The system further includes a matching intelligence server including an evaluation and refinement intelligence unit, the evaluation and refinement intelligence unit evaluates or refines the answer proposals of the generating AI and the web searcher, 3. The system of claim 2, wherein the evaluation refinement intelligence comprises knowledge or a database used to evaluate or refine the proposed answers.

4. The remote service support system of claim 2, further comprising a matching dialogue client having a man-machine dialogue unit, the man-machine dialogue unit supporting expert evaluation or correction of a conversation log to be passed to the generating AI or the Web searcher, or a proposed answer returned from the generating AI or the Web searcher.

5. The evaluation and improvement intelligence unit has a digital twin that simulates a human's judgment or action as an avatar, In the digital twin, learning or training is performed regarding the method and memory of evaluation or correction by the expert; The remote service support system according to claim 3, further comprising: a step of utilizing the results of learning or training to automatically extract and evaluate answer proposals of the generating AI or the Web searcher, and correcting the proposals if the evaluations are poor.

6. The remote service support system according to claim 5 , wherein the digital twin has a learning machine including a large-scale language model LLM, and the learning or training is performed by the learning machine.

7. 3. The remote service support system according to claim 2, wherein the counseling judgment memory unit, when judging that the user's wish or concern can be resolved from the proposed answer and the conversation response information of the user to the confirmation message, books, executes or evaluates the proposed answer service.

8. A remote service support method using the remote service support device according to claim 1 or the remote service support system according to any one of claims 2 to 7, comprising: a step of receiving a user's utterance including voice, image or text information in a counseling input / output unit, converting the voice or image into text, and transferring the user's text information, which is all the user's utterance in text, to a counseling judgment storage unit; A step of judging whether the user has a problem including a wish or a worry from the user text information by a counseling judgment storage unit, and whether the wish or the worry is problem-solvable; If it is determined that the problem can be solved, a natural language text response is generated including a paraphrase for attentive listening conversation for promoting self-awareness or self-realization, a concretization promotion, or a relation inquiry with the previously spoken phrase, and the natural language text response is passed to a counseling input / output unit; communicating, by the counseling input / output unit, the natural language text response to the user using a speaker, a display, or an external device; storing a conversation history including the wish or worry as a conversation log in a counseling judgment storage unit; determining whether the wish or worry has been resolved based on the words and emotional change information in the conversation log, and terminating the conversation service if the wish or worry has been resolved; a step of, when it is determined in the step of determining whether the wish or worry is solvable, a counseling determination storage unit delivering the conversation log to a matching concierge to request support for solving the problem; A matching concierge creates a question including a phrase from the conversation log; A matching concierge asks the question to a generating AI or a web searcher and extracts answer proposals; evaluating the answer plan with an evaluation and refinement intelligence unit; In the evaluation and improvement intelligence unit, if the evaluation value of the answer plan is equal to or greater than a specified value, the answer plan is returned to the user together with a confirmation message via the counseling judgment storage unit and the counseling input / output unit. In the evaluation and improvement intelligence unit, when the evaluation value of the answer plan is less than a specified value, the evaluation or correction is repeated within a specified range until the evaluation value becomes equal to or greater than the specified value, or the question or the answer plan is passed to a man-machine dialogue unit, and an expert evaluates or corrects the question or the answer plan so that the evaluation value becomes equal to or greater than the specified value; If the evaluation value is less than the predetermined value despite the evaluation or correction being performed a predetermined number of times, determining in the evaluation improvement intelligence unit that the evaluation value is the largest or that selected by the expert is the answer; storing or learning a history of the questions or answers in an evaluation and refinement intelligence unit for automatically generating, evaluating or revising the questions or answers; If the counseling judgment storage unit judges that the user to whom the answer proposal is returned cannot solve the problem by himself / herself, the step of continuing the conversation toward the self-solving is performed. If the user to whom the answer proposal is returned is able to solve the problem by himself / herself, assisting the user in reserving, performing, or evaluating the corresponding service; a step of returning a response plan including a comment on the result of the reservation or execution and a confirmation message to the user via the counseling judgment storage unit and the counseling input / output unit when the result of the reservation or execution is obtained; If the user has any concerns about the proposed answer, the conversation service section resumes the conversation to clarify and specify the problem so that the user can solve the problem on his / her own. If the user is unable to solve the problem on his / her own, the matching concierge is resumed. This process is repeated until the problem is resolved. If the problem is solved by the user, the step of requesting a matching concierge to reserve, execute or evaluate the answer and store or learn the history, and then terminating the conversation; A remote service support method comprising:

9. A program for executing the remote service support device according to claim 1 or the remote service support system according to any one of claims 2 to 7, a step of receiving a user's utterance including voice, image or text information in a counseling input / output unit, converting the voice or image into text, and transferring the user's text information, which is all the user's utterance in text, to a counseling judgment storage unit; A step of judging whether the user has a problem including a wish or a worry from the user text information by a counseling judgment storage unit, and whether the wish or the worry is problem-solvable; If it is determined that the problem can be solved, generating a natural language text response including a paraphrase for attentive listening conversation for promoting self-awareness or self-realization, promotion of concretization, or a relation inquiry to previously spoken words and phrases, and passing the generated natural language text response to a counseling input / output unit; communicating, by the counseling input / output unit, the natural language text response to the user using a speaker, a display, or an external device; storing a conversation history including the wish or worry as a conversation log in a counseling judgment storage unit; determining whether the wish or worry has been resolved based on the words and emotional change information in the conversation log, and terminating the conversation service if the wish or worry has been resolved; a step of, when it is determined in the step of determining whether the wish or worry is solvable, a counseling determination storage unit delivering the conversation log to a matching concierge to request support for solving the problem; A matching concierge creates a question including a phrase from the conversation log; A matching concierge asks the question to a generating AI or a web searcher and extracts answer proposals; evaluating the answer plan with an evaluation and refinement intelligence unit; In the evaluation and improvement intelligence unit, if the evaluation value of the answer plan is equal to or greater than a specified value, the answer plan is returned to the user together with a confirmation message via the counseling judgment storage unit and the counseling input / output unit. In the evaluation and improvement intelligence unit, when the evaluation value of the answer plan is less than a specified value, the evaluation or correction is repeated within a specified range until the evaluation value becomes equal to or greater than the specified value, or the question or the answer plan is passed to a man-machine dialogue unit, and an expert evaluates or corrects the question or the answer plan so that the evaluation value becomes equal to or greater than the specified value; If the evaluation value is less than the predetermined value despite the evaluation or correction being performed a predetermined number of times, determining in the evaluation improvement intelligence unit that the evaluation value is the largest or that selected by the expert is the answer; storing or learning a history of the questions or answers in an evaluation and refinement intelligence unit for automatically generating, evaluating or revising the questions or answers; If the counseling judgment storage unit judges that the user to whom the answer proposal is returned cannot solve the problem by himself / herself, the step of continuing the conversation toward the self-solving is performed. If the user to whom the answer proposal is returned is able to solve the problem by himself / herself, assisting the user in reserving, performing, or evaluating the corresponding service; a step of generating an answer plan and a confirmation message including a comment on the result of the reservation or execution by the counseling judgment storage unit when the result of the reservation or execution is obtained, and returning the answer plan and the confirmation message to the user via the counseling input / output unit; If the user has any concerns about the proposed answer, the conversation service section resumes the conversation to clarify and specify the problem so that the user can solve the problem on his / her own. If the user is unable to solve the problem on his / her own, the matching concierge is resumed. This process is repeated until the problem is resolved. If the problem is solved by the user, the step of requesting a matching concierge to reserve, execute, or evaluate the answer and store or learn the history, and then terminating the conversation; A program that executes.

10. A remote service support method executed by using a client-server system, the client-server system including a counseling input / output unit, a counseling judgment storage unit, and a matching concierge in a client or a server, a step of receiving a user's utterance including voice, image or text information in a counseling input / output unit, converting the voice or image into text, and transferring the user's text information, which is all the user's utterance in text, to a counseling storage unit; a step of judging, in a counseling judgment storage unit, whether or not the user has a problem including a wish or a worry from the conversation log of the user text information, and whether or not the wish or the worry is solvable; If it is determined that the wish or the worry is unsolvable, the conversation log is handed over to a matching concierge and a problem solving support request is made; A matching concierge creates a question including a phrase from the conversation log; A step of a matching concierge asking the question to a generation AI or a web searcher and extracting an answer proposal from the generation AI or the web searcher; The matching concierge passes the proposed answer to the counseling judgment memory unit, generates a confirmation message regarding the proposed answer in the counseling judgment memory unit, and returns the proposed answer and the confirmation message to the user via the counseling input / output unit.

11. The remote service support system further comprises an evaluation and improvement intelligence unit, The method for remote service support according to claim 10, further comprising the step of providing an evaluation value to an answer proposal from the generating AI or the Web searcher in the evaluation refinement intelligence unit.

12. The remote service support method of claim 11, further comprising a step of: if the evaluation value of the proposed answer is less than a specified value, a matching concierge repeats the evaluation or correction within a specified number of times until the evaluation value becomes equal to or greater than the specified value; or a step of supporting an expert in evaluating or correcting the question or the proposed answer so that the evaluation value becomes equal to or greater than the specified value.

13. The method of claim 12, further comprising the step of storing or learning a history of the questions or the proposed answers in an evaluation and refinement intelligence unit for automatically creating, evaluating or modifying the questions or the proposed answers.

14. The method for remote service assistance according to claim 11 , further comprising the step of: based on the proposed answer and the user's response to the confirmation message, the counseling judgment storage unit supports the reservation, execution or evaluation of a corresponding service.

Images