Program, information processing device, information processing system, and information processing method

Abstract

This invention provides a program, information processing device, information processing system, and information processing method that can facilitate the introduction of language models for email control. [Solution] An information processing device for delivering emails to a specified recipient receives an email sent from a sender. It evaluates the content of the email using a language model. Based on the evaluation result, it determines whether to deliver or hold the email. If it is determined that the email should be delivered, it delivers the email. If it is determined that the email should be held, it holds the email (delivery is suspended until a sending instruction is received).

Description

Technical Field

[0001] The present invention relates to a program, an information processing apparatus, an information processing system, and an information processing method.

Background Art

[0002] Conventionally, e-mail has been used as a communication tool, and there is an MTA (Mail Transfer Agent) that receives an e-mail from an MUA (Mail User Agent) and performs a process of delivering (distributing, transmitting, transferring) the e-mail.

[0003] Here, the MUA is a client program used to send and receive e-mails on a terminal on the Internet, and the MTA refers to a mail server that receives and delivers e-mails on the network through the SMTP (Simple Mail Transfer Protocol).

[0004] For example, as a conventional technique, in order to prevent misdelivery, there is a mail server that determines the delivery timing for each destination of an e-mail and holds the delivery to a destination whose delivery timing is not immediate (Patent Document 1).

[0005] In recent years, there is also a mail system that uses a large language model, such as obtaining subject candidates using a large language model (Patent Document 2).

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0007] However, in conventional email systems, the use of language models was limited to certain controls, such as obtaining subject line suggestions using language models. Furthermore, various language models have emerged and are evolving in recent years. It is expected that appropriate language models will be utilized for controlling email depending on the situation.

[0008] The present invention addresses the above-mentioned problems and aims to provide a program, an information processing device, an information processing system, and an information processing method that can facilitate the introduction of a language model in relation to email control. [Means for solving the problem]

[0009] (1) The present invention is A program that delivers emails to a specified recipient, The receiving department accepts emails sent from the sender, An evaluation unit that uses a language model to evaluate the content of an email, A delivery determination unit determines whether or not to deliver an email based on the evaluation results evaluated by the aforementioned evaluation unit, The present invention relates to a program characterized by causing a computer to function as a delivery unit that controls the delivery of an email when it is determined that an email should be delivered.

[0010] The present invention relates to an information storage medium storing the above-mentioned program, an information processing device (for example, a server device) configured as the above-mentioned parts, an information processing system configured as the above-mentioned parts, and an information processing method for executing the above-mentioned parts.

[0011] According to the present invention, a language model is used to evaluate the content of an email, and a decision is made on whether or not to deliver it based on the evaluation result, thus promoting the introduction of language models.

[0012] This invention can prevent erroneous transmissions by, for example, sending an email if the evaluation result is positive and canceling the delivery if the evaluation result is negative. As a result, it is possible to implement email delivery control that is highly convenient for users.

[0013] (2) Furthermore, the program, information storage medium, information processing device, information processing system, and information processing method of the present invention are A hold determination unit that determines whether or not to withhold delivery of an email to the recipient based on predetermined conditions, For emails that have been determined to be held for delivery to their destination, a computer is configured to function as a holding unit for those emails. The evaluation unit, For pending emails, a language model may be used to evaluate the content of those emails.

[0014] According to the present invention, the content of emails is evaluated using a language model, targeting only those emails that have been held among the received emails. This allows the use of the language model to be limited to emails that have been held, thereby reducing the processing load.

[0015] (3) Furthermore, the program, information storage medium, information processing device, information processing system, and information processing method of the present invention are A hold determination unit that determines whether or not to withhold delivery of an email to the recipient based on predetermined conditions, For emails that have been determined to be held for delivery to the recipient, the holding section that holds the email, The computer is further configured as an evaluation target determination unit that determines whether or not to include a pending email in the evaluation based on the evaluation target conditions, based on the evaluation target conditions. The evaluation unit, Of the pending emails, the emails that are subject to evaluation may be evaluated using a language model to assess the content of those emails.

[0016] According to the present invention, for appropriate emails, the content of the email can be evaluated using a language model.

[0017] (4) The present invention is A program for delivering an email with a specified destination, A receiving unit that receives an email sent from a sender, An evaluation unit that evaluates the content of an email using a language model, A hold determination unit that determines whether to hold the delivery of the email to the destination based on the evaluation result evaluated by the evaluation unit, A program characterized by causing a computer to function as a hold unit that holds the email for an email determined to hold the delivery to the destination.

[0018] The present invention relates to an information storage medium storing the above program, an information processing apparatus (for example, a server apparatus) configured as the above respective units, an information processing system configured as the above respective units, and an information processing method for executing the above respective units.

[0019] According to the present invention, since the content of an email is evaluated using a language model and it is determined whether to hold the delivery of the email to the destination based on the evaluation result, the introduction of the language model can be promoted.

[0020] (5) Further, the program, information storage medium, information processing apparatus, information processing system, and information processing method of the present invention are The hold determination unit May be configured to determine whether to hold the delivery of the email to the destination based on the evaluation result evaluated by the evaluation unit and a predetermined condition.

[0021] According to the present invention, since the content of an email is evaluated using a language model and it is determined whether to hold the delivery of the email to the destination based on the evaluation result and a predetermined condition, the introduction of the language model can be promoted.

[0022] (6) Furthermore, the program, information storage medium, information processing device, information processing system, and information processing method of the present invention are The computer may also be configured to function as a delivery determination unit that determines whether or not to deliver an email.

[0023] According to the present invention, since it is determined whether or not to deliver an email, it is possible to appropriately determine whether or not an email can be delivered.

[0024] (7) Furthermore, the program, information storage medium, information processing device, information processing system, and information processing method of the present invention are If it is determined that an email should be delivered, the computer may be further configured to function as a delivery unit that controls the delivery of said email.

[0025] According to the present invention, email delivery can be performed appropriately.

[0026] (8) Furthermore, the program, information storage medium, information processing device, information processing system, and information processing method of the present invention are The computer is further configured to function as a decision unit that determines at least one language model from among multiple language models based on the information in the email, The evaluation unit, The content of the email may be evaluated using the language model determined in the aforementioned determination unit.

[0027] According to the present invention, the optimal language model can be determined based on email information, and the introduction of an appropriate language model according to the situation can be facilitated. The email information includes, for example, the sender of the email envelope, the recipient of the email envelope, header information, body, and at least one of the attachments.

[0028] (9) Furthermore, the program, information storage medium, information processing device, information processing system, and information processing method of the present invention are The computer may also be configured to function as a presentation unit, which presents the evaluation results evaluated by the evaluation unit to the user who sent the information.

[0029] According to the present invention, the sender can recognize the evaluation results. As a result, the present invention can play an auxiliary role in email creation, for example, by giving the sender a reason to proofread the email.

[0030] (10) Furthermore, the program, information storage medium, information processing device, information processing system, and information processing method of the present invention are The aforementioned delivery determination unit, After presenting the evaluation results, the system may decide to deliver the email if it receives delivery instructions from the sender.

[0031] According to the present invention, the sender can review the evaluation results and, if they then issue a delivery instruction, the email can be delivered. This allows the sender to prevent accidental email transmissions based on their own judgment.

[0032] (11) Furthermore, the program, information storage medium, information processing device, information processing system, and information processing method of the present invention are The computer may be further configured to function as a presentation unit that presents the evaluation results evaluated by the evaluation unit to the relevant users associated with the sender.

[0033] According to the present invention, related users associated with the sender (for example, the sender's superior, or team members of the sender) can recognize the evaluation results. As a result, the present invention can play an auxiliary role in email creation, for example, by providing related users with an opportunity to proofread the email.

[0034] (12) Furthermore, the program, information storage medium, information processing device, information processing system, and information processing method of the present invention are The aforementioned delivery determination unit, After presenting the evaluation results, if a delivery instruction is received from the relevant user, the system may decide to deliver the email.

[0035] According to the present invention, when a relevant user reviews the evaluation results and issues a delivery instruction as a result, the email can be delivered. Therefore, the relevant user can prevent the misdelivery of emails based on their own judgment.

[0036] (13) Furthermore, the program, information storage medium, information processing device, information processing system, and information processing method of the present invention are The computer is further configured to function as an evaluation target determination unit, which determines whether or not to include an received email in the evaluation based on the evaluation target conditions, based on the evaluation target conditions. The evaluation unit, If it is determined that the aforementioned email should be evaluated based on a language model, the content of the email may be evaluated using the language model.

[0037] According to the present invention, it is possible to select emails to be evaluated based on evaluation conditions, and to perform evaluation based on the language model for appropriate emails.

[0038] (14) Furthermore, the program, information storage medium, information processing device, information processing system, and information processing method of the present invention are Based on the information in the received email, the language model or network outside the network is determined. The computer functions as a decision-maker that determines the language model within the work. The evaluation unit, The content of the email may be evaluated using the language model determined in the aforementioned determination unit.

[0039] According to the present invention, based on the information of the received email, a language model from outside the network (e.g., the internet) or from inside the network (e.g., an intranet) is determined, so it is possible to appropriately select whether or not to use the language model from outside or inside the network.

[0040] (15) Furthermore, the program, information storage medium, information processing device, information processing system, and information processing method of the present invention are The computer is further configured to function as a decision unit that uses the first language model to determine a second language model to be used for evaluating the content of the email. The evaluation unit, The content of the email may be evaluated using the second language model described above.

[0041] According to the present invention, a second language model to be used to evaluate the content of an email is determined using a first language model, and then the content of the email is evaluated using the second language model, thus further promoting the introduction of language models. [Brief explanation of the drawing]

[0042] [Figure 1] An example of a network diagram of the information processing system of this embodiment. [Figure 2] An example of a functional block diagram of the server device of this embodiment. [Figure 3] An example of a functional block diagram of the terminal device of this embodiment. [Figure 4] A schematic diagram of the processing of the server device 10 in this embodiment. [Figure 5] A diagram showing the correspondence between departments and language models in this embodiment. [Figure 6] An example of the evaluation requirements for this embodiment. [Figure 7] An example of an email received by the server device 10 of this embodiment. [Figure 8] An example of the evaluation results of this embodiment. [Figure 9]An example of a screen displaying the evaluation results of this embodiment. [Figure 10] An example of the hold screen in this embodiment. [Figure 11] An example of a screen displaying the evaluation results of this embodiment. [Figure 12] An example of a flowchart diagram of this embodiment. [Figure 13] A schematic diagram of the processing of the server device 10 in this embodiment. [Figure 14] A schematic diagram of the processing of the server device 10 in this embodiment. [Figure 15] A schematic diagram of the processing of the server device 10 in this embodiment. [Figure 16] An example of a functional block diagram of the server device of this embodiment. [Figure 17] A schematic diagram of the processing of the server device 10 in this embodiment. [Modes for carrying out the invention]

[0043] The following describes this embodiment. Note that the embodiment described below does not unduly limit the scope of the present invention as described in the claims. Furthermore, not all of the configurations described in this embodiment are necessarily essential components of the present invention.

[0044] [1] Network Figure 1 shows an example of a network diagram of an information processing system. The information processing system in this embodiment includes a server device 10 and a terminal device 20.

[0045] In this embodiment, the information processing system consists of a server device 10 and a user terminal device 20. Each is connected via a network (such as the Internet or an intranet). Hereafter, terminal device 20 refers to one of terminal device 20A, terminal device 20B, or terminal device 20C.

[0046] The server device 10 is an information processing device capable of providing a given service to a terminal device 20 that is connected to it via the Internet.

[0047] In particular, the server device 10 of this embodiment provides email-related information to the user's terminal device 20. In this embodiment, the server device 10 may also be referred to as a mail server (MTA).

[0048] The terminal device 20 is a client device, and is an information processing device such as a smartphone, mobile phone, PHS, computer, game device, PDA, or image generation device, and is a device that can connect to the server device 10 via a network such as the Internet (WAN) or LAN.

[0049] The language model server 30 is a device connected via the internet or intranet. It receives input information such as text from the user's terminal device 20, analyzes the received input information to generate appropriate response information (e.g., text response information), and transmits it to the terminal device 20. In other words, the language model server 30 hosts language models on the cloud and responds to user requests.

[0050] In this embodiment, the language model server 30 responds to requests from the terminal device 20 via the server device 10, but it may also respond to requests from the terminal device 20 without going through the server device 10, and the language model server 30 may be the same as the server device 10 or the terminal device 20.

[0051] [2] composition (1) Functional configuration of server device 10 Figure 2 is an example of a functional block diagram of the server device 10 of this embodiment. The server device 10 of this embodiment does not need to include all the parts shown in Figure 2, and may be configured with some parts omitted.

[0052] The memory unit 170 serves as a work area for the processing unit 100 and other components, and can store programs for making the computer function as each component of this embodiment (programs that cause the computer to execute the processing of each component).

[0053] The storage unit 170 stores programs and data that cause the computer to perform processing in this embodiment, and its function can be realized using computer-readable information storage media such as RAM (VRAM), optical discs (CD, DVD), magneto-optical discs (MO), magnetic discs, hard disks, magnetic tapes, or memory (ROM).

[0054] In this embodiment, the storage unit 170 stores a main storage unit 171, a user information storage unit 172, an email storage unit 173, a pending email storage unit 174, a history information storage unit 175, and a rule information storage unit 176. In this embodiment, part or all of the storage unit 170 may be a storage area on the internet (a storage area on the cloud).

[0055] The main memory unit 171 is used as the work area.

[0056] User information is stored in the user information storage unit 172. The user information storage unit 172 stores the user's (for example, an employee's) email address, which is associated with the user identification information (user ID). The system stores login passwords for the user UI (UI stands for User Interface, and the same applies hereafter), information about the department the user belongs to (department ID), and other relevant data.

[0057] The email storage unit 173 stores information about emails sent and received by the user. In this embodiment, the information about emails sent and received by the user may be stored in the storage area of ​​a server that provides an email service (for example, Gmail®) of a cloud-type mail server. The cloud-type mail server provides a cloud-type mail service that allows terminal devices connected to the internet to view email sending and receiving information and to compose emails.

[0058] The pending email storage unit 174 stores pending emails (pending emails) associated with pending email identification information (the email ID of the pending email). In other words, emails that are determined to require approval are stored in the pending email storage unit 174. The email ID is information used to identify an email. The email ID may be, for example, the message ID in the header. Alternatively, the email ID may be unique identification information determined for each email. Furthermore, the same email ID may be assigned to emails with the same content (for example, emails with the same header and body information).

[0059] The history information storage unit 175 stores the status of emails that have been put on hold in association with history identification information (history ID). In other words, the history information storage unit 175 stores the status of emails that have been processed for delivery or delivery cancellation in association with history ID. For example, information on emails that have been delivered or canceled based on approval or rejection instructions is accumulated and stored. The history information storage unit 175 may also store, in association with each email, the content of the instruction for that email (delivery instruction or delivery cancellation instruction), the user information at the time the delivery instruction or delivery cancellation instruction was received, the delivery time or delivery cancellation time, etc.

[0060] The rule information storage unit 176 stores rule information such as whether or not a rule needs to be held in reserve, in association with the rule identification information (rule ID).

[0061] The information storage medium 180 (a medium readable by a computer) stores programs, data, etc., and its function can be realized by optical discs (CDs, DVDs), magneto-optical discs (MOs), magnetic discs, hard disks, magnetic tapes, or memory (ROMs).

[0062] The processing unit 100 performs various processes of this embodiment based on the program (data) stored in the information storage medium 180. That is, the information storage medium 180 stores programs that cause the computer to function as each part of this embodiment (programs that cause the computer to execute the processing of each part).

[0063] The communication unit 196 performs various controls for communication with external devices (for example, terminal devices 20, other server devices, or other network systems), and its functions can be realized by hardware such as various processors or communication ASICs, or by programs.

[0064] The processing unit 100 performs various processes according to this embodiment based on the program (data) stored in the storage unit 170 or the information storage medium 180.

[0065] The processing unit 100 (processor) performs various processes using the main memory unit 171 within the storage unit 170 as the work area. The functions of the processing unit 100 can be realized by hardware such as various processors (CPU, DSP, etc.) or by programs.

[0066] The processing unit 100 includes at least the communication control unit 110, the reception unit 111, the delivery determination unit 112, the delivery unit 113, the decision unit 114, the evaluation unit 115, the presentation unit 116, the hold determination unit 117, the hold unit 118, and the instruction reception unit 119, as well as the evaluation target determination unit 120.

[0067] The communication control unit 110 transmits and receives data with the terminal device 20 and other server devices. For example, the communication control unit 110 receives (accepts) information sent from the user's terminal device 20 or other server devices. The communication control unit 110 also transmits information to the user's terminal device 20 or other server devices.

[0068] The reception unit 111 receives emails sent from the sender.

[0069] The delivery determination unit 112 determines whether or not to deliver the email. For example, the delivery determination unit 112 may use a language model to determine whether or not to deliver the email. For example, the delivery determination unit 112 may use the language model determined in the determination unit 114 to determine whether or not to deliver the email.

[0070] In particular, the delivery determination unit 112 of this embodiment may determine whether or not to deliver the email based on the evaluation results evaluated by the evaluation unit 115.

[0071] Furthermore, the delivery determination unit 112 may determine whether or not to deliver the email based on the user's instructions.

[0072] The delivery determination unit 112 may, with respect to an email, determine to deliver the email after presenting the evaluation result, which has been evaluated by the evaluation unit 115, to the user who sent the email, and then receiving a delivery instruction (send instruction) from the user who sent the email.

[0073] The delivery determination unit 112 may decide to deliver an email after presenting the evaluation result, which has been evaluated by the evaluation unit 115, to the relevant user, and receiving a delivery instruction from the relevant user.

[0074] The delivery unit 113 performs the delivery (sending) of an email if it determines that the email should be delivered.

[0075] The determination unit 114 selects at least one language model from among multiple language models based on information about the email (for example, at least one of the sender of the email envelope, the recipient of the email envelope, header information, body, and attachments). In other words, it may select one or more language models.

[0076] The determination unit 114 may determine a language model from among language models installed outside the network (e.g., the internet), or it may determine a language model from among language models installed inside the network (e.g., an intranet).

[0077] The determination unit 114 may determine an external language model or an internal language model based on the information in the received email.

[0078] Alternatively, the decision unit 114 may use the first language model to determine a second language model to be used for evaluating the content of the email.

[0079] The evaluation unit 115 evaluates the content of the email using a language model. For example, the evaluation unit 115 may evaluate the content of the email using the language model determined by the decision unit 114.

[0080] Furthermore, the evaluation unit 115 may evaluate the content of pending emails using a language model.

[0081] The evaluation unit 115 may use the language model to evaluate the content of emails that have been held and are designated as targets for evaluation (targets for evaluation by the language model). In other words, the evaluation unit 115 controls the system so that it does not use the language model to evaluate the content of held emails that are not designated as targets for evaluation.

[0082] The evaluation unit 115 may, if it determines that an email should be evaluated based on the language model, use the language model to evaluate the content of the email. In other words, the evaluation unit 115 may, if it determines that an email should not be evaluated based on the language model, control the evaluation unit 115 not to use the language model to evaluate the content of the email.

[0083] The evaluation unit 115 may evaluate the content of the email using a language model outside the network, or it may evaluate the content of the email using a language model inside the network.

[0084] The evaluation unit 115 may use a second language model to evaluate the content of the email.

[0085] The presentation unit 116 presents given information to the user. For example, the presentation unit 116 controls the display unit 290 of the user's terminal device 20 to present the information.

[0086] The presentation unit 116 may present the evaluation results evaluated by the evaluation unit 115 to the sender user. For example, the presentation unit 116 may present the evaluation results evaluated by the evaluation unit 115 to the sender user before delivering or canceling the delivery of the email, that is, while the email is on hold.

[0087] Furthermore, the presentation unit 116 may present the evaluation results evaluated by the evaluation unit 115 to the relevant users associated with the sender. For example, the presentation unit 116 may present the evaluation results evaluated by the evaluation unit 115 to the relevant users associated with the sender before delivering or canceling the delivery of the email, that is, while the email is on hold.

[0088] The hold determination unit 117 determines whether or not to hold delivery of an email to its recipient based on predetermined conditions. Holding means to suspend delivery until a given time (temporarily stop delivery). For example, holding means suspending delivery until an instruction to deliver (approve) or cancel (deny) is received from a given user (e.g., the sender, related users associated with the sender, etc.), or suspending delivery until a predetermined period (e.g., 1 hour) has elapsed since the email was received.

[0089] The hold determination unit 117 may determine whether or not to hold delivery of an email to its recipient based on predetermined conditions, regardless of the evaluation results evaluated by the evaluation unit 115.

[0090] The hold determination unit 117 may determine whether or not to hold delivery of the email to the recipient based on the evaluation result evaluated by the evaluation unit 115.

[0091] Furthermore, the holding determination unit 117 considers the evaluation result evaluated by the evaluation unit 115 and predetermined conditions. Based on this, a decision may be made as to whether or not to withhold delivery of the email to the recipient.

[0092] The holding unit 118 holds emails that it has determined should not be delivered to their destination.

[0093] Furthermore, if the presentation unit 116 presents the evaluation results evaluated by the evaluation unit 115 to the sender user before delivering or canceling the email, the holding unit 118 may hold the delivery of the email.

[0094] Furthermore, if the presentation unit 116 presents the evaluation results evaluated by the evaluation unit 115 to the relevant user of the sender before delivering or canceling the email, the holding unit 118 may hold the delivery of the email.

[0095] The instruction receiving unit 119 receives instructions from the user regarding email. For example, the instruction receiving unit 119 receives instructions from the user (e.g., the sender, related users associated with the sender, etc.) to approve or cancel delivery.

[0096] The evaluation target determination unit 120 may determine, based on the evaluation target conditions, whether or not to include the pending email as a target for evaluation based on the language model.

[0097] The evaluation target determination unit 120 determines that the pending email should be evaluated based on the language model if the evaluation target conditions are met.

[0098] Furthermore, the evaluation target determination unit 120 may determine whether or not to include the received email as an evaluation target based on the language model. For example, the evaluation target determination unit 120 may determine whether or not to include the received email as an evaluation target based on the language model based on the evaluation target conditions.

[0099] The evaluation target determination unit 120 determines whether or not to include the received email as a target for evaluation based on the language model, based on the information in the email and the user's instructions.

[0100] For example, the evaluation target determination unit 120 determines that an email should be evaluated based on a language model if the information in the received email satisfies the evaluation target conditions.

[0101] For example, the evaluation target determination unit 120 may determine that the evaluation target conditions are met when given instructions from a user.

[0102] Furthermore, the evaluation target determination unit 120 may determine that the evaluation target conditions are met if at least one of the following conditions is met: the recipient of the held email is a predetermined recipient, and the sender of the held email is a predetermined sender.

[0103] (2) Functional configuration of terminal device 20 Figure 3 is an example of a functional block diagram of the terminal device 20 of this embodiment. The terminal device 20 of this embodiment does not need to include all the parts shown in Figure 3, and may have a configuration in which some parts are omitted.

[0104] The memory unit 270 serves as a work area for the processing unit 200 and other components, and can store programs for making the computer function as each component of this embodiment (programs that cause the computer to execute the processing of each component).

[0105] The storage unit 270 stores programs and data that cause the computer to perform processing in this embodiment. This is a storage medium that stores data, and its function can be realized using computer-readable information storage media such as RAM (VRAM), optical discs (CD, DVD), magneto-optical discs (MO), magnetic discs, hard disks, magnetic tapes, or memory (ROM).

[0106] In this embodiment, the storage unit 270 stores the main storage unit 271. The main storage unit 271 is used as a work area. In this embodiment, part or all of the storage unit 270 may be a storage area on the internet (a storage area on the cloud).

[0107] The information storage medium 280 (a medium readable by a computer) stores programs, data, etc., and its function can be realized by optical discs (CDs, DVDs), magneto-optical discs (MOs), magnetic discs, hard disks, magnetic tapes, or memory (ROMs).

[0108] The processing unit 200 performs various processes of this embodiment based on the program (data) stored in the information storage medium 280. That is, the information storage medium 280 stores programs that cause the computer to function as each part of this embodiment (programs that cause the computer to execute the processing of each part).

[0109] The display unit 290 outputs information according to this embodiment, and its function can be realized by a monitor, CRT, LCD, touch panel display, or HMD (head-mounted display), etc.

[0110] The communication unit 296 performs various controls for communication with external devices (for example, terminal devices 20, other server devices, or other network systems), and its functions can be realized by hardware such as various processors or communication ASICs, or by programs.

[0111] The processing unit 200 performs various processes according to this embodiment based on the program (data) stored in the storage unit 270 or the information storage medium 280.

[0112] The processing unit 200 (processor) performs various processes using the main memory unit 271 within the storage unit 270 as the work area. The functions of the processing unit 200 can be realized by hardware such as various processors (CPU, DSP, etc.) or by programs.

[0113] The processing unit 200 includes at least a communication control unit 210 and a display control unit 211.

[0114] The communication control unit 210 transmits and receives data with the server device 10 and other server devices. For example, the communication control unit 210 receives (accepts) information transmitted from the server device 10 and other users' terminal devices 20.

[0115] For example, the communication control unit 210 transmits given information to the server device 10. The communication control unit 110 also receives information transmitted from the server device 10.

[0116] The display control unit 211 controls the display unit 290 (for example, a monitor) to display given information. For example, the display control unit 211 controls the display of information about emails sent and received by the user, and emails that have been put on hold. In particular, when the display control unit 211 receives evaluation results from the server device 10, it displays those evaluation results. The display control unit 211 also displays a screen for inputting instructions (for example, delivery instructions, delivery cancellation instructions, evaluation request instructions, etc.).

[0117] [3] Overview In this embodiment, a language model is used for controlling email. Furthermore, the server device 10 in this embodiment has an MTA function that receives emails containing the envelope destination (To), envelope sender (From), and message, and delivers the emails.

[0118] Figure 4 shows an overview of the processing performed by the server device 10 in this embodiment. Specifically, when the server device 10 receives an email sent from the sending terminal device 20, it determines whether or not to deliver the email. If the server device 10 determines that it should deliver the email, it delivers the email. On the other hand, if the server device 10 determines that it should not deliver the email, it cancels the delivery of the email. Canceling delivery means keeping the email on hold without delivering it, or deleting (discarding) the email, etc.

[0119] In particular, the server device 10 of this embodiment uses a language model to evaluate the content of the email and, based on the evaluation result, determines whether or not to deliver the email.

[0120] For example, when server device 10 receives an email, it inputs the email and a pre-prepared evaluation request into the language model. Server device 10 then obtains the evaluation result output from the language model in response to the evaluation request. Based on this evaluation result, it decides whether or not to deliver the email.

[0121] For example, if the server device 10 receives an evaluation result from the language model indicating that an email has been sent to a different user X (e.g., UserX), even though the content of the email is intended for user Y, the server device 10 will then cancel the delivery of the email based on this evaluation result. This prevents emails from being sent to the wrong recipient.

[0122] Thus, in this embodiment, a language model is used to determine whether or not to deliver an email, thus encouraging the introduction of language models.

[0123] For the sake of clarity, the following explanation will describe an example of how server device 10 processes an email (email ID=001) sent from the sender, assuming that the sender is User A and the recipient is User X. User A is assumed to belong to the support department.

[0124] [4] Language model decision process [4.1] Description of the language model In this embodiment, multiple language models exist. These language models include large-scale and small-scale language models. The size of a language model is determined by factors such as the number of model parameters and computational complexity.

[0125] Large-scale language models are also called LLMs (Large Language Models). Examples include OpenAI's language model GPT-4 (registered trademark), Google's language model (e.g., BERT), and Microsoft's Turing-NLG.

[0126] Large-scale language models are hosted on cloud servers and exist outside the organization (such as a company or school) (outside the organization's control). In other words, large-scale language models are service models provided by a third party outside the organization.

[0127] Small language models are called SLMs (short for Small Language Model), S It is also called the mall-scale model or Smaller model.

[0128] Small language models are primarily assumed to reside within an organization (under its management). For example, the small language models are stored on a server device 10 managed by the organization in this embodiment.

[0129] The language models in this embodiment (large-scale language model, small-scale language model) are pre-trained language models. In other words, they are trained by adjusting (optimizing) parameters and weights based on a large amount of data during the training process.

[0130] Furthermore, the large-scale language model may be located within the organization and maintained by the server device 10. Conversely, the small-scale language model may be located outside the organization.

[0131] [4.2] Explanation of the language model determination method The server device 10 may determine at least one language model from among multiple language models based on information about the received email (for example, at least one of the sender of the email envelope, the recipient of the email envelope, header information, body, and attachments).

[0132] Figure 5 shows the correspondence between departments (or groups in a broader sense) and language models.

[0133] For example, as shown in Figure 5, if user A, the sender of the email, belongs to a support department (for example, a department that handles customer inquiries, solves problems, and provides product support), then language model A (for example, GPT-4®) corresponding to the support department is determined as the language model for requesting evaluation.

[0134] Furthermore, if the user sending the email belongs to a legal department (for example, a department that handles the creation and review of contract documents, legal risk management, litigation response, etc.), then language model B, which corresponds to the legal department, will be selected as the language model for which the evaluation request will be made. For example, language model B is a language model that has been reinforced through learning regarding law and contracts.

[0135] In this way, the optimal language model is determined according to the department to which the user belongs (or, in a broader sense, the user's information), and that language model is used. This allows for obtaining the optimal evaluation results.

[0136] Although not shown in the diagram, a language model may be defined in advance for each user, and the language model corresponding to the user who sent the request may be determined as the language model used for evaluation requests.

[0137] Furthermore, in this embodiment, the language model for which an evaluation request is made may be determined depending on the destination. For example, if the destination is a predetermined domain (e.g., an internal domain), language model A is determined as the language model for which an evaluation request is made, and if the destination is not a predetermined domain (e.g., an external domain), language model B is determined as the language model for which an evaluation request is made. In this way, different language models can be used for internal and external communications.

[0138] Furthermore, in this embodiment, the language model that requests evaluation of the text may be determined based on the results of the language model that requests evaluation of the text, or depending on whether the text contains a specific string.

[0139] Furthermore, in this embodiment, the language model used to make the evaluation request may be determined depending on whether or not an attached file is present.

[0140] Furthermore, in this embodiment, if an attachment exists, the language model that requests evaluation of the content of the attachment may be determined based on the result of the language model that requests evaluation of the content of the attachment.

[0141] Furthermore, in this embodiment, if an attachment exists, the language model for which the evaluation request is made may be determined according to the content of the attachment (such as containing a specific string).

[0142] In this embodiment, the language model that requests evaluation of the content of the main text and attached files may be determined based on the results of the language model that requests evaluation of the content of the main text and attached files, or based on whether the main text or attached files contain a specific string.

[0143] Furthermore, in this embodiment, the language model for requesting evaluation of the main text and the language model for requesting evaluation of the content of the attached file may be determined to be different models.

[0144] [5] Evaluation process In this embodiment, a language model is used to evaluate the content of an email. Specifically, an evaluation request corresponding to the language model is prepared for each language model. This evaluation request may be prepared for each user who sent the email. Alternatively, an evaluation request may be prepared for each department. The evaluation request may also be referred to as a "request," "prompt," or "input information."

[0145] Figure 6 shows an example of an evaluation request to be input into language model A.

[0146] The evaluation request should be created to include information such as whether or not there is a possibility of misdelivery as an evaluation result. For example, the evaluation request should include content 72 in which the variable `possibly_miss_sending` should return 1 if there is a possibility of misdelivery, and 0 if there is no possibility of misdelivery. Furthermore, content 71 containing detailed information regarding the possibility of misdelivery may also be included in the evaluation request.

[0147] Figure 7 shows an example of the email information for email ID=001 received by the server device 10. In the following description, the email with email ID=001 refers to, for example, the email shown in Figure 7. The server device 10 inputs the email information for email ID=001 and the evaluation request into language model A.

[0148] The information contained in an email includes the recipient and sender of the email envelope, the message header information, and the message body information. The body information includes the email text and attachments.

[0149] In this embodiment, evaluation requirements may be predetermined for each language model. Alternatively, evaluation requirements may be predetermined for each user. Furthermore, evaluation requirements may be predetermined for each group (or department). In other words, in this embodiment, evaluation requirements may be determined based on at least one of the following: the language model to be used, user information, or group information.

[0150] Furthermore, the evaluation request may include a requirement to provide a response in English in order to facilitate control over the information regarding the evaluation results.

[0151] [6] Evaluation results When the server device 10 receives the email information for email ID=001 and the evaluation request as input to language model A, it obtains the evaluation result output from language model A.

[0152] Figure 8 shows an example of the evaluation result. As shown in Figure 8, for example, since possibly_miss_sending=1, it is determined that there is a possibility of misdelivery.

[0153] [7] Explanation of whether or not to send the email based on the evaluation results. In this embodiment, a language model is used to evaluate the content of the email, and based on the evaluation results, a decision is made as to whether or not to deliver the email.

[0154] In other words, in this embodiment, the decision of whether or not to deliver an email is made based on the evaluation results obtained from language model A.

[0155] In this embodiment, if the information indicates a possible misdelivery (for example, possibly_miss_sending=1), delivery is canceled.

[0156] On the other hand, in this embodiment, if the information indicates a possibility of misdelivery (for example, possibly_miss_sending=0), the message is delivered.

[0157] For example, as shown in Figure 8, if email information for email ID=001 and an evaluation request are entered into language model A, there is a possibility of misdelivery because possibly_miss_sending=1, so delivery is canceled.

[0158] [8] Presentation of evaluation results In this embodiment, before or before delivery of an email (for example, an email with email ID=001 sent by user A), that is, while the email is on hold, the evaluation results obtained by evaluation using a language model are presented to the sending user (for example, user A).

[0159] Furthermore, in this embodiment, the evaluation results may be presented to related users associated with the sender. These related users could be, for example, user B in the same department as user A (the sender), or user C, who is user A's superior.

[0160] Figure 9 shows an example of a screen displaying the evaluation results on the terminal device 20 of user A, the sender.

[0161] In this embodiment, as shown in Figure 9, if there is a possibility that the email with email ID=001 was sent to the wrong recipient, a message 61 based on the evaluation result, "This email may have been sent to the wrong recipient. The content of the email may not be related to the recipient," is displayed. The sender, user A, can view the details of the evaluation result by clicking the "View Details" link 62. In other words, when the server device 10 receives access to link 62, it displays a screen with detailed information to user A's terminal device 20.

[0162] For example, when server device 10 receives access to link 62, it displays detailed information on its screen. Specifically, on user A's terminal device 20, the detailed information displays, for example, the recipient's name (Jiro Tanaka), the recipient's company name (IF Company), and the email address (Xuser@xxx.ifpat.gr.jp) for email ID=001. Then, on user A's terminal device 20, the detailed information displays the recipient's information in the email body. For example, on user A's terminal device 20, the detailed information displays the recipient's name (Ichiro Suzuki) and the recipient's company name (XX Company). Furthermore, on user A's terminal device 20, the detailed information displays opinion information from language model A. For example, it displays opinion information such as, "There is an error in the recipient. The name of Jiro Tanaka of IF Company is not included in the email body." In this way, user A can easily recognize what the error is. In other words, user A can easily recognize that they have sent the email to the wrong recipient.

[0163] In this embodiment, the evaluation results and detailed information shown in Figure 9 may also be presented to User B, who is in the same department as User A, and to User C, who is User A's superior.

[0164] [9] Acceptance of instructions after the evaluation results have been presented. In this embodiment, if, before or before delivery of an email (for example, an email ID=001 sent by user A), that is, while the email is on hold, an evaluation result obtained using a language model is presented to the sender user A, and a delivery instruction (approval instruction) is subsequently received from user A, then it may be determined to deliver the email. On the other hand, if, after presenting the evaluation result to user A, a delivery cancellation (denial instruction) is received from user A, then it is determined not to deliver the email (delivery cancellation). In this way, the necessity of delivering the email can be controlled while respecting user A's wishes.

[0165] For example, as shown in Figure 9, if user A sends a cancellation link 58 (a link labeled "Discard") for email with email ID=001, the system determines that the email will not be delivered (denied). On the other hand, if user A sends a delivery link 59 (a link labeled "Send") for email with email ID=001, the system determines that the email will be delivered (approved).

[0166] Furthermore, in this embodiment, if the evaluation results obtained using a language model are presented to a related user (for example, user B in the same department as user A) before the email is delivered or before delivery is canceled, that is, while the email is on hold, and a delivery instruction is subsequently received from the related user (for example, user B), then it may be determined to deliver the email. On the other hand, if a delivery cancellation (denial instruction) is received from the related user (for example, user B), then it is determined not to deliver the email (delivery canceled). In this way, the necessity of delivering the email can be controlled quickly.

[0167] In this embodiment, if the evaluation results obtained using a language model are presented to multiple users before the delivery or cancellation of the email, that is, while the email is on hold (for example, to the sender user A, related user B, and related user C), the email delivery control (delivery or cancellation) may be performed based on the instruction received first (delivery instruction or delivery cancellation instruction).

[0168]

[10] Process to evaluate emails that are put on hold. In this embodiment, if the server device 10 evaluates all emails received using the language model and determines whether they can be delivered, the processing load may become high. Therefore, in this embodiment, the evaluation using the language model and determination of whether they can be delivered may be limited to only a portion of the emails received by the server device 10.

[0169] In this embodiment, a determination is made based on predetermined conditions whether or not to withhold delivery of an email to its recipient. If it is determined that an email should be withheld from delivery to its recipient, the email is then held.

[0170] The specified conditions are determined by the information of the received email (at least one of the recipient and sender of the email envelope, message header information, and message body information), and the evaluation results from the language model are not used.

[0171] The specified condition may be that the email recipient is external. In other words, if the recipient's (To) email address or domain is outside the network of the organization to which the sender (From) belongs (outside the domain of the organization to which they belong), the recipient is determined to be external. In other words, when sending an email to an email address of a user outside the organization, it is determined that the recipient is external. For example, if the sender (From) of email ID=001 is Auser@xxx.hennge.com and the recipient (To) is Xuser@xxx.ifpat.gr.jp, the recipient's domain is outside the network range of the organization to which the sender belongs, so it is determined that the recipient is external.

[0172] The specified conditions may include the condition that the recipient is a specific email address.

[0173] Furthermore, the specified conditions may include the condition that the sender is a specific email address.

[0174] Furthermore, the specified conditions may include the condition that at least one of the email body or attachment contains a specific string (for example, the strings "confidential" or "private company").

[0175] Furthermore, the specified conditions are not limited to a single condition, but may consist of multiple conditions.

[0176] In this embodiment, the server device 10 may evaluate the content of held emails using a language model. That is, the server device 10 in this embodiment uses a language model to determine whether or not to deliver held emails, and delivers the email if it is determined to deliver it. On the other hand, if it is determined not to deliver the email, the server device 10 cancels the delivery of the email. The server device 10 delivers emails that were not held immediately. In this way, the processing load can be reduced because the server evaluates held emails using a language model based on predetermined conditions and determines whether or not they can be delivered.

[0177] Furthermore, in this embodiment, it may be determined whether or not to include a pending email as a subject for evaluation based on the language model, based on the evaluation target conditions. For example, if an evaluation request is received from a given user regarding a pending email, it may be determined that the evaluation target conditions have been met, and the pending email may be included as a subject for evaluation based on the language model.

[0178] Furthermore, among the pending emails, the content of the emails that are subject to evaluation may be evaluated using a language model.

[0179] Furthermore, the evaluation criteria are not limited to accepting user evaluation requests; various other conditions may be included.

[0180] The evaluation criteria may be based on the information provided in the received email.

[0181] For example, the evaluation criteria could be that the recipient of the email is an external party. In other words, if the recipient of a pending email is an external party, it could be determined that the evaluation criteria are met.

[0182] Alternatively, the evaluation criteria could be that a predetermined string (for example, the string "No") exists in the header information (for example, Subject). In other words, if a predetermined string (for example, the string "No") exists in the header information (for example, Subject) of a pending email, it may be determined that the evaluation criteria are met.

[0183] Furthermore, the evaluation criteria are that the combination of sender (From) and recipient (To) is specified. The condition that it is a combination is also acceptable. For example, if the recipient (To) is Buser@xxx.hennge.com and the sender (From) is Yuser@xxx.ifpat.gr.jp, then it may be determined that the evaluation criteria are met.

[0184] In this embodiment, the control of pending emails that are subject to evaluation based on a language model will be specifically described. In this embodiment, when a user (for example, user A who is the sender or related users B, C, etc.) requests an evaluation using a language model for a pending email (for example, an email with pending email ID=001), the content of the email may be evaluated using the language model, and a decision may be made whether or not to deliver the email based on the evaluation result.

[0185] Figure 10 shows an example of a hold screen displayed on User A's terminal device 20 for emails that have been held based on predetermined conditions. For example, if User A selects a held email dated July 9, 2024 (for example, an email with email ID=001) from among the held emails that User A is the sender of, based on predetermined conditions, the content 53 of the held email will be displayed.

[0186] For example, as shown in Figure 10, when user A clicks on evaluation request link 60 (a link labeled "Ask AI"), the language model can be used to evaluate the email.

[0187] In other words, when the server device 10 receives access to the evaluation request link 60, it inputs the email information (for example, the email with email ID=001) and the evaluation request into the language model. It then retrieves the evaluation result output from the language model. The server device 10 then presents a message 61 based on the evaluation result to user A's terminal device 20, as shown in Figure 11. The explanation of how the evaluation result is presented is as described above.

[0188] On the other hand, if the server device 10 does not receive an evaluation request instruction (access to link 60) from user A for the pending email with email ID=001, it controls the system so as not to perform evaluation processing using the language model.

[0189] As shown in Figure 10, if user A submits a link 58 to cancel delivery (a link labeled "Discard") for email with email ID=001, the system will determine that the email will not be delivered (denied). Conversely, if user A submits a delivery link 59 (a link labeled "Send") for email with email ID=001, the system will determine that the email will be delivered (approved). In other words, user A can choose not to provide email information to the language model.

[0190] This approach allows for the use of language models with minimal overhead. This is particularly useful when there are language models with limited usage limits.

[0191]

[11] Flowchart The process flow of the server device 10 in this embodiment, which receives emails and controls email delivery, will be explained using Figure 12.

[0192] First, the email information and evaluation request are input into the language model (Step S1). Then, the evaluation result output from the language model is obtained (Step S2).

[0193] Then, it is determined whether or not to deliver the email (step S3). If it is determined that the email should be delivered (Y in step S3), the email is delivered (step S4). On the other hand, if it is determined that the email should not be delivered (N in step S3), the email delivery is canceled (step S5). The process then ends.

[0194]

[12] Supplementary explanation of the example of holding an email using a language model In this embodiment, the process of holding an email was described on the premise that the evaluation results would be presented to a given user (for example, user A, the sender, or a related user). However, the email may be held using a language model regardless of whether the evaluation results are presented.

[0195] In other words, in this embodiment, as shown in Figure 13, email information and evaluation requests (prompts) may be input to a language model, evaluation results may be obtained from the language model, and based on the evaluation results, a decision may be made as to whether or not to withhold delivery of the email to the recipient.

[0196] Then, for emails that are determined to be undeliverable to their recipient, the emails are held in abeyance.

[0197] In this embodiment, a decision is made regarding whether or not to deliver a pending email. If it is determined that the email should be delivered, control is performed to deliver the email.

[0198] For example, in this embodiment, a held email is delivered if it is approved by at least one of the sending user and related users associated with the sender. On the other hand, if a held email is rejected by at least one of the sending user and related users associated with the sender, delivery of the email is canceled.

[0199] For example, as described above, as shown in Figure 9, the evaluation results may be presented to User A, and a decision may be made on whether or not to deliver the email based on a delivery instruction (approval instruction) or delivery cancellation instruction (denial instruction) from a given user (for example, at least one of the sender user and related users associated with the sender).

[0200] For example, the evaluation request (prompt) could include information such as "whether or not the email body contains a predetermined number of attachments (e.g., whether or not there are 10 or more)." For instance, the evaluation request could be designed to return "1" to the 10_attached_files variable if there are 10 or more attachments, and "0" if there are fewer than 10 attachments. Note that various types of evaluation requests are possible, such as information that returns "1" if the recipient is a specific user, and "0" otherwise.

[0201] The server device 10 inputs the email information for email ID=001 and the evaluation request into a given language model. For example, it selects at least one language model from among several language models. Then, if the evaluation result output from the language model is, for example, 10_attached_files=1, the email for email ID=001 is put on hold.

[0202] In this way, the process of determining whether an email should be held can be replaced with an evaluation request (prompt).

[0203] Furthermore, "hold" can be a permanent hold, or it can be a temporary hold for a predetermined period (for example, 1 hour) after which the item is delivered.

[0204] Furthermore, in this embodiment, as shown in Figure 13, email information and evaluation request (prompt The input (Pt) is entered into a language model, and an evaluation result is obtained from the language model. At that time, a decision may be made on whether or not to withhold delivery of the email to the recipient based on the evaluation result and predetermined conditions.

[0205] For example, the server device 10 inputs the email information for email ID=001 and the evaluation request into a given language model. If the evaluation result output from the language model is, for example, 10_attached_files=1 and satisfies predetermined conditions (for example, if the destination domain belongs to a specific domain), the email for email ID=001 is put on hold.

[0206] In this way, part of the process of determining whether an email should be held can be replaced by an evaluation request (prompt).

[0207]

[13] Determination of whether or not to include it in the evaluation based on language models In this embodiment, a language model is introduced, but in some cases, using a language model may not be appropriate for certain types of email. For example, many users do not wish to input email information into a language model.

[0208] Therefore, in this embodiment, as shown in Figure 14, it is possible to determine whether or not to include the email in the evaluation based on the evaluation target conditions, and if it is determined that the email should be included in the evaluation based on the language model, the content of the email may be evaluated using the language model. Then, based on the evaluation result, it is determined whether or not to deliver the email. Alternatively, based on the evaluation result, it may be determined whether or not to withhold delivery of the email to the recipient.

[0209] On the other hand, in this embodiment, if it is determined that an email is not subject to evaluation based on the language model, that is, if an email is not subject to evaluation, the language model is controlled not to evaluate the content of that email. For example, for an email not subject to evaluation, a decision is made whether or not to deliver it based on predetermined conditions without using the language model. Alternatively, for an email not subject to evaluation, a decision may be made whether or not to hold it based on predetermined conditions without using the language model.

[0210] The evaluation criteria are as described above. For example, the evaluation criteria may be that the system accepts evaluation requests from users, or the evaluation criteria may be based on the information in the received email. For example, the evaluation criteria may be that the recipient of the email is an external party.

[0211]

[14] Description of the external and internal language models In this embodiment, as shown in Figure 15, a language model outside the network or an internal language model within the network may be determined based on the information of the received email. The content of the email may then be evaluated using the determined language model.

[0212] In other words, in this embodiment, a determination is made based on the information of the received email to use either an external language model or an internal language model. If a determination is made based on the information of the received email whether or not to evaluate the email using a language model, then the external or internal language model is determined if the email is determined to be an evaluation target using a language model.

[0213] For example, the recipient of the received email is an email address that has a designated domain of Company X. If this is the case, the network's internal language model (e.g., a small language model) is determined, and the content of the email is evaluated using the determined internal language model (e.g., a small language model). If there are multiple internal language models (e.g., language model A, language model B, etc.), at least one language model is determined from among the multiple internal language models.

[0214] On the other hand, if the recipient of an received email is not an email address belonging to a specified domain of Company X, an external language model (e.g., a large-scale language model) is determined, and the content of the email is evaluated using the determined external language model (e.g., a large-scale language model). If there are multiple external language models (e.g., language model K, language model L, etc.), at least one language model is determined from among the multiple external language models.

[0215] In this way, if some users (for example, users of Company X) do not wish to use an external language model, the internal language model can be used to evaluate the content of emails, further reducing the risk of data leakage and enhancing security.

[0216] An external language model is a language model that is accessible via the internet and is operated on devices outside the network managed by an organization (including devices in the cloud). For example, an external language model is a large-scale language model such as GPT-4®.

[0217] The language model within the network is a language model managed by a device located on the same network as the server device 10 (for example, an intranet), and is under the management of the server device 10. In other words, in this embodiment, the language model may be managed by a device located on the same network as the server device 10 managed by the organization.

[0218] Figure 16 shows an example of the functional configuration for managing language models in the server device 10. Points that are common to the explanation above are omitted.

[0219] For example, as shown in Figure 16, the language model may be stored in the language model storage unit 178 of the storage unit 170 of the server device 10. Alternatively, the language model managed by the server device 10 may be stored in the storage area of ​​the network connected to the server device 10. Furthermore, training data for training the language model (for example, a dataset of email information) may be stored in the training data storage unit 179 of the storage unit 170 of the server device 10.

[0220] When the server device 10 manages a language model, that language model is, for example, a small-scale language model. A small-scale language model has a more compact structure than a large-scale language model. Also, a small-scale language model can keep the data within the network of an organization (for example, an organization such as a company or school). For this reason, a small-scale language model can be said to be a suitable model for users who want to enhance security.

[0221]

[15] Language model determination methods In this embodiment, as shown in Figure 17, a second language model to be used for evaluating the content of an email may be determined using a first language model. Then, the content of the email may be evaluated using the second language model. Based on the evaluation result, a delivery decision or a hold decision may be made. Note that the first language model and the second language model may end up being the same language model. Furthermore, the first language model may be used to determine whether or not to include the content of the email in the evaluation based on the second language model used for evaluation.

[0222] For example, in this embodiment, if an attachment exists, a first language model (e.g., a small language model) may be used to identify the content of the attachment, such as a contract, invoice, promotional material, or questionnaire, and a second language model that is presumed to be suitable for evaluating the content of the attachment may be determined.

[0223] Furthermore, in this embodiment, a first language model (for example, a small-scale language model) may be used to identify from the content of the text what kind of content it is, such as a contract, invoice, promotional material, or questionnaire, and a language model that is presumed to be suitable for evaluating the content of the text may be determined.

[0224] Furthermore, in this embodiment, a first language model (e.g., a small-scale language model) may be used to identify the content of at least one of the main text and attached files, such as a contract, invoice, promotional material, or questionnaire, and a language model that is presumed to be suitable for evaluating the content of the main text and attached files may be determined.

[0225] Furthermore, in this embodiment, the language model for identifying the content of the main text (e.g., a small language model) and the language model for identifying the content of the attached files (e.g., a small language model) may be different language models.

[0226] There may be multiple first language models, such as language model K and language model L. To enhance security, in this embodiment, a small-scale language model managed by the server device 10 is used as the first language model, but a large-scale language model may also be used as the first language model.

[0227] When the server device 10 receives an email from a sender, it inputs information about the email and an evaluation request (prompt) regarding the selection of a second language model into the first language model K. The evaluation request is, for example, "Does this email contain contract-related information?" For example, the evaluation request (prompt) may include information on whether or not contract-related information is included in at least one of the email body and attachments. For example, the evaluation request may be created to include a response of "1" to the contract variable if contract-related information is included in at least one of the email body and attachments, and "0" if contract-related information is not included.

[0228] For example, the server device 10 inputs the email information for email ID=001 and an evaluation request regarding the selection of a second language model into the first given language model K.

[0229] Then, if the evaluation result output from the first language model is, for example, contract=1, the second language model for evaluating the content of the email with email ID=001 is determined to be language model B.

[0230] On the other hand, if the evaluation result output from the first language model is, for example, contract=0, then language model A is determined to be the language model for evaluating the content of the email with email ID=001.

[0231] Furthermore, if the user sending the email belongs to the legal department, language model B can be used only for necessary emails containing contractual information, allowing for the selection of the appropriate language model.

[0232] The evaluation request may also be information that responds to the email with the appropriate language model. For example, if language model A is appropriate, set the variable language_model to "1". The evaluation request is designed to respond by returning a numerical value corresponding to the language model. For example, if language model B is appropriate, return "2" to the language_model variable; if language model C is appropriate, return "3" to the language_model variable; and if no language model is used, return "0" to the language_model variable.

[0233] For example, the server device 10 inputs the email information for email ID=001 and an evaluation request for selecting a second language model into the first given language model K.

[0234] Then, if the evaluation result output from the first language model is, for example, language_model=1, then the second language model to evaluate the content of the email with email ID=001 is determined to be language model A. In this way, the appropriate language model for the judgment can be selected according to the information in the email.

[0235]

[16] Determining multiple language models As described above, the server device 10 determines at least one language model from among multiple language models based on the information of the received email (for example, at least one of the sender of the email envelope, the recipient of the email envelope, header information, body, and attachments).

[0236] If multiple language models are determined, the content of the email is evaluated using each language model, and the evaluation results are obtained.

[0237] If the evaluations of each language model are the same, the decision of whether or not to deliver (or hold) the email will be based on the same evaluation result.

[0238] There may be differences in the evaluation of each language model. In such cases, as a first method, the difference in the evaluation results of multiple language models may be presented to the sender and / or related users associated with the sender, and this may be recorded in the log. Furthermore, if there are differences in the evaluation results of multiple language models in the first method, only the evaluation result of a specific language model among the multiple language models may be adopted.

[0239] Then, based on the evaluation results, a decision is made as to whether or not to deliver (or hold) the email.

[0240] In this way, for example, users can check the impact of language model updates before performing them.

[0241] Furthermore, in this embodiment, if there are differences in the evaluation results of each of the multiple language models, instead of the first method, the language model to be adopted may be determined by a majority vote process using an algorithm as the second method.

[0242] In other words, among the evaluation results of multiple language models, the language model with the most identical evaluations is selected.

[0243] Then, based on the evaluation results, a decision is made as to whether or not to deliver (or hold) the email.

[0244] Furthermore, in the second method, the evaluation results of each of the multiple language models, the adopted language model, etc., may be presented to the sender and / or related users associated with the sender, and recorded in a log.

[0245] In this way, users can determine which language models are important for obtaining accurate evaluation results.

[0246]

[17] Application examples In this embodiment, a server device 10 that delivers emails to a specified recipient receives an email sent from the sender and evaluates the content of the email using a language model. In this embodiment, a given process may be performed based on the evaluated result.

[0247] In this embodiment, it is not necessary to use the evaluated results in the delivery decision process that determines whether or not to deliver the email.

[0248] Furthermore, in this embodiment, the evaluated results do not necessarily have to be used in the hold determination process to determine whether or not to hold delivery of the email to the recipient.

[0249]

[18] Others The present invention is not limited to the embodiments described above, and various modifications are possible. For example, terms cited in the specification or drawings as broad or synonymous terms can be replaced with broad or synonymous terms in other descriptions in the specification or drawings.

[0250] The present invention includes configurations that are substantially identical to those described in the embodiments (for example, configurations with the same function, method, and result, or configurations with the same purpose and effect). Furthermore, the present invention includes configurations in which non-essential parts of the configurations described in the embodiments are replaced. Furthermore, the present invention includes configurations that produce the same effects or achieve the same purpose as those described in the embodiments. Furthermore, the present invention includes configurations that add known technology to the configurations described in the embodiments.

[0251] As described above, embodiments of the present invention have been explained in detail, but it will be readily apparent to those skilled in the art that many modifications are possible without substantially departing from the novel aspects and effects of the present invention. Therefore, all such modifications are included within the scope of the present invention. [Explanation of Symbols]

[0252] 10 server devices, 20 terminal devices, 100 Processing unit, 110 Communication control unit, 111 Reception unit, 112 Delivery determination unit, 113 Delivery Department, 114 Decision Department, 115 Evaluation Department, 116 Presentation Department, 117 Hold determination unit, 118 Hold unit, 119 Instruction reception unit, 120 Evaluation target determination unit, 170 Storage unit, 171 Main memory unit, 172 User information storage unit, 173 Email storage unit, 174 Hold email storage unit, 175 History information storage unit, 176 Rule information storage unit, 178 Language model memory unit, 179 Training data memory unit, 180 Information storage media, 196 Communications Department, 200 Processing unit, 210 Communication control unit, 211 Display control unit, 270 Memory Department, 271 Main Memory Department, 280 Information Memory Media 290 Display Department, 296 Communication Department

Claims

1. A program that delivers emails to a specified recipient, The receiving department accepts emails sent from the sender, An evaluation unit that uses a language model to evaluate the content of an email, A delivery determination unit determines whether or not to deliver an email based on the evaluation results evaluated by the aforementioned evaluation unit, A program characterized by causing a computer to function as a delivery unit that controls the delivery of an email when it is determined that the email should be delivered.

2. In claim 1, A hold determination unit that determines whether or not to withhold delivery of an email to the recipient based on predetermined conditions, For emails that have been determined to be held for delivery to their destination, a computer is configured to function as a holding unit for those emails. The evaluation unit, A program that uses a language model to evaluate the content of pending emails.

3. In claim 1 or 2, A hold determination unit that determines whether or not to withhold delivery of an email to the recipient based on predetermined conditions, For emails that have been determined to be held for delivery to the recipient, the holding section that holds the email, The computer is further configured as an evaluation target determination unit that determines whether or not to include a pending email in the evaluation based on the evaluation target conditions, based on the evaluation target conditions. The evaluation unit, A program characterized by using a language model to evaluate the content of emails that have been held and are selected as targets for evaluation.

4. A program that delivers emails to a specified recipient, The receiving department accepts emails sent from the sender, An evaluation unit that uses a language model to evaluate the content of an email, Based on the evaluation results evaluated by the evaluation unit, a hold determination unit determines whether or not to withhold delivery of the email to the recipient, A program characterized by causing a computer to function as a holding unit for emails that have been determined to be held for delivery to their destination.

5. In claim 4, The aforementioned retention determination unit, A program characterized by determining whether or not to withhold delivery of an email to its recipient based on the evaluation results evaluated by the evaluation unit and predetermined conditions.

6. In claim 4, A program characterized by further enabling the computer to function as a delivery determination unit that determines whether or not to deliver an email.

7. In claim 6, A program characterized by further enabling the computer to function as a delivery unit that controls the delivery of an email when it is determined that the email should be delivered.

8. In claim 1 or 4, The computer is further configured to function as a decision unit that determines at least one language model from among multiple language models based on the information in the email, The evaluation unit, A program characterized by evaluating the content of an email using the language model determined in the aforementioned determination unit.

9. In claim 1 or 4, A program characterized by further enabling the computer to function as a presentation unit that presents the evaluation results evaluated in the evaluation unit to the user who sent the information.

10. In claim 1 or 6, The aforementioned delivery determination unit, A program characterized by determining to deliver the email after presenting the evaluation results and receiving delivery instructions from the sender user.

11. In claim 1 or 4, A program characterized by further enabling the computer to function as a presentation unit that presents the evaluation results evaluated in the evaluation unit to relevant users related to the sender.

12. In claim 1 or 6, The aforementioned delivery determination unit, A program characterized by determining to deliver the email if, after presenting the evaluation results, it receives a delivery instruction from a related user associated with the sender.

13. In claim 1 or 4, The computer is further configured to function as an evaluation target determination unit, which determines whether or not to include an received email in the evaluation based on the evaluation target conditions, based on the evaluation target conditions. The evaluation unit, A program characterized by evaluating the content of an email using a language model when it is determined that the email should be evaluated based on a language model.

14. In claim 1 or 4, Based on the information received via email, the computer functions as a decision unit to determine either the external or internal language model of the network. The evaluation unit, A program characterized by evaluating the content of an email using the language model determined in the aforementioned determination unit.

15. In claim 1 or 4, The computer is further configured to function as a decision unit that uses the first language model to determine a second language model to be used for evaluating the content of the email. The evaluation unit, A program characterized by evaluating the content of an email using the second language model described above.

16. An information processing device that delivers email to a specified recipient, The receiving department accepts emails sent from the sender, An evaluation unit that uses a language model to evaluate the content of an email, A delivery determination unit determines whether or not to deliver an email based on the evaluation results evaluated by the aforementioned evaluation unit, An information processing apparatus characterized by including a delivery unit that controls the delivery of an email when it is determined that an email should be delivered.

17. An information processing device that delivers email to a specified recipient, The receiving department accepts emails sent from the sender, An evaluation unit that uses a language model to evaluate the content of an email, Based on the evaluation results evaluated by the evaluation unit, a hold determination unit determines whether or not to withhold delivery of the email to the recipient, An information processing apparatus characterized by including a holding unit for holding emails that have been determined to be held for delivery to their destination.

18. An information processing system that delivers emails to a specified recipient, The receiving department accepts emails sent from the sender, An evaluation unit that uses a language model to evaluate the content of an email, A delivery determination unit determines whether or not to deliver an email based on the evaluation results evaluated by the aforementioned evaluation unit, An information processing system characterized by including a delivery unit that controls the delivery of an email when it is determined that an email should be delivered.

19. An information processing system that delivers emails to a specified recipient, The receiving department accepts emails sent from the sender, An evaluation unit that uses a language model to evaluate the content of an email, Based on the evaluation results evaluated by the evaluation unit, a hold determination unit determines whether or not to withhold delivery of the email to the recipient, An information processing system characterized by including a holding unit that holds emails that have been determined to be held for delivery to their destination.

20. An information processing method for delivering emails to a specified recipient, Steps to accept emails sent from the sender, The steps include evaluating the content of an email using a language model, Based on the evaluation results obtained in the aforementioned evaluation step, a step is made to determine whether or not to deliver the email, An information processing method characterized by including the step of performing control over the delivery of an email when it is determined that an email should be delivered.

21. An information processing method for delivering emails to a specified recipient, Steps to accept emails sent from the sender, The steps include evaluating the content of an email using a language model, Based on the evaluation results obtained in the aforementioned evaluation step, a step is made to determine whether or not to withhold delivery of the email to the recipient. An information processing method characterized by including the step of holding an email that has been determined to be undeliverable to its destination.

Images