Evaluation device, evaluation method, evaluation program, and information processing device.

Abstract

To facilitate the selection of processing methods for increased efficiency and sophistication. [Solution] The evaluation device includes a comprehensive evaluation unit. The comprehensive evaluation unit is configured to calculate a comprehensive evaluation index that integrates multiple evaluation indices related to having one information processing unit execute one information processing, or having multiple information processing units execute multiple information processing, for each of the multiple processing method patterns when the processing method in one information processing unit is swapped, or for each of the multiple processing method patterns when the processing method in each of the multiple information processing units is swapped.

Description

[Technical Field]

[0001] This disclosure relates to an evaluation device. [Background technology]

[0002] Patent Document 1 describes a parameter optimization device that searches for the optimal value for each of several parameters related to the performance of a predetermined function from a search range set for each parameter. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2019-8342 [Overview of the project] [Problems that the invention aims to solve]

[0004] In recent years, there has been an increasing trend to replace tasks previously performed by humans with systems (e.g., generative AI, AI agents) in order to improve the efficiency and sophistication of business operations. However, with parameter optimization methods such as the parameter optimization device disclosed in Patent Document 1, it is not easy to determine which system is optimal. This is because there are a wide range of evaluation metrics for evaluating the system.

[0005] This disclosure aims to facilitate the selection of processing methods for efficiency and sophistication. [Means for solving the problem]

[0006] One aspect of this disclosure is an evaluation device equipped with a comprehensive evaluation unit. In an information processing apparatus having one information processing execution unit that executes one information processing or a plurality of information processing execution units that execute each of a plurality of information processings, for each of a plurality of processing method patterns when the processing method in one information processing executed by one information processing execution unit is replaced, or for each of a plurality of processing method patterns when the processing methods in each of a plurality of information processings executed by each of a plurality of information processing execution units are replaced, it is configured to calculate a comprehensive evaluation index that integrates a plurality of evaluation indexes related to causing one information processing execution unit to execute one information processing or causing a plurality of information processing execution units to execute a plurality of information processings.

[0007] The evaluation apparatus of the present disclosure configured as described above calculates a single comprehensive evaluation index that integrates a plurality of evaluation indexes for each of a plurality of processing method patterns when the processing method is replaced. Therefore, the evaluation apparatus of the present disclosure can select a processing method in one or a plurality of information processing execution units using a single evaluation index calculated for each of a plurality of processing method patterns, and can facilitate the selection of a processing method for efficiency improvement and sophistication.

[0008] Another aspect of the present disclosure is an evaluation method executed by an evaluation apparatus. In the evaluation method of the present disclosure, in an information processing apparatus having one information processing execution unit that executes one information processing or a plurality of information processing execution units that execute each of a plurality of information processings, for each of a plurality of processing method patterns when the processing method in one information processing executed by one information processing execution unit is replaced, or for each of a plurality of processing method patterns when the processing methods in each of a plurality of information processings executed by each of a plurality of information processing execution units are replaced, it calculates a comprehensive evaluation index that integrates a plurality of evaluation indexes related to causing one information processing execution unit to execute one information processing or causing a plurality of information processing execution units to execute a plurality of information processings.

[0009] The evaluation method disclosed herein is a method performed using the evaluation apparatus disclosed herein, and by performing this method, the same effects as those of the evaluation apparatus disclosed herein can be obtained. Another aspect of this disclosure is an evaluation program for causing a computer to function as an overall evaluation unit.

[0010] A computer controlled by the evaluation program of this disclosure can constitute part of the evaluation apparatus of this disclosure and can achieve the same effects as the evaluation apparatus of this disclosure. A further aspect of this disclosure is an information processing device having one information processing execution unit that performs one information processing, or multiple information processing execution units that perform each of a plurality of information processing, which is optimized by an evaluation method that calculates a comprehensive evaluation index that integrates multiple evaluation indexes related to having one information processing execution unit perform one information processing, or having multiple information processing execution units perform multiple information processing, for each of a plurality of processing method patterns when the processing method in one information processing execution unit is swapped, or for each of a plurality of processing method patterns when the processing method in each of a plurality of information processing execution units is swapped.

[0011] The information processing device disclosed herein is a device optimized by the evaluation method disclosed herein, and can achieve the same effects as the evaluation method disclosed herein. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram showing the configuration of the evaluation system. [Figure 2] This figure shows a specific example of the business process in the first embodiment. [Figure 3] This is a flowchart showing the business process of the first embodiment. [Figure 4] This diagram shows multiple generative AIs under consideration. [Figure 5] This is a diagram illustrating multiple evaluation metrics. [Figure 6] This figure shows specific examples of multiple evaluation indicators in the first embodiment. [Figure 7] This is a flowchart showing the overall evaluation process of the first embodiment. [Figure 8] This figure shows a specific example of the business process in the second embodiment. [Figure 9] This is a flowchart showing the business process of the second embodiment. [Figure 10] This figure shows the multiple algorithms under consideration. [Figure 11] This figure shows specific examples of multiple evaluation indicators in the second embodiment. [Figure 12] This is a flowchart showing the overall evaluation process of the second embodiment. [Figure 13] This diagram shows the formulas for calculating multiple evaluation metrics. [Figure 14] This figure shows a specific example of the business process for the first modified example. [Figure 15] This figure shows a specific example of the business process for the second modified example. [Figure 16] This figure shows a specific example of the business process for the third modified example. [Modes for carrying out the invention]

[0013] [First Embodiment] A first embodiment of this disclosure is described below with reference to the drawings. [1-1. Structure] As shown in Figure 1, the evaluation system 1 of this embodiment comprises an information processing device 2 and an evaluation device 3.

[0014] The information processing device 2 comprises a control unit 21, a communication unit 22, a storage unit 23, a display unit 24, an operation input unit 25, and a data input / output unit 26. The control unit 21 is an electronic control unit centered around a microcomputer equipped with a CPU 211, ROM 212, RAM 213, etc. The various functions of the microcomputer are realized by the CPU 211 executing a program stored in a non-transitional physical recording medium. In this example, ROM 212 corresponds to the non-transitional physical recording medium that stores the program. Furthermore, the execution of this program executes the method corresponding to the program. Note that some or all of the functions performed by the CPU 211 may be configured hardware-wise by one or more ICs, etc. Also, the number of microcomputers constituting the control unit 21 may be one or more.

[0015] The communication unit 22 communicates data with a server equipped with a generation AI via a wide-area communication network. AI stands for Artificial Intelligence. Examples of generation AIs include ChatGPT, Gemini, and Claude. The generation AI is configured to, for example, generate and output a response in text format when a prompt representing specific instructions in text format is input.

[0016] The memory unit 23 is a storage device for storing various types of data. The memory unit 23 stores a business processing program 231 for executing business processes, which will be described later. The display unit 24 is a display device that displays various images on the display screen.

[0017] The operation input unit 25 is equipped with a keyboard and a mouse. The operation input unit 25 outputs input operation information to identify the input operations performed by the user via the keyboard and mouse.

[0018] The data input / output unit 26 performs data input and output with external devices connected via wired or wireless connection. The evaluation device 3 comprises a control unit 31, a communication unit 32, a storage unit 33, a display unit 34, an operation input unit 35, and a data input / output unit 36.

[0019] The control unit 31 is an electronic control device centered around a microcomputer equipped with a CPU 311, ROM 312, RAM 313, etc. The microcomputer may also include a cloud. The various functions of the microcomputer are realized by the CPU 311 executing a program stored in a non-transitional physical recording medium. In this example, ROM 312 corresponds to the non-transitional physical recording medium that stores the program. Furthermore, the execution of this program executes a method corresponding to the program. Note that some or all of the functions performed by the CPU 311 may be configured in hardware using one or more ICs, etc. Also, the number of microcomputers constituting the control unit 31 may be one or more.

[0020] The communication unit 32 performs data communication with devices installed outside the evaluation device 3 via a wide-area communication network. The memory unit 33 is a storage device for storing various types of data. The memory unit 33 stores a comprehensive evaluation processing program 331 for executing the comprehensive evaluation processing described later.

[0021] The display unit 34 is a display device that displays various images on the display screen. The operation input unit 35 is equipped with a keyboard and a mouse. The operation input unit 35 outputs input operation information to identify the input operations performed by the user via the keyboard and mouse.

[0022] The data input / output unit 36 ​​performs data input and output with external devices connected via wired or wireless connection. [1-2. Processing] Next, we will explain the procedure for the business processes executed by the information processing device 2.

[0023] As shown in Figure 2, the business process of the first embodiment is a process that uses a generation AI to verify the consistency between a quotation and an invoice, and functions as a processing execution unit 40. Specifically, the information processing device 2 first inputs quotation data, which shows the contents of the quotation, and invoice data, which shows the contents of the invoice corresponding to the quotation, into the processing execution unit 40. The processing execution unit 40 performs a comparison process using a generation AI to compare the numerical values ​​written in the quotation, which is indicated by the input quotation data, with the numerical values ​​written in the invoice, which is indicated by the input invoice data. The processing execution unit 40 then outputs the comparison results obtained using the generation AI.

[0024] For example, in order to perform the task of translating English documents, the system may input English document data, use a generation AI to translate the content of the input English document data into Japanese, and output the translated Japanese document.

[0025] For example, in order to perform the task of summarizing external documents, the system may input data from external documents, use a generation AI to summarize the content of the input external documents, and output a summary.

[0026] For example, in order to perform the task of proofreading a document, the system may input document data, use a generative AI to proofread the content indicated by the input document data, and output the proofread document.

[0027] For example, in order to perform the task of creating meeting minutes, the process may involve inputting meeting audio data, having an AI generate a transcript and summary based on the input meeting audio data, and then outputting the meeting minutes.

[0028] For example, in order to collect information as part of a business task, the process may involve prompting the AI ​​to perform information collection and aggregation, and then outputting a summary document.

[0029] For example, in order to generate ideas as part of a task, the process could involve prompting an AI to perform idea generation and then outputting the generated ideas.

[0030] For example, in order to generate an image as part of a task, the process may involve prompting the AI ​​to perform the image generation and then outputting the generated image. Next, the procedure for business processing performed by the information processing device 2 will be explained. Business processing is performed by launching the business processing program 231 based on the user's input operation on the information processing device 2. In the case of using a chat-style application such as ChatGPT, the business processing may be executed when the user inputs prompts and input data into the application and presses the execute button or Enter key. In such cases, the business processing program 231 is not required.

[0031] When a business process is executed, the CPU 211 of the control unit 21 determines in S10, as shown in Figure 3, whether or not the quotation data and invoice data have been input to the information processing device 2 via the data input / output unit 26.

[0032] If the quotation data and invoice data have not been entered, the process in S10 is repeated until the quotation data and invoice data are entered. Once the quotation data and invoice data are entered, the CPU 211 creates a prompt in S20 (hereinafter referred to as the numerical comparison prompt) that instructs the system to compare the numerical values ​​in the quotation indicated by the quotation data entered in S10 with the numerical values ​​in the invoice indicated by the invoice data entered in S10. The CPU 211 then sends the created numerical comparison prompt, along with the quotation data and invoice data, to the server equipped with the generation AI that is the target of the instruction (hereinafter referred to as the instruction target server).

[0033] CPU211 retrieves the comparison results sent from the target server in S30. In S40, CPU211 displays the comparison results obtained in S30 on the display screen of display unit 24 and terminates the business process.

[0034] As shown in Figure 4, in the first embodiment, a first generation AI 41, a second generation AI 42, and a third generation AI 43 are considered as generation AIs to be used in business processing. The first generation AI 41 is, for example, ChatGPT. The second generation AI 42 is, for example, Gemini. The third generation AI 43 is, for example, Claude.

[0035] Therefore, when using the first generation AI41 in business processing, the evaluator rewrites the business processing program 231 in S20 of the business processing to send a numerical comparison prompt to the instruction target server equipped with the first generation AI41, and then has the information processing device 2 execute the business processing program 231.

[0036] Similarly, when using the second generation AI42 in business processing, the evaluator rewrites the business processing program 231 in step S20 of the business processing to send a numerical comparison prompt to the target server equipped with the second generation AI42, and then has the information processing device 2 execute the business processing program 231.

[0037] Furthermore, when using the third generation AI43 in business processing, the evaluation worker rewrites the business processing program 231 in step S20 of the business processing to send a numerical comparison prompt to the target server equipped with the third generation AI43, and then has the information processing device 2 execute the business processing program 231.

[0038] In this embodiment, as described above, the evaluator rewrites the business processing program 231 to send a numerical comparison prompt to the target server for each of the first generating AI 41, the second generating AI 42, and the third generating AI 43. However, multiple programs that send a numerical comparison prompt to the target server for each of the multiple generating AIs may be prepared in advance, and the evaluator may select one of these programs on the UI screen. Alternatively, the evaluator's intervention may not be required, and another AI or program may automatically switch the target server sequentially.

[0039] As shown in Figure 5, in the first embodiment, the following evaluation metrics are used to evaluate the generation AI under consideration (i.e., the first generation AI 41, the second generation AI 42, and the third generation AI 43): processing time evaluation metric T, cost evaluation metric C, accuracy evaluation metric A, operability evaluation metric U, infrastructure compatibility evaluation metric I, external compatibility evaluation metric E, security evaluation metric S, data privacy evaluation metric P, copyright protection compliance evaluation metric R, and commercial use feasibility evaluation metric L.

[0040] The processing time evaluation index T is set so that the shorter the time it takes to execute the business process (i.e., processing time), the higher its value. In other words, the processing time evaluation index T is set to have a negative correlation with processing time. Processing time is obtained as an actual measured value. "Having a negative correlation with processing time" means not only that the processing time evaluation index T decreases continuously as processing time increases, but also that the processing time evaluation index T decreases gradually as processing time increases.

[0041] The cost evaluation metric C is set so that its value increases as the server usage fees and license fees (i.e., costs) incurred in executing business processes decrease. In other words, the cost evaluation metric C is set to have a negative correlation with cost. Cost is obtained as an actual measured value. "Having a negative correlation with cost" means not only that the cost evaluation metric C decreases continuously as costs increase, but also that the cost evaluation metric C decreases gradually as costs increase. Server usage fees are calculated, for example, by multiplying the server operating time by the unit price. License fees are calculated by multiplying the amount of prompts input to the generating AI by the unit price.

[0042] Accuracy evaluation index A represents the accuracy and reliability of the output results of the business process. Accuracy evaluation index A is obtained as a score of 0 to 1 through actual measurement or qualitative evaluation by an evaluator. The Usability Evaluation Index U represents the ease of use of the interface and the quality of the user experience. The Usability Evaluation Index U is obtained as a score of 0 to 1 through qualitative evaluation by the evaluator.

[0043] Infrastructure Compatibility Evaluation Index I assesses compatibility with existing IT infrastructure and ease of implementation. The Infrastructure Compatibility Evaluation Index I is obtained as a score between 0 and 1 by evaluators comparing the specifications of each generated AI.

[0044] External connectivity evaluation index E represents the flexibility and ease of integration with other applications or services. External connectivity evaluation index E is obtained as a score between 0 and 1 by the evaluator comparing the specifications of each generated AI.

[0045] The safety evaluation index S represents the strength of security for the infrastructure and systems. The safety evaluation index S is obtained as a score between 0 and 1 by the evaluator comparing the specifications of each generated AI.

[0046] The data privacy evaluation index P is the presence or absence of rules and settings regarding the prevention of secondary use of input data. The data privacy evaluation index P is obtained as a score of 0 or 1 by the evaluator confirming whether or not each generated AI has such measures in place.

[0047] The copyright protection compliance evaluation index R indicates the presence or absence of mechanisms to control and detect whether the output content infringes on the copyrights of others. The copyright protection compliance evaluation index R is obtained as a score of 0 or 1 by the evaluator confirming the presence or absence of compliance in each generating AI.

[0048] The commercial use evaluator index L is the availability of a commercial version (e.g., an enterprise version). The commercial use evaluator index L is obtained as a score of 0 or 1 by checking whether each generated AI supports the target audience.

[0049] As shown in Figure 6, by having the information processing device 2 execute business processing using the first generated AI 41, for example, a processing time evaluation index T=0.9, a cost evaluation index C=1.2, an accuracy evaluation index A=0.85, an operability evaluation index U=0.80, an infrastructure compatibility evaluation index I=0.60, an external linkage evaluation index E=0.40, a security evaluation index S=1.00, a data privacy evaluation index P=1, a copyright protection compliance evaluation index R=1, and a commercial use feasibility evaluation index L=1 can be obtained.

[0050] Next, the procedure for the comprehensive evaluation process performed by the evaluation device 3 will be explained. The comprehensive evaluation process is performed by activating the comprehensive evaluation program 331 through input operations by the user of the evaluation device 3. Alternatively, the comprehensive evaluation process may be automatically executed when the necessary evaluation indicators are obtained by the information processing device 2 through business processing.

[0051] When the comprehensive evaluation process is executed, the CPU 311 of the control unit 31 acquires a predetermined number of evaluation indicators for each of the multiple generated AIs under consideration in S110, as shown in Figure 7. The predetermined number of evaluation indicators are the processing time evaluation indicator T, cost evaluation indicator C, accuracy evaluation indicator A, operability evaluation indicator U, infrastructure compatibility evaluation indicator I, external compatibility evaluation indicator E, safety evaluation indicator S, data privacy evaluation indicator P, copyright protection compliance evaluation indicator R, and commercial use feasibility evaluation indicator L. In this embodiment, it is assumed that a predetermined number of evaluation indicators for each of the multiple generated AIs are stored in the storage unit 33 before the comprehensive evaluation process is executed. That is, in S110, the CPU 311 acquires a predetermined number of evaluation indicators for each of the multiple generated AIs from the storage unit 33.

[0052] In S120, CPU311 calculates an overall evaluation value Z for each of the multiple generated AIs under consideration using a predetermined number of evaluation indices obtained in S110, according to equation (1). In equation (1), α, β, γ, δ, ε, ζ, and η are weighting coefficients.

[0053]

number

[0054] In S130, CPU311 selects the generation AI with the highest overall evaluation value Z, based on the multiple overall evaluation values ​​Z calculated in S120, as the generation AI to be used for business processing. However, CPU311 may also select the generation AI by setting constraints such as "accuracy evaluation index A is 0.90 or higher" and "cost evaluation index C is 1.0 or lower".

[0055] In S140, CPU 311 displays the generated AI selected in S130 and the multiple overall evaluation values ​​Z calculated in S120 on the display screen of display unit 34, and then terminates the overall evaluation process. [1-3. Effects] According to the embodiments described above, the following effects can be obtained.

[0056] (1a) The evaluation device 3 is an information processing device 2 having a processing execution unit 40 that performs a pre-set comparison process, and is configured to calculate a comprehensive evaluation value Z which integrates multiple evaluation indicators related to causing the processing execution unit 40 to perform a comparison process for each of the multiple generated AI patterns when the generated AI in the processing execution unit 40 is replaced.

[0057] Furthermore, the comparison process performed by the processing execution unit 40 is a process that can be executed using only one generation AI. That is, the comparison process can be executed using only one of the first generation AI 41, the second generation AI 42, and the third generation AI 43. In other words, the comparison process can be executed without using at least two of the first generation AI 41, the second generation AI 42, and the third generation AI 43.

[0058] This evaluation device 3 calculates a single comprehensive evaluation index for each of the multiple generated AI patterns when the generated AI is replaced, by integrating multiple evaluation indices. Therefore, the evaluation device 3 can use the single evaluation index calculated for each of the multiple generated AI patterns to select a processing method in the processing execution unit 40, making it easier to select a processing method for efficiency and sophistication.

[0059] (1b) The evaluation device 3 is configured to select the generated AI in the processing execution unit 40 based on the overall evaluation value Z calculated for each of the multiple generated AI patterns. This eliminates the need for the evaluation device 3 to have the evaluation operator select the processing method in the processing execution unit 40, and further facilitates the selection of processing methods for efficiency and sophistication.

[0060] (1c) The evaluation device 3 is configured to calculate an overall evaluation value Z using at least one of several evaluation indicators: a processing time evaluation indicator T, an accuracy evaluation indicator A, and a cost evaluation indicator C. This enables the evaluation device 3 to select a generated AI based on an evaluation indicator that comprehensively evaluates processing time, accuracy, and cost.

[0061] (1d) The evaluation device 3 is further configured to calculate an overall evaluation value Z using at least one of the following: operability evaluation index U, infrastructure affinity evaluation index I, external connectivity evaluation index E, safety evaluation index S, data privacy evaluation index P, copyright protection compliance evaluation index R, and commercial use feasibility evaluation index L. This enables the evaluation device 3 to select generated AI based on a comprehensive index that integrates even more evaluation indicators.

[0062] (1e) The evaluation device 3 is configured to calculate the overall evaluation value Z by a weighted sum obtained by summing multiple products calculated by multiplying each of the values ​​of the processing time evaluation index T, cost evaluation index C, accuracy evaluation index A, operability evaluation index U, infrastructure affinity evaluation index I, external connectivity evaluation index E, and safety evaluation index S by a predetermined weighting coefficient α, β, γ, δ, ε, ζ, η. In this way, the evaluation device 3 can set the importance of the processing time evaluation index T, cost evaluation index C, accuracy evaluation index A, operability evaluation index U, infrastructure affinity evaluation index I, external connectivity evaluation index E, and safety evaluation index S using the weighting coefficients α, β, γ, δ, ε, ζ, η and calculate the overall evaluation value Z.

[0063] (1f) The evaluation device 3 is configured to calculate an overall evaluation value Z by multiplying the weighted sum of the data privacy evaluation index P, the copyright protection compliance evaluation index R, and the commercial use feasibility evaluation index L. This allows the evaluation device 3 to set the data privacy evaluation index P, the copyright protection compliance evaluation index R, and the commercial use feasibility evaluation index L as knockout factors.

[0064] (1g) The evaluation device 3 is configured to display the comparison results of the overall evaluation value Z for multiple processing method patterns. This allows the evaluation device 3 to make the evaluation operator aware of the comparison results of the overall evaluation value Z.

[0065] (1h) The information processing device 2 is optimized by an evaluation method that calculates a comprehensive evaluation value Z which integrates multiple evaluation indicators related to causing the processing execution unit 40 to perform a comparison process for each of the multiple generated AI patterns when the generated AI in the processing execution unit 40 is replaced. Since the evaluation device 3 makes it easy to select a processing method for efficiency and sophistication of such an information processing device 2, the manufacturing of the information processing device 2 can be made easier.

[0066] [1-4. Correspondence of Terms] In the embodiments described above, the comparison process corresponds to one information processing operation, the processing execution unit 40 corresponds to one information processing execution unit, the first generated AI 41, the second generated AI 42, and the third generated AI 43 correspond to processing methods, and the generated AI pattern corresponds to a processing method pattern.

[0067] Furthermore, the processing time evaluation index T, cost evaluation index C, accuracy evaluation index A, operability evaluation index U, infrastructure compatibility evaluation index I, external connectivity evaluation index E, security evaluation index S, data privacy evaluation index P, copyright protection compliance evaluation index R, and commercial use feasibility evaluation index L correspond to multiple evaluation indicators, the overall evaluation value Z corresponds to the overall evaluation index, and S120 corresponds to processing as the overall evaluation unit.

[0068] Furthermore, S130 corresponds to the processing as a selection unit, and the processing time evaluation index T, cost evaluation index C, accuracy evaluation index A, operability evaluation index U, infrastructure compatibility evaluation index I, external connectivity evaluation index E, and safety evaluation index S correspond to weighted evaluation indicators, S140 corresponds to the processing as a result display unit, and the comprehensive evaluation processing program 331 corresponds to the evaluation program.

[0069] [Second Embodiment] A second embodiment of this disclosure is described below with reference to the drawings. Note that the second embodiment will describe parts that differ from the first embodiment. Common components will be denoted by the same reference numerals.

[0070] [2-1. Processing] The evaluation system 1 of the second embodiment differs from the first embodiment in that the business processing and overall evaluation processing have been modified.

[0071] As shown in Figure 8, the business process of the second embodiment functions as a first processing execution unit 50, a second processing execution unit 60, and a third processing execution unit 70, and the first processing execution unit 50, the second processing execution unit 60, and the third processing execution unit 70 sequentially execute the processes.

[0072] The first processing unit 50 receives the business negotiation audio data, executes a speech segment detection process to detect speech segments, and outputs the audio data of the speech segments. The second processing unit 60 receives audio data of the speech section, performs a transcription process to write text from the input audio data, and outputs the transcribed text data.

[0073] The third processing unit 70 takes the transcribed text data as input, executes a meeting minutes creation process to create meeting minutes, and outputs the created meeting minutes data. Next, the procedure for processing business in the second embodiment will be described.

[0074] When the business process of the second embodiment is executed, the CPU 211 of the control unit 21 acquires negotiation audio data in S210, as shown in Figure 9, detects the speech intervals in the acquired negotiation audio data, and extracts the audio data from the detected speech intervals.

[0075] CPU211 performs transcription from the speech segment audio data extracted in S220, and generates transcription text data showing the transcribed characters. CPU211, in S230, creates meeting minutes by summarizing the business negotiation as indicated by the audio data from the transcribed text data generated in S220, and generates meeting minutes data that shows the created meeting minutes.

[0076] In S240, CPU211 stores the meeting minutes data generated in S230 into storage unit23 and terminates the business process. As shown in Figure 10, in the second embodiment, the first utterance interval detection algorithm 51, the second utterance interval detection algorithm 52, and the third utterance interval detection algorithm 53 are considered as algorithms to be used in the first processing execution unit 50.

[0077] As algorithms to be used in the second processing unit 60, we consider the first character transcription algorithm 61, the second character transcription algorithm 62, and the third character transcription algorithm 63.

[0078] We will consider the first minutes creation algorithm 71, the second minutes creation algorithm 72, and the third minutes creation algorithm 73 as algorithms to be used in the third processing execution unit 70. Therefore, by selecting three types of algorithms in each of the first processing unit 50, the second processing unit 60, and the third processing unit 70, the number of combinations of algorithms considered in business processing becomes 3 × 3 × 3 = 27.

[0079] For example, if the first processing unit 50 uses the first utterance detection algorithm 51, the second processing unit 60 uses the second character transcription algorithm 62, and the third processing unit 70 uses the third meeting minutes creation algorithm 73, the evaluator rewrites the business processing program 231 to use the first utterance detection algorithm 51 in S210 of the business processing, the second character transcription algorithm 62 in S220, and the third meeting minutes creation algorithm 73 in S230, and then has the information processing device 2 execute the business processing program 231.

[0080] As shown in Figure 11, by having the information processing device 2 execute business processing using the first utterance interval detection algorithm 51 in the first processing execution unit 50, for example, a processing time evaluation index T1=0.4, a cost evaluation index C1=0.7, an accuracy evaluation index A1=0.85, an operability evaluation index U1=0.80, an infrastructure compatibility evaluation index I1=0.60, an external compatibility evaluation index E1=0.40, a security evaluation index S1=1.00, a data privacy evaluation index P1=1, a copyright protection compliance evaluation index R1=1, and a commercial use feasibility evaluation index L1=1 can be obtained.

[0081] By having the information processing device 2 execute business processing using the first character transcription algorithm 61 in the second processing execution unit 60, for example, the following can be obtained: processing time evaluation index T2=0.2, cost evaluation index C2=0.5, accuracy evaluation index A2=0.90, operability evaluation index U2=0.60, infrastructure compatibility evaluation index I2=0.80, external compatibility evaluation index E2=0.60, security evaluation index S2=1.00, data privacy evaluation index P2=1, copyright protection compliance evaluation index R2=1, and commercial use feasibility evaluation index L2=1.

[0082] By having the third processing execution unit 70 execute business processing using the first minutes creation algorithm 71 on the information processing device 2, for example, the following can be obtained: processing time evaluation index T3=0.7, cost evaluation index C3=1.2, accuracy evaluation index A3=0.80, operability evaluation index U3=1.00, infrastructure compatibility evaluation index I3=0.80, external compatibility evaluation index E3=0.60, security evaluation index S3=0.80, data privacy evaluation index P3=1, copyright protection compliance evaluation index R3=0, and commercial use feasibility evaluation index L3=1.

[0083] Next, the procedure for the comprehensive evaluation process in the second embodiment will be described. When the comprehensive evaluation process of the second embodiment is executed, the CPU 311 of the control unit 31 acquires a predetermined number of evaluation indicators for the first, second, and third processing execution units 50, 60, and 70 for all algorithm combinations under consideration in S310, as shown in Figure 12.

[0084] The predetermined number of evaluation indicators for the first processing execution unit 50 are: processing time evaluation indicator T1, cost evaluation indicator C1, accuracy evaluation indicator A1, operability evaluation indicator U1, infrastructure compatibility evaluation indicator I1, external compatibility evaluation indicator E1, safety evaluation indicator S1, data privacy evaluation indicator P1, copyright protection compliance evaluation indicator R1, and commercial use feasibility evaluation indicator L1.

[0085] A predetermined number of evaluation indicators for the second processing execution unit 60 are a processing time evaluation indicator T2, a cost evaluation indicator C2, an accuracy evaluation indicator A2, an operability evaluation indicator U2, an infrastructure affinity evaluation indicator I2, an external cooperation evaluation indicator E2, a security evaluation indicator S2, a data privacy evaluation indicator P2, a copyright protection support evaluation indicator R2, and a commercial availability evaluation indicator L2.

[0086] A predetermined number of evaluation indicators for the third processing execution unit 70 are a processing time evaluation indicator T3, a cost evaluation indicator C3, an accuracy evaluation indicator A3, an operability evaluation indicator U3, an infrastructure affinity evaluation indicator I3, an external cooperation evaluation indicator E3, a security evaluation indicator S3, a data privacy evaluation indicator P3, a copyright protection support evaluation indicator R3, and a commercial availability evaluation indicator L3.

[0087] In this embodiment, it is assumed that, before the comprehensive evaluation process is executed, a predetermined number of evaluation indicators for the first, second, and third processing execution units 50, 60, and 70 in all algorithm combinations are stored in the storage unit 33.

[0088] At S320, for each of all algorithm combinations to be considered, the CPU 311 calculates a predetermined number of overall evaluation indicators as shown in FIG. 13 using the predetermined number of evaluation indicators acquired at S310. The predetermined number of overall evaluation indicators are an overall processing time evaluation indicator T all [[ID=z14]]、an overall cost evaluation indicator C all 、an overall accuracy evaluation indicator A all 、an overall operability evaluation indicator U all 、an overall infrastructure affinity evaluation indicator I all 、an overall external cooperation evaluation indicator E all 、an overall security evaluation indicator S all 、an overall data privacy evaluation indicator P all 、an overall copyright protection support evaluation indicator R all and an overall commercial availability evaluation indicator L all are.

[0089] In FIG. 13, the reference value of the evaluation indicator X for the i-th process is denoted as X i,st and the weight of the evaluation indicator X for the i-th process is denoted as w X,iThis is how it is written. The formula for calculating the predetermined number of overall evaluation indicators shown in Figure 13 is just one example, and the method for calculating the overall evaluation indicators is not limited to the formula shown in Figure 13.

[0090] Once processing in S320 is complete, CPU311, as shown in Figure 12, calculates an overall evaluation value Z in S330 for each of the algorithm combinations under consideration, using a predetermined number of overall evaluation indices calculated in S320, according to equation (2).

[0091]

number

[0092] In S340, CPU311 selects the algorithm combination with the highest overall evaluation value Z, based on the multiple overall evaluation values ​​Z calculated in S330, as the algorithm combination to be used for business processing. However, CPU311 may also select the algorithm combination by setting constraints such as "the accuracy of the first processing execution unit 50 is 0.90 or higher" and "the cost evaluation index for the entire system is 1.0 or lower."

[0093] In S350, CPU 311 displays the algorithm combination selected in S340 and the multiple overall evaluation values ​​Z calculated in S330 on the display screen of display unit 34, and then terminates the overall evaluation process.

[0094] [2-2. Effects] According to the embodiments described above, the following effects can be obtained. (2a) The evaluation device 3 is an information processing device 2 having a first processing execution unit 50, a second processing execution unit 60, and a third processing execution unit 70 configured to perform speech segment detection processing, transcription processing, and meeting minutes creation processing, respectively, and for each of the multiple algorithm patterns when the algorithms in the first processing execution unit 50, the second processing execution unit 60, and the third processing execution unit 70 are swapped, the first processing execution unit 50, the second processing execution unit 60, and the third processing execution unit 70 are configured to perform speech segment detection processing and transcription processing. The system is configured to calculate an overall evaluation value Z by integrating the following metrics related to the execution of data processing and meeting minute creation: processing time evaluation indices T1, T2, T3; cost evaluation indices C1, C2, C3; accuracy evaluation indices A1, A2, A3; operability evaluation indices U1, U2, U3; infrastructure compatibility evaluation indices I1, I2, I3; external compatibility evaluation indices E1, E2, E3; security evaluation indices S1, S2, S3; data privacy evaluation indices P1, P2, P3; copyright protection compliance evaluation indices R1, R2, R3; and commercial use feasibility evaluation indices L1, L2, L3.

[0095] Furthermore, the speech segment detection process, the transcription process, and the meeting minutes creation process, which are performed by the first processing unit 50, the second processing unit 60, and the third processing unit 70 respectively, are processes that can be executed using a single algorithm.

[0096] In other words, the speech segment detection process is a process that can be executed using only one of the first speech segment detection algorithm 51, the second speech segment detection algorithm 52, and the third speech segment detection algorithm 53.

[0097] Furthermore, the transcription process can be performed using only one of the following: the first transcription algorithm 61, the second transcription algorithm 62, and the third transcription algorithm 63.

[0098] Furthermore, the minutes creation process can be executed using only one of the following: the first minutes creation algorithm 71, the second minutes creation algorithm 72, and the third minutes creation algorithm 73.

[0099] This evaluation device 3 calculates a single comprehensive evaluation index by integrating multiple evaluation indices for each of the multiple algorithm patterns when the algorithms are swapped. Therefore, the evaluation device 3 can use the single evaluation index calculated for each of the multiple algorithm patterns to select the algorithms in the first processing execution unit 50, the second processing execution unit 60, and the third processing execution unit 70, thereby facilitating the selection of processing methods for efficiency and sophistication.

[0100] (2b) The evaluation device 3 is configured to select an algorithm in the first processing execution unit 50, the second processing execution unit 60, and the third processing execution unit 70 based on the overall evaluation value Z calculated for each of the multiple algorithm patterns. This eliminates the need for the evaluation device 3 to have an evaluator select the algorithms in the first processing execution unit 50, the second processing execution unit 60, and the third processing execution unit 70, making it even easier to select a processing method for efficiency and sophistication.

[0101] (2c) The evaluation device 3 has at least one evaluation index, the processing time evaluation index T all And, accuracy evaluation index A all and cost evaluation index C all The system is configured to calculate an overall evaluation value Z using these values. This allows the evaluation device 3 to select an algorithm based on an evaluation index that comprehensively evaluates processing time, accuracy, and cost.

[0102] (2d) The evaluation device 3 further includes the operability evaluation index U all Infrastructure compatibility evaluation index I all External collaboration evaluation index E all Safety evaluation index S all Data privacy evaluation metric P all Copyright protection compliance evaluation index R all and the commercial use evaluator L allThe system is configured to calculate the overall evaluation value Z using at least one of the following. This allows the evaluation device 3 to select an algorithm based on a comprehensive index that integrates even more evaluation indicators.

[0103] (2e) The evaluation device 3 uses the processing time evaluation index T all Cost evaluation index C all Accuracy evaluation index A all , Operability evaluation index U all Infrastructure compatibility evaluation index I all External collaboration evaluation index E all Safety evaluation index S all The system is configured to calculate the overall evaluation value Z by a weighted sum, which is obtained by multiplying each of the values ​​by a predetermined weighting coefficient α, β, γ, δ, ε, ζ, η and summing the products. This allows the evaluation device 3 to calculate the overall evaluation value Z by setting the importance of processing time, cost, accuracy, operability, infrastructure compatibility, external connectivity, and safety using the weighting coefficients α, β, γ, δ, ε, ζ, η.

[0104] The evaluation device 3 assigns a weight w to the evaluation index X for the i-th process. X,i The system is configured to calculate an overall evaluation value Z by multiplying by multiple products obtained by w. Thus, the evaluation device 3 assigns weights w to the importance of the evaluation index X for the i-th process. X,i By setting these parameters, the overall evaluation value Z can be calculated.

[0105] (2f) The evaluation device 3 is the data privacy evaluation index P all Copyright protection compliance evaluation index R all and the commercial use evaluator L all The system is configured to calculate the overall evaluation value Z by multiplying the weighted sum by the value P. Thus, the evaluation device 3 calculates the data privacy evaluation index P. all Copyright protection compliance evaluation index R all and the commercial use evaluator L all This can be set as the knockout factor.

[0106] [2-3. Correspondence of Terms] In the embodiments described above, the speech segment detection process, the transcription process, and the minutes creation process correspond to multiple information processing operations, and the first processing execution unit 50, the second processing execution unit 60, and the third processing execution unit 70 correspond to multiple information processing execution units.

[0107] Furthermore, the first, second, and third speech segment detection algorithms 51, 52, and 53, the first, second, and third transcription algorithms 61, 62, and 63, and the first, second, and third meeting minute creation algorithms 71, 72, and 73 correspond to processing methods, while the algorithm patterns correspond to processing method patterns. Each algorithm is composed of in-house developed models, third-party services, open-source libraries, LLMs, etc. LLM stands for Large Language Models.

[0108] Furthermore, processing time evaluation indices T1, T2, T3, T all Cost evaluation indicators C1, C2, C3, C all Accuracy evaluation index A1, A2, A3, A all Operability evaluation index U1, U2, U3, U all Infrastructure compatibility evaluation index I1, I2, I3, I all External collaboration evaluation indicators E1, E2, E3, E all , safety evaluation index S1,S2,S3,S all Data privacy evaluation metrics P1, P2, P3, P all Copyright protection compliance evaluation index R1, R2, R3, R all and commercial use evaluator indicators L1, L2, L3, L all This corresponds to multiple evaluation metrics.

[0109] Furthermore, the processing time evaluation index T all Cost evaluation index C all Accuracy evaluation index A all , Operability evaluation index U all Infrastructure compatibility evaluation index I all External collaboration evaluation index E all Safety evaluation index S all S330 corresponds to the weighted evaluation index, S340 corresponds to the processing as the overall evaluation unit, and S340 corresponds to the processing as the selection unit.

[0110] [3. Other Embodiments] Although one embodiment of the present disclosure has been described above, the present disclosure is not limited to the above embodiment and can be implemented in various modified forms.

[0111] (3a) In the second embodiment described above, the business process was shown in which the first processing execution unit 50, the second processing execution unit 60, and the third processing execution unit 70 sequentially executed the processes. However, as shown in Figure 14, the business process may also be configured to sequentially execute the first processing execution unit 50, the second processing execution unit 60, the third processing execution unit 70, and the fourth processing execution unit 80.

[0112] The first processing unit 50 receives the business negotiation audio data, detects the speech interval, and outputs the audio data of the speech interval. The second processing unit 60 receives the speech data of the utterance section and the registered speaker's speech data, performs speaker separation, and outputs the speech data with the speaker separated (hereinafter referred to as "speaker-separated speech data").

[0113] The third processing unit 70 receives speaker-separated audio data, performs transcription from the input speaker-separated audio data, and outputs the transcribed text data. The fourth processing unit 80 takes the transcribed text data as input, creates meeting minutes data, and outputs the created meeting minutes data.

[0114] (3b) In the second embodiment described above, the business process was shown in which the first processing execution unit 50, the second processing execution unit 60, and the third processing execution unit 70 sequentially executed the processes. However, as shown in Figure 15, the business process may be configured such that the first processing execution unit 50 and the second processing execution unit 60 sequentially execute the processes, and then the third processing execution unit 70 and the fourth processing execution unit 80 execute the processes in parallel.

[0115] The first processing unit 50 receives the business negotiation audio data, detects the speech interval, and outputs the audio data of the speech interval. The second processing unit 60 receives audio data of the speech section, performs transcription from the input audio data, and outputs the transcribed text data.

[0116] The third processing unit 70 takes the transcribed text data as input, creates meeting minutes data, and outputs the created meeting minutes data. The fourth processing unit 80 takes the transcribed text data as input, performs a business negotiation evaluation, and outputs business negotiation evaluation result data showing the results of the business negotiation evaluation.

[0117] (3c) In the second embodiment described above, the business process was shown in which the first processing execution unit 50, the second processing execution unit 60, and the third processing execution unit 70 sequentially executed the processes. However, as shown in Figure 16, the business process may be configured such that after the first processing execution unit 50, the second processing execution unit 60, and the third processing execution unit 70 sequentially execute the processes, the fourth processing execution unit 80 and the fifth processing execution unit 90 execute the processes in parallel.

[0118] The first processing unit 50 receives the business negotiation audio data, detects the speech interval, and outputs the audio data of the speech interval. The second processing unit 60 receives the speech data of the utterance section and the registered speaker's speech data, performs speaker separation, and outputs the speaker-separated speech data.

[0119] The third processing unit 70 receives speaker-separated audio data, performs transcription from the input speaker-separated audio data, and outputs the transcribed text data. The fourth processing unit 80 takes the transcribed text data as input, creates meeting minutes data, and outputs the created meeting minutes data.

[0120] The fifth processing unit 90 receives the speaker-separated audio data and the transcribed text data, performs a business negotiation evaluation, and outputs business negotiation evaluation result data. (3d) In the first embodiment described above, a method for calculating the overall evaluation value Z was shown using the processing time evaluation index T, cost evaluation index C, accuracy evaluation index A, operability evaluation index U, infrastructure affinity evaluation index I, external connectivity evaluation index E, security evaluation index S, data privacy evaluation index P, copyright protection compliance evaluation index R, and commercial use feasibility evaluation index L. However, the overall evaluation value Z may also be calculated using only the processing time evaluation index T, cost evaluation index C, and accuracy evaluation index A. That is, the processing time evaluation index T, cost evaluation index C, and accuracy evaluation index A are mandatory evaluation indices, while the operability evaluation index U, infrastructure affinity evaluation index I, external connectivity evaluation index E, security evaluation index S, data privacy evaluation index P, copyright protection compliance evaluation index R, and commercial use feasibility evaluation index L are optional evaluation indices.

[0121] (3e) In the above embodiment, the three information processing operations performed by the first processing unit 50, the second processing unit 60, and the third processing unit 70 are shown to be different from each other. However, in some or all of the two or more information processing operations performed by each of the two or more processing units, the information processing operations may be the same from each other.

[0122] The control unit 31 and its method described in this disclosure may be implemented by a dedicated computer provided by configuring a processor and memory programmed to perform one or more functions embodied by a computer program. Alternatively, the control unit 31 and its method described in this disclosure may be implemented by a dedicated computer provided by configuring a processor by one or more dedicated hardware logic circuits. Alternatively, the control unit 31 and its method described in this disclosure may be implemented by one or more dedicated computers configured by a combination of a processor and memory programmed to perform one or more functions and a processor configured by one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by the computer on a computer-readable non-transitional tangible recording medium. The method for implementing the functions of each part included in the control unit 31 does not necessarily need to include software, and all of its functions may be implemented using one or more hardware components.

[0123] Multiple functions of one component in the above embodiment may be realized by multiple components, or one function of one component may be realized by multiple components. Furthermore, multiple functions of multiple components may be realized by one component, or one function realized by multiple components may be realized by one component. Also, some parts of the configuration of the above embodiment may be omitted. Furthermore, at least some parts of the configuration of the above embodiment may be added to or replaced with the configuration of other above embodiments.

[0124] In addition to the evaluation device 3 described above, this disclosure can also be realized in various forms, such as a system that uses the evaluation device 3 as a component, a program for causing the computer to function as the evaluation device 3, a non-transitional physical recording medium such as a semiconductor memory on which this program is recorded, and an evaluation method. [Technical concepts disclosed in this specification] [Item 1] An information processing device having one information processing execution unit that executes one information processing, or multiple information processing execution units that each execute a plurality of information processing, wherein the comprehensive evaluation unit is configured to calculate a comprehensive evaluation index that integrates multiple evaluation indicators related to causing the one information processing execution unit to execute the one information processing, or causing the plurality of information processing execution units to execute the plurality of information processing, for each of the plurality of processing method patterns when the processing method in the one information processing execution unit that executes the one information processing, is swapped, or for each of the plurality of processing method patterns when the processing method in each of the plurality of information processing units that executes the plurality of information processing, is swapped. An evaluation device equipped with the following features.

[0125] [Item 2] The evaluation device described in item 1, An evaluation device in which the multiple information processing operations performed by each of the multiple information processing execution units include operations that are identical to each other or operations that are different to each other.

[0126] [Item 3] An evaluation device as described in item 1 or item 2, further, A selection unit is configured to select the processing method in one of the information processing execution units, or the processing method in each of the multiple information processing execution units, based on the overall evaluation index calculated for each of the multiple processing method patterns. An evaluation device equipped with the following features.

[0127] [Item 4] An evaluation device described in any one of items 1 to 3, The aforementioned comprehensive evaluation unit includes, as the plurality of evaluation indicators, at least, A processing time evaluation index relating to the processing time required when one information processing unit executes one information processing unit, or when each of the multiple information processing units executes the multiple information processing units, A precision evaluation index relating to the precision of the result of the one information processing execution unit performing the one information processing, or the result of each of the multiple information processing execution units performing the multiple information processing, A cost evaluation index relating to the cost incurred by the one information processing unit executing the one information processing, or by each of the multiple information processing units executing the multiple information processing, An evaluation device configured to calculate the overall evaluation index using the above-mentioned method.

[0128] [Item 5] The evaluation device described in item 4, further, The comprehensive evaluation unit is an evaluation device configured to calculate the comprehensive evaluation index using at least one of the following: an operability evaluation index, an infrastructure compatibility evaluation index, an external connectivity evaluation index, a safety evaluation index, a data privacy evaluation index, a copyright protection compliance evaluation index, and a commercial use feasibility evaluation index.

[0129] [Item 6] An evaluation device described in any one of items 1 to 5, The comprehensive evaluation unit is configured to calculate the comprehensive evaluation index by a weighted sum, which is obtained by summing one or more products calculated by multiplying each of the values ​​of the weighted evaluation index by a weighting coefficient predetermined for each of the weighted evaluation index, with at least one of the plurality of evaluation indexes being the weighted evaluation index.

[0130] [Item 7] The evaluation device described in item 6, The overall evaluation unit is an evaluation device configured to calculate the overall evaluation index by multiplying the weighted sum by one or more evaluation indicators other than the weighted target evaluation index from among the plurality of evaluation indicators.

[0131] [Item 8] An evaluation device described in any one of items 1 to 7, further, An evaluation device comprising a results display unit configured to display comparison results obtained by comparing the respective overall evaluation indicators for the aforementioned plurality of processing method patterns.

[0132] [Item 9] An evaluation method for calculating a comprehensive evaluation index that integrates multiple evaluation indicators related to causing the one information processing unit to execute the one information processing, or causing the multiple information processing units to execute the multiple information processing, in an information processing device having one information processing unit to execute one information processing, or multiple information processing units to execute each of the multiple information processing, when the processing method of the one information processing unit to execute the one information processing is swapped, or when the processing method of each of the multiple information processing units to execute the multiple information processing is swapped.

[0133] [Item 10] Computers, An information processing device having one information processing execution unit that executes one information processing, or multiple information processing execution units that each execute a plurality of information processing, wherein the comprehensive evaluation unit is configured to calculate a comprehensive evaluation index that integrates multiple evaluation indicators related to causing the one information processing execution unit to execute the one information processing, or causing the plurality of information processing execution units to execute the plurality of information processing, for each of the plurality of processing method patterns when the processing method in the one information processing execution unit that executes the one information processing, is swapped, or for each of the plurality of processing method patterns when the processing method in each of the plurality of information processing units that executes the plurality of information processing, is swapped. An evaluation program designed to function as such.

[0134] [Item 11] An information processing device having one information processing execution unit that performs one information processing, or multiple information processing execution units that perform each of multiple information processing, An information processing apparatus optimized by an evaluation method that calculates a comprehensive evaluation index that integrates multiple evaluation indexes related to causing the one information processing execution unit to execute the one information processing, or causing the multiple information processing execution units to execute the multiple information processing, for each of the multiple processing method patterns when the processing method in the one information processing execution unit that executes the one information processing, is swapped, or for each of the multiple processing method patterns when the processing method in each of the multiple information processing, which is executed by each of the multiple information processing execution units, is swapped. [Explanation of Symbols]

[0135] 2... Information processing device, 3... Evaluation device, 40... Processing execution unit, 50... First processing execution unit, 60... Second processing execution unit, 60... Processing execution unit, 70... Third processing execution unit, 331... Comprehensive evaluation processing program

Claims

1. An information processing device having one information processing execution unit that executes one information processing, or multiple information processing execution units that each execute a plurality of information processing, wherein the comprehensive evaluation unit is configured to calculate a comprehensive evaluation index that integrates multiple evaluation indicators related to causing the one information processing execution unit to execute the one information processing, or causing the plurality of information processing execution units to execute the plurality of information processing, for each of the plurality of processing method patterns when the processing method in the one information processing execution unit that executes the one information processing, is swapped, or for each of the plurality of processing method patterns when the processing method in each of the plurality of information processing units that executes the plurality of information processing, is swapped. Equipped with, The aforementioned comprehensive evaluation unit includes, as the plurality of evaluation indicators, at least, A processing time evaluation index relating to the processing time required when one information processing unit executes one information processing unit, or when each of the multiple information processing units executes the multiple information processing units, A precision evaluation index relating to the precision of the result of the one information processing execution unit performing the one information processing, or the result of each of the multiple information processing execution units performing the multiple information processing, A cost evaluation index relating to the cost incurred by the one information processing unit executing the one information processing, or by each of the multiple information processing units executing the multiple information processing, The system is configured to calculate the overall evaluation index using the above, The aforementioned comprehensive evaluation unit is further configured to calculate the comprehensive evaluation index using at least one of the following: an operability evaluation index, an infrastructure compatibility evaluation index, an external connectivity evaluation index, and a safety evaluation index. The overall evaluation unit is configured to calculate the overall evaluation index by a weighted sum, which is obtained by summing one or more products calculated by multiplying each of the values ​​of the weighted evaluation index by a weighting coefficient predetermined for each of the weighted evaluation index, with at least one of the plurality of evaluation indexes being the weighted evaluation index. The comprehensive evaluation unit is an evaluation device configured to calculate the comprehensive evaluation index by multiplying the weighted sum by at least one of the multiple evaluation indicators, namely the data privacy evaluation index, the copyright protection compliance evaluation index, and the commercial use feasibility evaluation index, as a knockout factor.

2. An evaluation apparatus according to claim 1, An evaluation device in which the multiple information processing operations performed by each of the multiple information processing execution units include operations that are identical to each other or operations that are different to each other.

3. An evaluation apparatus according to claim 1 or claim 2, further, A selection unit configured to select the processing method in one of the information processing execution units, or the processing method in each of the multiple information processing execution units, based on the overall evaluation index calculated for each of the multiple processing method patterns. An evaluation device equipped with the following features.

4. An evaluation apparatus according to claim 1 or claim 2, further, An evaluation device comprising a results display unit configured to display comparison results obtained by comparing the respective overall evaluation indicators for the aforementioned plurality of processing method patterns.

5. In an information processing device having one information processing execution unit that executes one information processing, or multiple information processing execution units that each execute a plurality of information processing, a comprehensive evaluation index is calculated by integrating multiple evaluation indicators related to causing the one information processing execution unit to execute the one information processing, or causing the plurality of information processing execution units to execute the plurality of information processing, for each of the plurality of processing method patterns when the processing method in the one information processing execution unit that executes the one information processing, is swapped, or for each of the plurality of processing method patterns when the processing method in each of the plurality of information processing units that executes the plurality of information processing, each of the plurality of information processing execution units that executes the one information processing, As the aforementioned multiple evaluation indicators, at least, A processing time evaluation index relating to the processing time required when one information processing unit executes one information processing unit, or when each of the multiple information processing units executes the multiple information processing units, A precision evaluation index relating to the precision of the result of the one information processing execution unit performing the one information processing, or the result of each of the multiple information processing execution units performing the multiple information processing, A cost evaluation index relating to the cost incurred by the one information processing unit executing the one information processing, or by each of the multiple information processing units executing the multiple information processing, The overall evaluation index is calculated using the above, Furthermore, the overall evaluation index is calculated using at least one of the following: operability evaluation index, infrastructure compatibility evaluation index, external collaboration evaluation index, and safety evaluation index. The overall evaluation index is calculated by summing the products of one or more values ​​obtained by multiplying each of the values ​​of the weighted evaluation index by a predetermined weighting coefficient, with at least one of the aforementioned multiple evaluation indices being used as the weighted evaluation index, and then adding these products together. An evaluation method for calculating the overall evaluation index by multiplying the weighted sum by at least one of the above multiple evaluation indicators, namely the data privacy evaluation index, the copyright protection compliance evaluation index, and the commercial use feasibility evaluation index, as a knockout factor.

6. Computers, An information processing device having one information processing execution unit that executes one information processing, or multiple information processing execution units that each execute a plurality of information processing, wherein the comprehensive evaluation unit is configured to calculate a comprehensive evaluation index that integrates multiple evaluation indicators related to causing the one information processing execution unit to execute the one information processing, or causing the plurality of information processing execution units to execute the plurality of information processing, for each of the plurality of processing method patterns when the processing method in the one information processing execution unit that executes the one information processing, is swapped, or for each of the plurality of processing method patterns when the processing method in each of the plurality of information processing units that executes the plurality of information processing, is swapped. To make it function as, The aforementioned comprehensive evaluation unit includes, as the plurality of evaluation indicators, at least, A processing time evaluation index relating to the processing time required when one information processing unit executes one information processing unit, or when each of the multiple information processing units executes the multiple information processing units, A precision evaluation index relating to the precision of the result of the one information processing execution unit performing the one information processing, or the result of each of the multiple information processing execution units performing the multiple information processing, A cost evaluation index relating to the cost incurred by the one information processing unit executing the one information processing, or by each of the multiple information processing units executing the multiple information processing, The system is configured to calculate the overall evaluation index using the above, The aforementioned comprehensive evaluation unit is further configured to calculate the comprehensive evaluation index using at least one of the following: an operability evaluation index, an infrastructure compatibility evaluation index, an external connectivity evaluation index, and a safety evaluation index. The overall evaluation unit is configured to calculate the overall evaluation index by a weighted sum, which is obtained by summing one or more products calculated by multiplying each of the values ​​of the weighted evaluation index by a weighting coefficient predetermined for each of the weighted evaluation index, with at least one of the plurality of evaluation indexes being the weighted evaluation index. The comprehensive evaluation unit is an evaluation program configured to calculate the comprehensive evaluation index by multiplying the weighted sum by at least one of the multiple evaluation indicators, namely the data privacy evaluation index, the copyright protection compliance evaluation index, and the commercial use feasibility evaluation index, as a knockout factor.

7. An information processing device having one information processing execution unit that performs one information processing, or multiple information processing execution units that perform each of multiple information processing, For each of the multiple processing method patterns when the processing method in the single information processing executed by the single information processing unit is swapped, or for each of the multiple processing method patterns when the processing method in each of the multiple information processing executed by each of the multiple information processing units is swapped, a comprehensive evaluation index is calculated by integrating multiple evaluation indicators related to having the single information processing unit execute the single information processing, or having the multiple information processing units execute the multiple information processing, As the aforementioned multiple evaluation indicators, at least, A processing time evaluation index relating to the processing time required when one information processing unit executes one information processing unit, or when each of the multiple information processing units executes the multiple information processing units, A precision evaluation index relating to the precision of the result of the one information processing execution unit performing the one information processing, or the result of each of the multiple information processing execution units performing the multiple information processing, A cost evaluation index relating to the cost incurred by the one information processing unit executing the one information processing, or by each of the multiple information processing units executing the multiple information processing, The overall evaluation index is calculated using the above, Furthermore, the overall evaluation index is calculated using at least one of the following: operability evaluation index, infrastructure compatibility evaluation index, external collaboration evaluation index, and safety evaluation index. The overall evaluation index is calculated by summing the products of one or more values ​​obtained by multiplying each of the values ​​of the weighted evaluation index by a predetermined weighting coefficient, with at least one of the aforementioned multiple evaluation indices being used as the weighted evaluation index, and then adding these products together. An information processing device optimized by an evaluation method that calculates the overall evaluation index by multiplying the weighted sum by at least one of the above multiple evaluation indexes, namely the data privacy evaluation index, the copyright protection compliance evaluation index, and the commercial use feasibility evaluation index, as a knockout factor.

Images