Feedback Processing System

Abstract

This system provides a feedback processing system that appropriately provides feedback on response data collected from multiple respondents. [Solution] The feedback processing system comprises a data management unit that manages response data collected from multiple respondents, a response management platform that allows viewing and editing of response data via a web interface, an AI processing means that generates evaluation information and feedback content using a generation AI on the response data stored in the data management unit and records the generated evaluation information and feedback content in the data management unit, and an interface operation means that automatically performs input operations for evaluation information and feedback content to corresponding input areas on the administrator interface provided by the response management platform, and operations to save the input content.

Description

Technical Field

[0001] The present invention relates to a technique for generating evaluation information and feedback content based on response data collected from respondents, and reflecting the evaluation information and feedback content on a response management platform.

Background Art

[0002] Conventionally, for answers transmitted via a communication network such as the Internet, an automatic scoring and marking method has been proposed in which it is determined whether the answer is in a coded form or a text (description) form, and the answer is automatically scored, and comments prepared in advance are automatically displayed in a predetermined column of the answer sheet, and the corrector can modify or add the comments as necessary (Japanese Patent Application Laid-Open No. 2001-249608).

[0003] However, in the conventional technology as described above, since the comments can depend on the prepared text, manual work may be involved in creating flexible feedback according to the questions and answer contents. In addition, when the number of respondents or questions increases, the workload of inputting and saving on the administrator interface provided by the response management platform may increase. Furthermore, when adopting a configuration in which processing is performed using an external service, the processing may be interrupted due to communication failures, timeouts, etc.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The problem to be solved by the present invention is to efficiently automate the generation of evaluation information and feedback content based on response data, and the reflection (input / saving) of the evaluation information and feedback content on a response management platform. [Means for solving the problem]

[0006] The data management department manages the response data collected from multiple respondents, A response management platform that allows viewing and editing of the aforementioned response data via a web interface, An AI processing means that generates evaluation information and feedback content using a generating AI from the response data stored in the data management unit, and records the generated evaluation information and feedback content in the data management unit, Based on the evaluation information and feedback content recorded in the data management unit, an interface operation means automatically performs the operation of inputting evaluation information and feedback content into the corresponding input area on the administrator interface provided by the response management platform, and the operation of saving the input content. A feedback processing system characterized by comprising the following:

[0007] In a feedback processing system, The AI ​​processing means has a target identification function that identifies and extracts only data columns or data items containing descriptive answers from the response data stored in the data management unit, and performs batch processing that sends the descriptive answers for multiple respondents included in the identified data columns or data items to the generating AI in a single communication request, and excludes the multiple-choice response data from the processing target of the generating AI.

[0008] In a feedback processing system, The AI ​​processing means is characterized by performing structured recording processing, which structures the feedback content for each of the multiple questions for a single respondent using predetermined tags or delimiters, and aggregates and records it in a single data field on the data management unit.

[0009] In a feedback processing system, The feedback processing system is characterized in that the interface operation means has the function of analyzing the tags or delimiters from the data aggregated in the single data field to extract the feedback content for each question, and individually mapping and inputting it into the feedback input field corresponding to each question on the administrator interface.

[0010] In a feedback processing system, When the interface operation means performs sequential processing on multiple respondents, for the first respondent to be processed, it opens the administrator interface using a URL that directly specifies the respondent's editing screen, A feedback processing system characterized by using a hybrid transition method for screen transitions for the second and subsequent respondents, by emulating operations on user interface elements that perform transition processing to the next response, which are provided in the administrator interface to display the editing screen for the next respondent.

[0011] In a feedback processing system, The AI ​​processing means can be configured externally to specify a starting position, which includes at least question identification information that identifies the question to be processed and respondent identification information that identifies the respondent to be processed, as the starting position for generating evaluation and feedback content for the response data. A feedback processing system characterized by having a control function that starts the generation process based on the start position specification information, and restarts the generation process based on the start position specification information if the process is interrupted midway.

[0012] In a feedback processing system, The AI ​​processing means analyzes the response content from the generated AI and, if it contains a predetermined string indicating an error or does not conform to a predetermined response format, performs retry control to automatically retry the question. A feedback processing system characterized by having a determination logic that outputs predetermined feedback content indicating that no response was given without performing the retry if the response data itself is blank.

[0013] In a feedback processing system, The interface operation means is configured using at least one of RPA (Robotic Process Automation) software, a browser extension, or a script program, and by mimicking human browser operations, it enables the input and saving of the evaluation information and feedback content to the administrator interface. A feedback processing system characterized by having a waiting control function that, when a screen transition or save operation is performed on the administrator interface, waits for the next operation until it detects the appearance of an inputtable user interface element on the destination screen, or the occurrence of a reflection completion state indicating that the input results of the evaluation information and the feedback content have been reflected in the response management platform. [Effects of the Invention]

[0014] According to the present invention, the process from generating evaluation information and feedback content to reflecting it in the response management platform can be automated, contributing to improved operational efficiency and stability. Other issues, configurations, and effects not mentioned above will be clarified by the following description of the embodiments. [Brief explanation of the drawing]

[0015] [Figure 1] This is a block diagram showing the configuration of a feedback processing system. [Figure 2] This is a flowchart showing the main processing steps of the feedback system. [Figure 3] This flowchart shows the processing procedure by the AI ​​processing means. [Figure 4]It is a flowchart showing the reflection processing procedure by the interface operation means.

Embodiment for Carrying Out the Invention

[0016] Hereinafter, the terms used in this specification are defined. The definition of each term is for convenience of explanation and does not limit the technical scope of the present invention.

[0017] In this specification, the "respondent" refers to a person who submits an answer to a test, task, questionnaire, etc.

[0018] In this specification, the "answer data" refers to data including the answer content to the questions input by the respondent. The answer data may include at least one of descriptive answers and selective answers.

[0019] In this specification, the "data management unit" refers to a storage area or functional unit that manages the answer data collected from a plurality of respondents and can record evaluation information and feedback content in association with the answer data. The data management unit 10 can be realized by a spreadsheet, a database, a file, etc. Note that the answer data may be managed in a table format consisting of rows in units of respondents and columns in units of questions, and at this time, units such as each column or cell can be treated as data columns or data items.

[0020] In this specification, the "answer management platform" refers to a platform that enables viewing and editing of answer data via a web interface, and can be realized by the answer management platform 20 shown in FIG. 1. In this specification, the "administrator interface" refers to a screen (user interface) provided by the answer management platform 20 and having an input area for evaluation information and feedback content and a save operation, and can be realized by the administrator interface 21 shown in FIG. 1.

[0021] In this specification, "evaluation information" refers to evaluation results assigned to each question or respondent, and may include at least one of the following: scores, ratings, graded ratings, etc. In this specification, "feedback content" refers to information such as comments presented to the respondent, and may include corrections of correctness or incorrectness, reasons for deductions, suggestions for improvement, etc.

[0022] In this specification, "Generative AI" refers to an external AI (service or model) capable of generating evaluation information and feedback content based on input text, etc., and may be provided as the Generative AI service 40 shown in Figure 1. The Generative AI may be used via network communication.

[0023] In this specification, "AI processing means" refers to a functional unit that inputs at least a portion of the response data stored in the data management unit 10 to the generating AI (generating AI service 40), acquires evaluation information and feedback content, and records it in the data management unit 10. The AI ​​processing means 31 can perform a target identification function that identifies and extracts only data columns or data items that contain descriptive answers from the response data, a batch processing function that processes descriptive answers for multiple respondents together, a response analysis function and an error detection function that analyze the response content from the generating AI (generating AI service 40) to detect errors and format inconsistencies, a retry control function that automatically re-executes according to the detection result, and a blank determination function that determines whether or not the response data is blank.

[0024] In this specification, "target identification" refers to the process of identifying and extracting data columns or data items containing descriptive answers from the response data stored in the data management unit 10, and excluding the selection-type response data from processing by the generation AI (generation AI service 40). In this specification, the function that performs such target identification is referred to as the target identification function. In this specification, "batch processing" refers to the process of sending descriptive answers from multiple respondents to the generation AI (generation AI service 40) in a single communication request, and the unit of grouping (batch size) may be fixed or variable.

[0025] In this specification, "tag" or "delimiter" refers to an identifier or delimiter expression assigned to enable the identification of feedback content for multiple questions on a question-by-question basis. In this specification, "structuring" refers to enabling the identification of feedback content for each question using tags or delimiters. In this specification, "single data field" refers to a field for aggregating and recording the structured feedback content for multiple questions into a single storage unit (e.g., a single cell, a single record attribute, etc.) on the data management unit 10. In this specification, a series of processes for structuring the feedback content using tags or delimiters and aggregating and recording it in the single data field may be referred to as structured recording processing. For example, for the feedback content for multiple questions, the identifier corresponding to each question (e.g., question number or question ID) and the feedback text may be linked together with a predetermined tag or delimiter and stored in a single data field as a string, and the feedback content for each question may be extracted in a later stage using this identifier. The specific format of the tags or delimiters used in this case can be arbitrarily determined according to the system being operated. For example, you could use a format that combines a tag indicating the question number with the feedback content, such as "Q1: ..." and "Q2: ...", or a format like "Question 1: ..." and "Question 2: ...".

[0026] In this specification, "mapping input" refers to the process of analyzing tags or delimiters from the feedback content aggregated in a single data field to extract the feedback content for each question, and then individually associating and inputting it into the feedback input field corresponding to each question on the administrator interface 21.

[0027] In this specification, "interface operation means" refers to a functional unit that automatically performs input and save operations on the corresponding input area on the administrator interface 21 based on evaluation information and feedback content recorded in the data management unit 10, and can be realized by the interface operation means 32 shown in Figure 1. The interface operation means 32 is composed of RPA (Robotic Process Automation) software, browser extensions, or script programs, and can perform processing by mimicking human browser operations. For example, Power Automate for Desktop (PAD) may be used as RPA software to automate operations on the administrator interface 21.

[0028] In this specification, "URL that directly specifies the editing screen" refers to identification information (such as a URL) that directly displays the editing screen corresponding to a specific respondent. In this specification, "transition process to the next response" refers to a transition function provided in the administrator interface 21 to display the editing screen of the next respondent, and can be performed by operations on user interface elements such as buttons, links, and menus.

[0029] In this specification, "wait control" means a control that waits after the execution of a save operation or screen transition, before moving on to the next automated operation, until it detects the appearance of an inputtable user interface element or the occurrence of a state indicating that the input result has been reflected in the response management platform 20. In this specification, "reflection complete state" means a state indicating that the input result has been reflected, and may be in any form such as a display message, a change in status display, or activation of an input field.

[0030] In this specification, "start position information" refers to information that identifies the position at which the evaluation and feedback content generation process should begin, and includes at least question identification information that identifies the question to be processed and respondent identification information that identifies the respondent to be processed. The start position information 11 can be set externally, and if the process is interrupted, the process can be resumed based on the start position information 11. Furthermore, the start position information 11 may be cleared when the generation process for all respondents is completed.

[0031] In this specification, "retry" refers to a control that automatically re-executes the generation AI (generation AI service 40) processing related to a question when the response content from the generation AI (generation AI service 40) contains a predetermined string indicating an error or does not conform to a predetermined response format. Conversely, if the answer data itself is blank, a predetermined feedback content indicating that the question is unanswered may be output without performing a retry. This process of automatically controlling whether or not to retry and the number of retries depending on the state of the response content and answer data is referred to as retry control in this specification. Furthermore, the processing logic for determining whether or not an error string is included in the response content or whether or not the answer data is blank may be referred to as determination logic in this specification.

[0032] In this specification, "tabular file" refers to a tabular data file (for example, in a format viewable by spreadsheet software) obtained by exporting the contents of the data management unit 10, and can be implemented as the tabular file 12 shown in Figure 1. The interface operation means 32 may directly refer to the data management unit 10 or refer to the tabular file 12.

[0033] As shown in Figure 1, the feedback processing system according to this embodiment consists of a data management unit 10 for managing response data, a response management platform 20 that allows viewing and editing of response data via a web interface, a feedback processing system main unit 30 that operates in cooperation with these, and a generation AI (generation AI service 40) as its basic elements. These elements can be configured to communicate with each other via a network.

[0034] The data management unit 10 is configured to hold response data collected from multiple respondents and to record evaluation information and feedback content in association with said response data. The data management unit 10 can be implemented as a spreadsheet, database, file, etc., and may be located on the cloud. The data management unit 10 may also be configured to hold start position specification information 11 (including question identification information and respondent identification information).

[0035] The response management platform 20 is a platform that enables viewing and editing of response data via a web interface. The response management platform 20 provides an administrator interface 21, on which input areas corresponding to questions (e.g., input areas for evaluation information and feedback input fields) are arranged, and it may be configured to allow saving operations to confirm the input content. Furthermore, the administrator interface 21 may have a transition function (transition process to the next response) for sequentially displaying editing screens for multiple respondents.

[0036] The feedback processing system main unit 30 is a functional block that works in conjunction with the data management unit 10 and the response management platform 20 to automate the generation and reflection of evaluation information and feedback content. The feedback processing system main unit 30 comprises at least an AI processing means 31 and an interface operation means 32. The feedback processing system main unit 30 may be implemented on a single information processing device or distributed across multiple information processing devices.

[0037] The AI ​​processing means 31 is a functional unit that acquires response data stored in the data management unit 10, generates evaluation information and feedback content for the response data using a generation AI (generation AI service 40), and records the generation results in the data management unit 10. The AI ​​processing means 31 can receive the generation results through communication with the generation AI (generation AI service 40), and can analyze and record the received generation results in a predetermined format. If start position specification information 11 is set, the AI ​​processing means 31 can start or resume processing based on the start position specification information 11. For example, the AI ​​processing means 31 may be implemented by a scripting environment that operates on a cloud-based spreadsheet, such as Google Apps Script (GAS).

[0038] The Generative AI (Generative AI Service 40) is an external AI service or model capable of generating evaluation information and feedback content based on input response data, etc. The Generative AI (Generative AI Service 40) is used via network communication and can be invoked by a communication interface such as an API.

[0039] The interface operation means 32 is a functional unit that automatically performs input and save operations on the corresponding input area on the administrator interface 21 provided by the response management platform 20, based on the evaluation information and feedback content recorded in the data management unit 10. The interface operation means 32 is composed of RPA software, browser extensions, or script programs, and can perform input and save operations by mimicking human browser operations.

[0040] The interface operation means 32 may include a waiting control function that waits for the next operation after a screen transition or save operation on the administrator interface 21 until it detects the appearance of an inputtable user interface element or the occurrence of a completed reflection state. Furthermore, when performing sequential processing for multiple respondents, the administrator interface 21 may be opened for the first respondent to be processed via a URL that directly specifies the editing screen, and screen transitions may be performed for subsequent respondents by emulating operations on user interface elements that execute the transition process to the next answer. In this specification, a screen transition method that combines transitions via a URL that directly specifies the editing screen and transitions to the next answer may be referred to as a hybrid transition method.

[0041] The data for reflection referenced by the interface operation means 32 may be obtained by directly referencing the data management unit 10, or it may be obtained by referencing a tabular file 12 obtained by exporting the contents of the data management unit 10. Furthermore, a configuration may be adopted in which the input results to the administrator interface 21 are reflected in the data management unit 10 by a function of the response management platform 20, but the manner of such reflection is not limited to the present invention.

[0042] Figure 2 is a flowchart showing an example of the main processing procedure of the feedback processing system according to this embodiment. The processing shown in Figure 2 mainly consists of the generation, recording, and completion processing of evaluation information and feedback content by the AI ​​processing means 31 (processes S1 to S4), and the reflection to the administrator interface 21 by the interface operation means 32 (processes S5 to S7). The processes from process S1 to process S7 will be described in order below according to Figure 2.

[0043] In step S1, a starting position is specified (optional). That is, if starting position specification information 11, which includes question identification information and respondent identification information, is set externally, the AI ​​processing means 31 sets the processing target position based on the starting position specification information 11 as the position to start the evaluation and feedback content generation process. On the other hand, if the starting position specification information 11 is not set, the processing target position is set so that processing starts from the beginning.

[0044] In step S2, AI processing is performed. The AI ​​processing means 31 acquires the response data stored in the data management unit 10 and generates evaluation information and feedback content for the response data using the generation AI (generation AI service 40). At this time, the AI ​​processing means 31 may extract data columns or data items that contain descriptive answers and exclude selection-type response data from processing by the generation AI (generation AI service 40). Alternatively, batch processing may be performed to send descriptive answers from multiple respondents to the generation AI (generation AI service 40) in a single communication request. Furthermore, the response content from the generation AI (generation AI service 40) may be analyzed, and if a predetermined string indicating an error is included or if it does not conform to a predetermined response format, retries may be automatically performed up to a predetermined number of times. If the response data itself is blank, predetermined feedback content indicating that no response has been given may be used without performing retries. A detailed example of the processing in step S2 will be described later (see Figure 3).

[0045] In process S3, results are recorded. The AI ​​processing means 31 records the evaluation information and feedback content generated in process S2 in the data management unit 10. When recording the feedback content, the feedback content for each of the multiple questions may be structured using tags or delimiters and then aggregated and recorded in a single data field on the data management unit 10.

[0046] As shown in Figure 2, if there are still items to be scored, steps S2 and S3 are repeatedly executed. The items to be scored may include, for example, the questions to be processed, the respondents to be processed, or a combination thereof. This sequentially generates and records evaluation information and feedback content for the response data within a predetermined range.

[0047] In step S4, completion processing is performed. That is, once steps S2 and S3 are completed for all scoring targets, the AI ​​processing means 31 may perform control to clear the start position specification information 11, even if it had been set. This makes it possible to start processing from the beginning the next time it is executed. On the other hand, if processing is interrupted midway, the system may terminate while retaining the start position specification information 11, and then resume processing based on that start position specification information 11 the next time.

[0048] In step S5, the data is reflected in the administrator interface 21. Based on the evaluation information and feedback content recorded in the data management unit 10, the interface operation means 32 automatically performs the input operation of evaluation information and feedback content to the corresponding input area on the administrator interface 21 provided by the response management platform 20, and the operation of saving the input content. At this time, if the feedback content is aggregated in a single data field in the data management unit 10, the interface operation means 32 may analyze the tags or delimiters from the aggregated data to extract the feedback content for each question, and individually map and input it into the feedback input field corresponding to each question on the administrator interface 21. Furthermore, the data for reflection may be referenced by directly referencing the data management unit 10, or by referring to the tabular file 12 exported from the contents of the data management unit 10. A detailed example of the processing in step S5 will be described later (see Figure 4).

[0049] In step S6, standby control is performed. After executing a save operation or screen transition, the interface operation means 32 waits for the next operation until it detects the appearance of an inputtable user interface element or the occurrence of a reflection completion state indicating that the input result has been reflected in the response management platform 20. This enables stable continuous processing according to the state of the administrator interface 21.

[0050] In step S7, a transition occurs. When processing the reflection process for multiple respondents consecutively, the interface operation means 32 may be configured to open the administrator interface 21 using a URL that directly specifies the editing screen for the first respondent to be processed, and for the second and subsequent respondents, to transition to the editing screen of the next respondent by emulating operations on user interface elements in the administrator interface 21 that cause the transition process to the next response to be executed.

[0051] As shown in Figure 2, if there are still items to be reflected, steps S5 to S7 are repeatedly executed to complete the input and saving process for all items to be reflected. Once processing for all items to be reflected is complete, this main process ends. Figure 3 is a flowchart detailing the process corresponding to steps S1 to S4 in Figure 2, and Figure 4 is a flowchart detailing the process corresponding to steps S5 to S7 in Figure 2.

[0052] Figure 3 is a flowchart illustrating an example of the processing procedure of the AI ​​processing means 31, showing in detail the processes corresponding to steps S1 to S4 in Figure 2 (from specifying the starting position to generating and recording evaluation information and feedback content, and to the completion process). The processes from steps S31 to S36 will be described in order below, following Figure 3.

[0053] In step S31, the starting position is set (optional). That is, if the AI ​​processing means 31 has set the starting position information 11 (including question identification information and respondent identification information) from an external source, it sets the processing target position based on the starting position information 11. On the other hand, if the starting position information 11 has not been set, it sets the processing target position so that processing starts from the beginning. This makes it possible to resume processing based on the starting position information 11 even if the processing is interrupted midway.

[0054] In step S32, target extraction and batching are performed. The AI ​​processing means 31 identifies and extracts data columns or data items, etc. that contain descriptive answers from the response data stored in the data management unit 10, and excludes the selection-type response data from processing by the generation AI (generation AI service 40). Such processing can be performed by the aforementioned target identification function. The method for identifying data columns or data items, etc. that contain descriptive answers can be any method based on column attributes, column names, setting information, etc.

[0055] Furthermore, the AI ​​processing means 31 may perform batch processing on the extracted descriptive answers, sending the answer data for multiple respondents in a single communication request to the generating AI (generating AI service 40). Batch processing reduces the number of communications to the generating AI (generating AI service 40), contributing to improved processing efficiency. The batch size may be fixed or variable and can be set appropriately according to the number of items to be processed, the number of questions, the communication environment, etc.

[0056] In step S33, generation AI processing is performed. The AI ​​processing means 31 sends the batch-unit input (including descriptive answers) created in step S32 to the generation AI (generation AI service 40) and receives evaluation information and feedback content corresponding to each answer data included in the batch. If the answer data is blank, the generation AI (generation AI service 40) may output predetermined feedback content indicating that the answer is not answered without performing any processing. As an example of an input format to the generation AI (generation AI service 40), the AI ​​processing means 31 sends a data sequence (text sequence or structured data in a predetermined format) combining question identification information and descriptive answers to the question to the generation AI (generation AI service 40) in batch units, and in response to this, the generation AI (generation AI service 40) receives data in which evaluation information and feedback content for each question are stored in a predetermined item structure. However, the specific input format and response format can be set as appropriate depending on the implementation.

[0057] The AI ​​processing means 31 analyzes the response from the generated AI (generated AI service 40) obtained in step S33. If the response contains a predetermined string indicating an error, or if it does not conform to a predetermined response format, the AI ​​processing means 31 determines that the response is not normal. This determination corresponds to the conditional determination at the determination node D1 ("Response Normal?") shown in Figure 3. The suitability of the response format may be determined by any rules defined in the implementation, such as whether the evaluation information and feedback content are obtained in a predetermined item structure.

[0058] In step S34, a retry is performed. That is, if the response is determined to be abnormal, the AI ​​processing means 31 re-executes the generation AI (generation AI service 40) processing related to the question up to a predetermined number of times. If the number of retries exceeds the predetermined number, the processing may be interrupted and terminated. In this case, the start position specification information 11 may be retained so that the processing can be resumed next time based on the start position specification information 11. This ensures operational continuity even if a temporary malfunction occurs on the generation AI (generation AI service 40) side.

[0059] In step S35, results are recorded. The AI ​​processing unit 31 records the evaluation information and feedback content obtained as normal responses in the data management unit 10. The evaluation information may be recorded as scores for each question, or as evaluations for each respondent.

[0060] When recording the feedback content, the feedback content for each of the multiple questions may be structured using predetermined tags or delimiters and recorded in a single data field on the data management unit 10. Such processing corresponds to the structured recording processing described above. For example, the feedback content for multiple questions may be recorded as a string by concatenating them using tags corresponding to the question identifiers, but the specific format of the tags or delimiters is not limited to the present invention.

[0061] In step S36, the completion process is performed. That is, if there are still items to be scored (the questions and respondents to be processed, or a combination thereof), the processes from step S32 onwards are repeated, and once the generation and recording of evaluation information and feedback content is complete for all items to be scored, the start position specification information 11 may be cleared even if it was set. This makes it possible to start processing from the beginning in the next execution.

[0062] Figure 4 is a flowchart illustrating an example of the processing procedure of the interface operation means 32, focusing on steps S5 (reflection to the administrator interface 21), S6 (standby control), and S7 (transition) in Figure 2. The processes from steps S41 to S47 will be described in order below, following Figure 4.

[0063] In step S41, data for reflection is acquired. Specifically, the interface operation means 32 refers to the evaluation information and feedback content recorded in the data management unit 10 and acquires the data to be reflected in the administrator interface 21. The data for reflection may be acquired by directly referring to the data management unit 10, or by referring to the tabular file 12 obtained by exporting the contents of the data management unit 10.

[0064] In step S42, the administrator interface is displayed. When processing multiple respondents sequentially, the interface operation means 32 may open the administrator interface 21 for the first respondent to be processed using a URL that directly specifies the respondent's editing screen.

[0065] In step S43, tag analysis and question-specific extraction are performed. That is, if the feedback content is aggregated in a single data field in the data management unit 10, the interface operation means 32 analyzes tags or delimiters from the aggregated feedback content and extracts the feedback content for each question. This generates data for mapping to input fields on a question-by-question basis. Alternatively, the feedback content may be recorded individually as fields for each question, in which case step S43 may be replaced with a reference to those individual fields.

[0066] In step S44, input and saving are performed. Based on the acquired evaluation information and feedback content, the interface operation means 32 performs an input operation on the corresponding input area on the administrator interface 21, and then performs a save operation to finalize the input content. The areas to be input may include the evaluation information input area and the feedback input field.

[0067] In step S45, standby control (save and reflect) is performed. After executing the save operation, the interface operation means 32 waits for the next operation until it detects that a reflection completion state has occurred, indicating that the input result has been reflected in the response management platform 20. The reflection completion state can take any form, such as a display message, a change in status display, or a change in the state of the input field. This allows the system to proceed to the next step after the screen state has stabilized, potentially improving the stability of continuous processing.

[0068] In step S46, a transition process to the next answer is performed. That is, if there is a next respondent, the interface operation means 32 performs the transition process to the next answer, which is provided in the administrator interface 21 to display the editing screen for the next respondent, by emulating an operation on the user interface element that executes the said transition process. This transition is not limited to pressing a button, but may also be in the form of selecting a link, operating a menu, etc.

[0069] In step S47, a waiting control (transition completion) is performed. After the screen transition is executed, the interface operation means 32 waits until it detects that a state indicating the completion of the transition has occurred, such as the appearance of an inputtable user interface element on the destination screen. This ensures that the input and saving process for the next respondent is reliably executed.

[0070] By repeating steps S41 to S47 as long as there are remaining items to be reflected, the process of reflecting evaluation information and feedback content for multiple respondents is executed sequentially. The interface operation means 32 is composed of RPA software, browser extensions, or script programs, and can perform the aforementioned input, saving, transitions, etc., by mimicking human browser operations.

[0071] Although embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications are possible without departing from the spirit of the invention as described in the claims.

[0072] The response management platform 20 is not limited to a specific service and may be any platform that allows viewing and editing of response data via a web interface. For example, it can be applied to form services, learning management systems (LMS), in-house examination systems, etc.

[0073] The data management unit 10 is not limited to a spreadsheet; it may be a database, a file, or a combination thereof. The manner in which the response data, evaluation information, and feedback content are stored (row / column structure, record structure, etc.) is arbitrary, as long as it can be managed in association with the respondent and the question.

[0074] The generating AI (generating AI service 40) is not limited to a specific AI service or model, and any generating AI capable of generating evaluation information and feedback content can be used. The method of calling the generating AI (generating AI service 40) is not limited to API communication, and may also be a configuration via a proxy server, etc.

[0075] The method for identifying data columns containing descriptive answers can be any method based on column names, column attributes, configuration information, metadata, etc. The batch size can be fixed or variable and may be set appropriately depending on the number of items to be processed, the network environment, and the constraints on the generation AI (generation AI service 40).

[0076] The format of tags or delimiters used to structure feedback content for multiple questions is arbitrary. Aggregation into a single data field is not limited to a single cell; it can be achieved using any storage unit, such as a single record attribute or a single area within a single file. Alternatively, feedback content may be recorded in individual fields for each question without aggregation.

[0077] Error detection is not limited to the presence or absence of an error string; it may be performed based on any criteria, such as the number of response items, format consistency, or timeout. The upper limit for the number of retries is arbitrary and is not limited to a configuration that interrupts processing when the limit is exceeded. The method of resuming processing based on the start position specification information 11 is not limited to the format of the question identification information and respondent identification information (number, ID, key, etc.), but can be implemented with any information that can identify the processing target position.

[0078] The interface operation means 32 may be RPA software, a browser extension, a script program, etc., and can be implemented individually or in combination. Operations on the administrator interface 21 are not limited to button presses, but may be any form such as link selection, menu operation, shortcut operation, etc.

[0079] The completion status, which is the target of detection in the standby control, can take any form, such as a display message, a popup, the appearance of an input field after a screen transition, or a change in the status display. The detection method can also be any method, such as acquiring screen elements, monitoring the DOM state, image recognition, or event hooks.

[0080] The data for reflection referenced by the interface operation means 32 may be obtained by directly referencing the data management unit 10, or by referencing a tabular file 12 obtained by exporting the contents of the data management unit 10. The format of the tabular file 12 is arbitrary.

[0081] The response management platform 20 may be configured to automatically reflect the input results to the administrator interface 21 in the data management unit 10 through its own functions. For example, by using a form service such as Google Forms as the response management platform 20 and associating the form with the data management unit 10, which consists of a cloud-based spreadsheet, it is possible to achieve a so-called round-trip synchronization configuration between the form and the spreadsheet, in which evaluation information and feedback content entered by the interface operation means 32 on the administrator interface 21 are synchronized with the data management unit 10 in conjunction with updates to the form. Whether or not such reflection occurs and the manner of reflection are not limiting to the present invention. [Industrial applicability]

[0082] This invention can be used to automate the generation and return of evaluation information and feedback content in education, training, qualification examinations, etc. [Explanation of Symbols]

[0083] 10: Data Management Department 11:Start position specification information 12: Tabular files 20: Answer Management Platform 21: Administrator Interface 30: Feedback processing system (main unit) 31: AI Processing Methods 32: Interface operation means 40: Generative AI services

Claims

1. The data management department manages the response data collected from multiple respondents, A response management platform that allows viewing and editing of the aforementioned response data via a web interface, AI processing means that, with respect to the response data stored in the data management unit, generates evaluation information indicating the evaluation result corresponding to the response data and feedback content which is information presented to the respondent corresponding to the response data in relation to the evaluation information, and records the generated evaluation information and feedback content in the data management unit. An interface operation means that automatically performs an input operation to input the evaluation information and feedback content into the corresponding input area on the administrator interface provided by the response management platform, based on the evaluation information and feedback content recorded in the data management unit, and a save operation to save the evaluation information and feedback content entered by the input operation, A feedback processing system characterized by comprising the following:

2. In the feedback processing system according to claim 1, The AI ​​processing means has a target identification function that identifies and extracts only data columns or data items containing descriptive answers from the response data stored in the data management unit, and performs batch processing that sends the descriptive answers for multiple respondents included in the identified data columns or data items to the generating AI in a single communication request, while excluding multiple-choice response data from the processing target of the generating AI.

3. In the feedback processing system according to claim 1, The AI ​​processing means is characterized by performing structured recording processing, which structures the feedback content for each of the multiple questions for a single respondent using predetermined tags or delimiters, and aggregates and records it in a single data field on the data management unit.

4. In the feedback processing system described in claim 3, The feedback processing system is characterized in that the interface operation means has the function of analyzing the tags or delimiters from the data aggregated in the single data field to extract the feedback content for each question, and individually mapping and inputting it into the feedback input field corresponding to each question on the administrator interface.

5. In the feedback processing system according to claim 1, When the interface operation means performs sequential processing on multiple respondents, for the first respondent to be processed, it opens the administrator interface using a URL that directly specifies the respondent's editing screen, A feedback processing system characterized by using a hybrid transition method for screen transitions for the second and subsequent respondents, by emulating operations on user interface elements that perform transition processing to the next response, which are provided in the administrator interface to display the editing screen for the next respondent.

6. In the feedback processing system according to claim 1, The AI ​​processing means can be configured externally to set a starting position specification information that includes at least question identification information that identifies the question to be processed and respondent identification information that identifies the respondent to be processed, as the starting position for starting the evaluation and feedback content generation process for the response data. A feedback processing system characterized by having a control function that starts the generation process based on the start position specification information, and restarts the generation process based on the start position specification information if the process is interrupted midway.

7. In the feedback processing system according to claim 1, The AI ​​processing means analyzes the response content from the generated AI and, if it contains a predetermined string indicating an error or does not conform to a predetermined response format, performs retry control to automatically retry the question corresponding to the response content. A feedback processing system characterized by having a determination logic that outputs predetermined feedback content indicating that no response was given without performing the retry if the response data itself is blank.

8. In the feedback processing system according to claim 1, The interface operation means is configured using at least one of RPA (Robotic Process Automation) software, a browser extension, or a script program, and enables the input and saving of the evaluation information and feedback content to the administrator interface by mimicking human browser operations. A feedback processing system characterized by having a waiting control function that, when a screen transition or save operation is performed on the administrator interface, waits for the next operation until it detects the appearance of an inputtable user interface element on the destination screen, or the occurrence of a reflection completion state indicating that the input results of the evaluation information and the feedback content have been reflected in the response management platform.

Images