Information processing method, computer program, and information processing device

Abstract

An information processing method, a computer program, and an information processing device are provided that are expected to assist users in debugging application programs. [Solution] In the information processing method of this embodiment, an information processing device acquires trace information and log information related to the execution of an application program to be tested, acquires the source code of the application program, acquires investigation content information indicating the content to be investigated for an error that occurs during the execution of the application program, inputs the acquired trace information, log information, source code and investigation content information into a language model, acquires investigation result information output by the language model, and outputs the acquired investigation result information.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing method, a computer program, and an information processing device. [Background technology]

[0002] Patent Document 1 proposes a debugging support device that collects test case data in which information about the execution state of an application program and expected values ​​for the execution results of each process are set, reproduces the execution state of each process of the application program based on the test case data, and determines whether or not there is a defect in each process of the application program based on the results of comparing the execution results of the application program processes with the expected values. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-177272 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides an information processing method, a computer program, and an information processing device that are expected to assist users in debugging application programs. [Means for solving the problem]

[0005] In one embodiment of an information processing method, an information processing device acquires trace information and log information related to the execution of an application program to be tested, acquires the source code of the application program, acquires investigation content information indicating the content to be investigated for an error that occurs during the execution of the application program, inputs the acquired trace information, log information, source code, and investigation content information into a language model, acquires investigation result information output by the language model, and outputs the acquired investigation result information. [Effects of the Invention]

[0006] The present disclosure is expected to assist users in debugging application programs. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a schematic diagram illustrating an overview of an information processing system according to an embodiment of the present invention. [Figure 2] 1 is a block diagram showing an example of a configuration of a debugging server device according to an embodiment of the present invention; [Figure 3] FIG. 10 is a schematic diagram showing an example of a display of survey results on a user terminal device. [Figure 4] FIG. 2 is a block diagram showing an example of the configuration of a user terminal device according to the present embodiment. [Figure 5] FIG. 2 is a block diagram showing an example of the configuration of a testing server device according to the present embodiment. [Figure 6] 10 is a flowchart showing an example of a procedure for information collection processing performed by the debugging server device according to the present embodiment. [Figure 7] 10 is a flowchart showing an example of a procedure of an investigation process performed by the debugging server device according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Specific examples of information processing systems according to embodiments of the present disclosure will be described below with reference to the drawings. Note that the present disclosure is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope of the claims.

[0009] <System Overview> FIG. 1 is a schematic diagram for explaining an overview of an information processing system according to this embodiment. The information processing system according to this embodiment is configured to include, for example, a user terminal device 1, a testing server device 2, and a debugging server device 3. The user terminal device 1 is an information processing device such as a PC (personal computer), a smartphone, or a tablet terminal device, and is a device on which a user performs tasks such as creating and debugging an application program. The testing server device 2 is an information processing device for executing an application program 21 to be tested. The debugging server device 3 is an information processing device for assisting a user in debugging the application program 21, and performs processing such as identifying the cause of an error using an LLM (Large Language Model) 41 based on information collected by executing the application program 21.

[0010] In this embodiment, the application program 21 to be tested is assumed to be a program that provides a service that a user uses via the browser 11. Such an application program 21 includes, for example, a front-end program and a back-end program. The front-end program is a program that provides a user interface for exchanging information with the user, displays information such as text and images on the browser 11, and accepts input operations by the user. The back-end program is a program that processes data related to the service provided to the user and links with external services.

[0011] The user terminal device 1 according to this embodiment includes a test execution unit 12. When the application program 21 is executed, the test execution unit 12 is interposed between the browser 11 and the front-end program of the application program 21, and thereby acquires information input to the application program 21 and information output from the application program 21, i.e., information on test cases for the application program 21 to be tested, as trace information. The test execution unit 12 assigns a trace ID that identifies the test case to the acquired trace information and transmits it to the debugging server device 3.

[0012] The testing server device 2 according to this embodiment includes an information collection unit 22. The information collection unit 22 collects trace information and log information in conjunction with the execution of the application program 21 and transmits the information to the debugging server device 3. The information collection unit 22 may collect information on both the front-end program and the back-end program of the application program 21, or on only either the front-end program or the back-end program.

[0013] The trace information collected by the information collection unit 22 may include, for example, information such as the function name of a function executed in the application program 21, the value of an argument passed to this function, the return value of this function, the execution start time or execution end time of this function, etc. The trace information may also include, for example, information such as values ​​stored in variables when the application program 21 is being executed, data stored in memory, registers, stacks, etc., or data sent and received in communication with the outside. However, the trace information may include various information other than the above information.

[0014] The log information collected by the information collecting unit 22 is information that is output (displayed or output as a file, etc.) to notify the user of the details of an error, etc., when an error, etc. occurs during execution of the application program 21. The log information may be, for example, information output by the execution environment of the application program 21, or may be, for example, information output by the application program 21 itself.

[0015] The debugging server device 3 according to this embodiment is a device that accumulates information about the application program 21 to be tested and investigates the cause of an error based on the accumulated information using an LLM 41. The debugging server device 3 has three databases for accumulating information: a trace DB (database) 42, a source DB 43, and an overview DB 44. The trace DB 42 stores trace information transmitted from the test execution unit 12 of the user terminal device 1, and trace information and log information transmitted from the information collection unit 22 of the testing server device 2. The source DB 43 stores the source code (source program) of the application program 21 to be tested. In this embodiment, a user performing debugging work uploads the source code from the user terminal device 1 to the debugging server device 3 in advance. The overview DB 44 stores overview information of a program generated based on the source code of the application program 21 to be tested.

[0016] The debugging server device 3 according to this embodiment also includes an information storage unit 31, an outline generation unit 32, a trace reference unit 33, an outline reference unit 34, and a source reference unit 35. The information storage unit 31 communicates with the user terminal device 1 and the testing server device 2 to acquire trace information collected by the user terminal device 1 and trace information and log information collected by the testing server device 2, and stores and accumulates the acquired information in a trace DB 42. The information storage unit 31 acquires trace information from the user terminal device 1 when the application program 21 is executed according to a predetermined test case, the trace information being assigned a trace ID that identifies the test case, and the information storage unit 31 acquires trace information and log information collected by the testing server device 2 in conjunction with the execution of the application program 21 according to the test case, and stores the information in the trace DB 42 in association with the trace ID. This allows the trace information and log information related to a test case in which an error has occurred to be referenced based on the trace ID assigned to the test case.

[0017] Based on the source code of the application program 21 to be tested stored in the source DB 43, the summary generation unit 32 generates summary information, such as information about the processing performed by the source code. In this embodiment, the summary generation unit 32 generates summary information about the source code using an LLM. The summary generation unit 32 stores the generated summary information in the summary DB 44. In this embodiment, the summary DB 44 stores vectorized summary information. The summary generation unit 32 vectorizes the generated summary information using an encoder that converts text information into vector information. The generation of summary information and vectorization of summary information by the summary generation unit 32 may be performed using the LLM 41 used to investigate the cause of an error, or an LLM, encoder, or the like different from the LLM 41 may be used.

[0018] In this embodiment, the debugging server device 3 generates summary information from source code using the LLM 41, but this is not limiting. The debugging server device 3 may generate summary information using, for example, a learning model other than the LLM, or may generate summary information from source code using, for example, rule-based processing. Alternatively, for example, the debugging server device 3 may not create summary information, but may instead accept input of summary information created by a user and store it in the summary DB 44. Summary information may be created by any other method.

[0019] The trace reference unit 33 is interposed between the LLM 41 and the trace DB 42, and performs processing to acquire trace information and log information required for investigation by the LLM 41 from the trace DB 42. The trace reference unit 33 refers to the trace DB 42 based on the trace ID of the test case in which an error occurred, reads out trace information and log information for investigating the cause of the error, and provides the information to the LLM 41.

[0020] The overview reference unit 34 is interposed between the LLM 41 and the overview DB 44, and performs processing to acquire overview information required for investigation by the LLM 41 from the overview DB 44. For example, based on the trace information and log information acquired by the trace reference unit 33, the overview reference unit 34 reads overview information of the related source code from the overview DB 44 and provides it to the LLM 41.

[0021] The source reference unit 35 is interposed between the LLM 41 and the source DB 43, and performs processing to acquire source code required for investigation by the LLM 41 from the source DB 43. The source reference unit 35 reads relevant source code from the source DB 43 based on, for example, trace information and log information acquired by the trace reference unit 33, and provides the source code to the LLM 41. Note that the source reference unit 35 does not read all of the source code of the application program 21 to be tested from the source DB 43, but only the part related to the error, and provides the part to the LLM 41. At this time, the source reference unit 35 can use the LLM 41 to determine which source code is related to the error based on the summary information acquired by the summary reference unit 34 from the summary DB 44, and acquire the source code determined to be related from the source DB 43.

[0022] The LLM 41 may employ a learning model such as a Transformer with an attention mechanism in a large-scale neural network, a Bidirectional Encoder Representations from Transformers (BERT), or a Generative Pre-trained Transformer (GPT). The LLM 41 used by the information processing system according to this embodiment may be a widely published general-purpose large-scale language model, or may be one that has been trained on information related to program debugging through fine tuning.

[0023] In this embodiment, the debugging server device 3 is configured to include the LLM 41, but this is not limiting. The LLM 41 may operate on a server device or the like different from the debugging server device 3. The debugging server device 3 can perform processing using the LLM 41 included in the other server device in the same way as if it itself were equipped with the LLM 41, by transmitting and receiving data to and from the other server device equipped with the LLM 41 using, for example, an API (Application Programming Interface) for incorporating the LLM 41 into an application program.

[0024] The debugging server device 3 according to this embodiment investigates the cause of an error, for example, when an error occurs during execution of the application program 21 to be tested, using the LLM 41 based on information stored in the database. The debugging server device 3 acquires trace information and log information from the trace DB 42 based on, for example, the trace ID of the test case in which the error occurred, and acquires related source code and its summary information from the source DB 43 and summary DB 44 based on the trace information and log information. The debugging server device 3 also acquires investigation content information indicating the type of investigation to be performed on the error from a user, for example, via the user terminal device 1. The debugging server device 3 generates a prompt including information such as the trace ID of the test case in which the error occurred and the investigation content information acquired from the user, as well as an instruction to investigate the cause of the error in the application program 21, and inputs the prompt along with the information acquired from the database to the LLM 41. In response to the received prompt, the LLM 41 outputs text information including the results of the investigation into the cause of the error in the application program 21 based on the trace information, log information, source code, and summary information acquired from the database. The debugging server device 3 acquires the text information output by the LLM 41 and transmits the acquired text information as the error investigation results to the user terminal device 1. The user terminal device 1 displays the text information of the investigation results received from the debugging server device 3, and presents the investigation results of the cause of the error to the user.

[0025] In FIG. 1, the information processing system includes three devices: a user terminal device 1, a testing server device 2, and a debugging server device 3. However, the present invention is not limited to this configuration. The information processing system may include two or fewer devices, or may include four or more devices. The example shown in FIG. 1 assumes that a user uses a service provided by an application program 21 via a browser 11. However, if the application program 21 is executed on the user terminal device 1, the functions of the user terminal device 1 and the testing server device 2 shown in FIG. 1 can be integrated into a single device. For example, the execution of the application program 21 to be tested and the investigation process by the LLM 41 may be performed on a single server device. For example, the process of storing information in a database by the debugging server device 3 and the investigation process by the LLM 41 may be performed on separate server devices. Alternatively, for example, the three databases may each be maintained on a separate server device.

[0026] <Device configuration> 2 is a block diagram showing an example of the configuration of the debugging server device 3 according to this embodiment. The debugging server device 3 according to this embodiment can be realized by installing a computer program according to this embodiment in an information processing device such as a general-purpose server computer. The debugging server device 3 according to this embodiment is configured to include a processing unit 311, a storage unit 312, and a communication unit 313.

[0027] The processing unit 311 is configured using an arithmetic processing device such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), a GPU (Graphics Processing Unit), an NPU (Neural network Processing Unit), or a quantum processor, and storage devices such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The processing unit 311 reads and executes a program 312a stored in the storage unit 312, thereby performing various processes such as collecting trace information and log information of the application program 21 to be tested and investigating the cause of an error based on the collected information.

[0028] The storage unit 312 is configured using a large-capacity storage device such as a hard disk or an SSD (Solid State Drive). The storage unit 312 stores various programs executed by the processing unit 311 and various data required for the processing of the processing unit 311. In this embodiment, the storage unit 312 stores a program 312a executed by the processing unit 311. The storage unit 312 also has a model information storage unit 312b that stores information about a trained learning model that has been trained in advance by machine learning, i.e., the LLM 41, and three databases, namely the trace DB 42, source DB 43, and summary DB 44 described above.

[0029] In this embodiment, the program (computer program, program product) 312a is provided in a form recorded on a recording medium 99 such as a memory card or an optical disk, and the debugging server device 3 reads the program 312a from the recording medium 99 and stores it in the storage unit 312. However, the program 312a may also be written to the storage unit 312, for example, during the manufacturing stage of the debugging server device 3. Alternatively, the program 312a may be distributed by a remote server device or the like and acquired by the debugging server device 3 via communication. For example, the program 312a may be recorded on the recording medium 99 and read by a writing device and written into the storage unit 312 of the debugging server device 3. The program 312a may be provided in a form distributed via a network or in a form recorded on the recording medium 99.

[0030] The model information storage unit 312b stores information about various learning models generated in advance. The information about the learning models may include, for example, information indicating the configuration of the learning model and information such as predetermined internal parameter values. In this embodiment, the model information storage unit 12b stores information about the model configuration and parameters for the LLM 41.

[0031] The trace DB 42 stores trace information and log information obtained during the execution of the application program 21 under test in association with a trace ID. The source DB 43 stores the source code of the application program 21 under test. The summary DB 44 stores vectorized summary information generated by the LLM 41 or the like based on the source code of the application program 21 under test.

[0032] The communication unit 313 communicates with other devices via a network N such as a LAN (Local Area Network) or the Internet. In this embodiment, the communication unit 313 communicates between the user terminal device 1 and the testing server device 2. The communication unit 313 receives, for example, trace information transmitted from the user terminal device 1 and trace information and log information transmitted from the testing server device 2, and provides these to the processing unit 311. The communication unit 313 also transmits the investigation results provided by the processing unit 311 to the user terminal device 1.

[0033] The storage unit 312 may be an external storage device connected to the debugging server device 3. The debugging server device 3 may be a multi-computer including multiple computers, or may be a virtual machine virtually constructed by software. The debugging server device 3 is not limited to the above configuration, and may include, for example, a reading unit that reads information stored in a portable storage medium, an input unit that accepts operation input, or a display unit that displays images.

[0034] In the debugging server device 3 according to this embodiment, the processing unit 311 reads and executes the program 312a stored in the storage unit 312, whereby the information accumulation unit 31, the outline generation unit 32, the trace reference unit 33, the outline reference unit 34, the source reference unit 35, the investigation processing unit 36, and the like are realized as software functional units in the processing unit 311. Note that in the figure, functional units related to the process of collecting trace information and log information of the application program 21 to be tested and the process of investigating the cause of an error based on the collected information are shown as functional units of the processing unit 311, and functional units related to other processes are not shown.

[0035] The information storage unit 31 communicates with the user terminal device 1 via the communication unit 313, receives trace information sent by the user terminal device 1, and stores the received trace information in the trace DB 42 in association with a trace ID. The information storage unit 31 also communicates with the testing server device 2 via the communication unit 313, receives trace information and log information sent by the testing server device 2, and stores the received trace information and log information in the trace DB 42 in association with a trace ID.

[0036] The trace ID is assigned to the trace information by the user terminal device 1. The trace ID generated by the user terminal device 1 may be associated with the trace information and log information transmitted by the testing server device 2 by the testing server device 2 or by the debugging server device 3. For example, the user terminal device 1 may transmit a trace ID to the testing server device 2, and the testing server device 2 may attach this trace ID to the trace information and log information and transmit them to the debugging server device 3. Furthermore, for example, the debugging server device 3 may associate the trace ID based on the timing at which it receives the trace information from the user terminal device 1 and the timing at which it receives the trace information and log information from the testing server device 2.

[0037] For example, when a user uploads source code of a test application program 21 and stores it in the source DB 43, the summary generator 32 inputs the source code to the LLM 41 and provides the LLM 41 with a prompt including an instruction to generate a sentence explaining the summary of the source code. The summary generator 32 acquires summary information output by the LLM 41 in response to the prompt, associates the acquired summary information with information such as the name or identification information of the application program 21, and stores the associated information in the summary DB 44. In this embodiment, the summary generator 32 vectorizes the summary information generated by the LLM 41 using a pre-prepared encoder or the like, and stores the vectorized summary information in the summary DB 44. Note that the process of generating summary information from source code is not limited to the method performed using the LLM 41. For example, the summary generator 32 may generate summary information using various methods, such as automatically generating summary information using pre-defined rules or templates.

[0038] For example, the summary generator 32 can generate the following summary information: " File name: SampleController.py ·overview: The role of the file: It is a controller that oversees operations and executes various actions (especially sending notification emails) based on changes in the state of the target data. File processing details: Within the method, when the status changes to "xxx", the corresponding information is obtained and a notification email is sent. "

[0039] The above summary information is an example of summary information output by the LLM 41. In this embodiment, the summary information is information expressed in a language that the user can understand, such as Japanese or English. However, the summary generation unit 32 stores information obtained by vectorizing summary information such as Japanese in the summary DB 44. Note that the summary DB 44 may store both the vectorized summary information and the summary information before vectorization. The above summary information is an example and is not limited to this.

[0040] The trace reference unit 33 acquires the trace ID of a test case in which an error occurred during execution of the application program 21 to be tested, and acquires trace information and log information by referring to the trace DB 42 based on the acquired trace ID. The overview reference unit 34 acquires summary information of the related source code from the overview DB 44 based on the trace information and log information acquired by the trace reference unit 33. In this embodiment, the overview DB 44 stores vectorized summary information, and the overview reference unit 34 can acquire desired summary information by vectorizing the trace information and log information and calculating the similarity with the vector information stored in the overview DB 44. The source reference unit 35 acquires related source code from the source DB 43 based on the trace information and log information acquired by the trace reference unit 33 or the summary information acquired by the overview reference unit 34.

[0041] The investigation processing unit 36 ​​investigates the cause of the error using the LLM 41, based on the trace information and log information acquired by the trace reference unit 33, the summary information acquired by the summary reference unit 34, and the source code acquired by the source reference unit 35. In this embodiment, if an error occurs during execution of the application program 21 to be tested, the user terminal device 1 accepts input of investigation details related to the error from the user performing debugging, and transmits the accepted investigation details to the debugging server device 3. The investigation processing unit 36 ​​generates a prompt including a command to investigate the error in the application program 21 to be tested, based on the investigation details received from the user terminal device 1 and the trace ID of the test case in which the error occurred.

[0042] For example, the survey processor 36 may generate the following prompt: " You will be given the trace ID where the error occurred and the details you want to investigate. Based on these details, check the code where the problem occurred and investigate the cause. Trace ID: {trace_id} Investigation details: When I pressed the update button, the message "Update failed" was displayed. Please investigate the reason. "

[0043] The above prompt assumes a situation in which the application program 21 under test is a Web application with a user registration function, login function, etc., and an error such as "User information update failed" occurs in response to the input of a test case. The first sentence of the above prompt, "You will be given the ID of the trace where the error occurred and the content you wish to investigate. Based on this content, please check the code where the problem occurred and investigate the cause," is a predetermined command statement. The second sentence, "Trace ID: {trace_id}," has the actual trace ID inserted in the "{trace_id}" part. The third sentence, "Investigation content: When I pressed the update button, the message "Update failed" was displayed. Please investigate the reason." is an example of an investigation content entered by the user. The investigation processing unit 36 ​​can generate a prompt by inserting the trace ID, investigation content information, etc. into a prompt template containing a predetermined command statement. Note that the above prompt is merely an example and is not limited to this.

[0044] In this embodiment, a configuration is adopted in which the user inputs the details of an investigation into an error, but the present invention is not limited to this. For example, the debugging server device 3 can use the LLM 41 to generate a statement of the investigation details based on log information output when an error occurs. Also, for example, if the log information includes information such as an error code indicating the type of error, the debugging server device 3 may generate a prompt with a statement of the investigation details predetermined for the error code as the investigation details.

[0045] The investigation processing unit 36 ​​configures the LLM 41 based on the information stored in the model information storage unit 312b, and inputs the generated prompt, as well as the trace information, log information, summary information, and source code acquired from the database, to the LLM 41. The LLM 41 investigates the cause of the error based on this input information and outputs text information that is the investigation result. The investigation processing unit 36 ​​acquires the investigation result output by the LLM 41 and transmits the acquired investigation result to the user terminal device 1. This allows the user terminal device 1 to display the investigation result of the cause of the error that occurred during execution of the application program 21 to be tested and provide it to the user.

[0046] FIG. 3 is a schematic diagram showing an example of an investigation result display by the user terminal device 1. The illustrated investigation result display screen has an investigation content input area at the top of the screen where the user can input investigation content, and an investigation result display area below that where the investigation results are displayed. The investigation content input area includes, for example, a message such as "Please enter investigation content," a text box for entering investigation content, and a button labeled "Start Investigation." The user can start investigating the cause of the error by using the keyboard to enter the investigation content in the text box and, for example, clicking the "Start Investigation" button with the mouse. The user terminal device 1 acquires the entered investigation content and transmits it to the debugging server device 3 as investigation content information. In response, the debugging server device 3 begins investigating the cause of the error. At this time, the user terminal device 1 may transmit the trace ID of the test case in which the error occurred along with the investigation content information to the debugging server device 3.

[0047] The investigation result display area displays the text of the investigation result that the user terminal device 1 received from the debugging server device 3. In this example, the following investigation result text is displayed. Note that the investigation result below is an example and is not limited to this. " When I checked the error, I found that an exception had occurred in the `sample` method of `SampleController`. Specifically, when sending an email, the error "An email must have a "To", "Cc", or "Bcc" header." This means that the recipients (To, Cc, Bcc) were not specified when sending the email. The code where the error occurs is: if ($status === 'sample_status') { Mail::to($sample->email)->send(new SampleMail($sample->login_id, $sample->name, $sample->email)); / / ... } This code attempts to send an email to notify users that the status has changed to "sample_status". However, if `$sample->email` is empty or does not have the correct email address format, the above error will occur. "

[0048] 4 is a block diagram showing an example of the configuration of a user terminal device 1 according to this embodiment. The user terminal device 1 according to this embodiment can be realized by installing a predetermined program in a general-purpose information processing device such as a personal computer, a smartphone, or a tablet terminal device. The user terminal device 1 according to this embodiment is configured to include a processing unit 111, a storage unit 112, a communication unit 113, a display unit 114, an operation unit 115, etc.

[0049] The processing unit 111 is configured using an arithmetic processing unit such as a CPU or an MPU, a ROM, and the like. The processing unit 111 reads and executes a program 112a stored in the storage unit 112 to perform various processes, such as inputting and outputting information to and from the application program 21 under test and collecting the input and output information. The storage unit 112 is configured using a large-capacity storage device, such as a hard disk or SSD. The storage unit 112 stores the program 112a executed by the processing unit 111, as well as the program for the browser 11 and source code 112c of the application program 21 created by the user. In this embodiment, the program (computer program, program product) 112a is provided in a form recorded on a storage medium 98 such as a memory card or an optical disc, and the user terminal device 1 reads the program 112a from the storage medium 98 and stores it in the storage unit 112. However, the program 112a may also be provided in a form distributed via a network.

[0050] The communication unit 113 communicates with various devices such as the testing server device 2 and the debugging server device 3 via a network N such as a LAN or the Internet. The display unit 114 is configured using a liquid crystal display or the like, and displays various images, characters, etc. based on the processing of the processing unit 111. The operation unit 115 accepts user operations and notifies the processing unit 111 of the accepted operations. For example, the operation unit 115 accepts user operations using an input device such as a mechanical button or a touch panel provided on the surface of the display unit 114. Furthermore, for example, the operation unit 115 may be an input device such as a mouse and a keyboard, and these input devices may be configured to be detachable from the user terminal device 1.

[0051] Furthermore, in the user terminal device 1 according to this embodiment, the processing unit 111 reads and executes the program 112a stored in the storage unit 112, thereby realizing the test implementation unit 12, display processing unit 13, investigation content receiving unit 14, source code transmission processing unit 15, etc. as software functional units in the processing unit 111. In this embodiment, input and output of information to and from the application program 21 to be tested is performed via the browser 11. The test implementation unit 12 is located between the front-end program of the application program 21 and the browser 11, and acquires information input and output to and from the application program 21 as trace information. The test implementation unit 12 assigns a trace ID that identifies the test case to the acquired trace information, and transmits the trace ID and the trace information to the debugging server device 3.

[0052] The display processing unit 13 performs processing to display various images, characters, etc. on the display unit 114 based on processing by the processing unit 111. The display processing unit 13 displays, for example, a screen for accepting investigation details and displaying investigation results, as shown in FIG. 3, on the display unit 114. The display processing unit 13 displays the investigation results received from the debugging server device 3 on this screen. The investigation details accepting unit 14 acquires the investigation details entered by the user by operating the operation unit 115 on the screen of FIG. 3 displayed by the display processing unit 13, and transmits them to the debugging server device 3. At this time, the investigation details accepting unit 14 may transmit the trace ID determined by the test implementation unit 12 to the debugging server device 3 together with the investigation details.

[0053] Prior to an error investigation by the debugging server device 3, the source code transmission processing unit 15 performs processing to transmit the source code 112c of the application program 21 to be tested to the debugging server device 3. For example, the source code transmission processing unit 15 receives a file selection of the source code 112c from a user, and transmits the selected source code 112c to the debugging server device 3 via the communication unit 113. At this time, the source code transmission processing unit 15 may transmit information such as version information of the source code 112c or creation date and time information such as a timestamp together with the source code 112c.

[0054] FIG. 5 is a block diagram showing an example configuration of a testing server device 2 according to this embodiment. The testing server device 2 according to this embodiment can be realized by installing a computer program according to this embodiment in an information processing device such as a general-purpose server computer. The testing server device 2 according to this embodiment is configured with a processing unit 211, a storage unit 212, a communication unit 213, etc. The processing unit 211 is configured using an arithmetic processing unit such as a CPU, an MPU, a GPU, an NPU, or a quantum processor, and storage units such as a ROM and a RAM. The processing unit 211 reads and executes a program 212a stored in the storage unit 212 to perform various processes, such as executing the application program 21 to be tested and collecting trace information and log information related to the execution of the application program 21.

[0055] The storage unit 212 is configured using a large-capacity storage device such as a hard disk or SSD. The storage unit 212 stores the program 212a executed by the processing unit 211, the application program 21 to be tested, and the like. In this embodiment, the program (computer program, program product) 212a is provided in a form recorded on a recording medium 97 such as a memory card or an optical disk, and the testing server device 2 reads the program 212a from the recording medium 97 and stores it in the storage unit 212. However, the program 212a may also be provided in a form distributed via a network. The application program 21 is a program to be tested created by a user and may be the same as the source code 112c, or may be intermediate code or binary code (object code) compiled from the source code 112c. The communication unit 213 communicates with various devices, such as the user terminal device 1 and the debugging server device 3, via a network N such as a LAN or the Internet.

[0056] In addition, in the testing server device 2 according to this embodiment, the processing unit 211 reads and executes the program 212a stored in the storage unit 212, whereby the application execution unit 23, the information collection unit 22, etc. are realized as software functional units in the processing unit 211. The application execution unit 23 executes the application program 21 to be tested. The application execution unit 23 is, for example, an interpreter that reads and executes source code or intermediate code, etc. If the application program 21 is executable binary code, etc., the testing server device 2 does not need to include the application execution unit 23.

[0057] The information collection unit 22 collects trace information and log information while the application program 21 is being executed. The trace information may include various information such as functions called during execution of the application program 21, arguments to these functions, and return values ​​of these functions, as well as values ​​stored in variables, data stored in memory, and register values. The log information may include messages such as errors output by the application execution unit 23 that executes the application program 21, and messages output by the application program 21. The information collection unit 22 transmits the collected trace information and log information to the debugging server device 3 via the communication unit 213. If a trace ID related to the test case being executed is provided from the user terminal device 1, the information collection unit 22 may transmit the trace ID together with the trace information and log information.

[0058] <Flowchart> 6 is a flowchart showing an example of the procedure of information collection processing performed by the debugging server device 3 according to this embodiment. The information accumulation unit 31 of the processing unit 311 of the debugging server device 3 according to this embodiment determines whether or not the communication unit 313 has received trace information transmitted from the user terminal device 1, or trace information and log information transmitted from the testing server device 2 (step S1). If such information has been received (S1: YES), the information accumulation unit 31 stores the received information in the trace DB 42 in association with the trace ID (step S2), and returns the process to step S1.

[0059] If the trace information and the log information have not been received (S1: NO), the processing unit 311 determines whether or not the source code 112c of the application program 21 to be tested has been received from the user terminal device 1 (step S3). If the source code 112c has not been received (S3: NO), the processing unit 311 returns the process to step S1. If the source code 112c has been received (S3: YES), the processing unit 311 stores the received source code 112c in the source DB 43 (step S4).

[0060] Next, the summary generation unit 32 of the processing unit 311 generates summary information for the source code 112c stored in the source DB 43 in step S4 (step S5). At this time, the summary generation unit 32 generates the summary information for the source code 112c by, for example, inputting the source code 112c and a prompt including an instruction to generate summary information for the source code 112c to the LLM 41 and acquiring the summary information output by the LLM 41. The summary generation unit 32 vectorizes the summary information generated in step S5 (step S6). At this time, the summary generation unit 32 vectorizes the summary information by, for example, inputting the text of the summary information to the encoder using an encoder model stored in advance in the model information storage unit 312b and acquiring the vector information output by the encoder. The summary generation unit 32 stores the summary information vectorized in step S6 in the summary DB 44 (step S7) and returns the process to step S1.

[0061] 7 is a flowchart showing an example of the procedure of the investigation process performed by the debugging server device 3 according to this embodiment. The investigation processing unit 36 ​​of the processing unit 311 of the debugging server device 3 according to this embodiment receives information transmitted from the user terminal device 1 via the communication unit 313, and thereby acquires the investigation content information input by the user and the trace ID of the test case in which an error has occurred (step S21).

[0062] The investigation processing unit 36 ​​generates a prompt including an instruction to investigate the cause of the error based on the investigation content information acquired in step S21 and the trace ID of the test case in which the error occurred (step S22). At this time, the investigation processing unit 36 ​​generates the prompt by, for example, reading a prompt template stored in advance in the debugging server device 3 and applying information such as the trace ID and investigation content to this template. The investigation processing unit 36 ​​inputs the prompt generated in step S22 to the LLM 41 (step S23). An example of the prompt input to the LLM 41 in step S23 is shown below.

[0063] 1. Execute the get_trace function based on the given trace ID to obtain trace and log information. You can execute this process any number of times. 2. For parts that seem to be related to the investigation request, execute the get_program_specification function based on the file name information obtained from the trace information and log information obtained in step 1 to check the program overview. You can perform this task as many times as you like. 3. For parts that seem to be related to the investigation request, execute the get_program_file function to check the contents of the source code file. You can perform this task as many times as you like. 4. If you have performed steps 1 to 3 above and have been able to analyze the cause of the problem, please report the results of your analysis. If the cause is unknown, please repeat steps 1 to 3 as many times as necessary. -- Trace ID: {trace_id} Investigation request details: {input}

[0064] The functions described in the prompt in this example are predefined functions that can be used by the LLM 41. For example, the get_trace function is a function that takes a trace ID as an input parameter and performs processing to acquire trace information and log information. In response to the call of this get_trace function, the trace reference unit 33 of the processing unit 311 reads information related to the test case of the trace ID from the trace DB 42. For example, the get_program_specification function is a function that takes the file name of the source code acquired from the trace information and log information as an input parameter and performs processing to acquire summary information of the source code. In response to the call of this get_program_specification function, the summary reference unit 34 of the processing unit 311 reads information related to the trace information and log information from the summary DB 44. For example, the get_program_file function is a function that takes the file name of the source code acquired from the trace information and log information as an input parameter and performs processing to acquire source code information. In response to the call of this get_program_file function, the source reference unit 35 of the processing unit 311 reads source code related to errors in the trace information and log information from the source DB 43.

[0065] The LLM 41, which has received the prompt in step S23, understands the contents of the input prompt. Based on item 1 of the above prompt, the LLM 41 determines that "it is necessary to refer to trace information" as a plan, and executes the "get_trace function" as an action to obtain trace information and log information corresponding to the trace ID (step S24).

[0066] The LLM 41 understands the parts that are likely to be related to the investigation request content from the trace information and log information acquired in step S24. Based on item 2 of the above prompt, the LLM 41 decides as a plan that "it is necessary to refer to the program overview of the parts that are likely to be related to the investigation request content," and as an action, executes the "get_program_specification function" to acquire the program overview information of the parts that are likely to be related to the investigation request content based on the file name information acquired from the trace information and log information (step S25).

[0067] The LLM 41 further understands the parts of the program that are likely to be related to the investigation request from the summary information of the program acquired in step S25. Based on item 3 of the above prompt, the LLM 41 decides as a plan that "it is necessary to refer to the source code of the parts that are likely to be related to the investigation request," and as an action, executes the "get_program_file function" to acquire the source code of the parts that are likely to be related to the investigation request based on the file name information acquired from the trace information and log information (step S26).

[0068] The LLM 41 further understands the parts of the source code acquired in step S26 that are thought to be related to the investigation request. If the LLM 41 determines that the cause of the problem has not been analyzed based on the information acquired in steps S24 to S26, it repeats the processing of steps S24 to S26 to collect more information. If it determines that the cause of the problem has been analyzed, the LLM 41 considers that "the analysis results need to be output" as a plan and performs "output of the analysis results" as an action.

[0069] The investigation processing unit 36 ​​acquires the information output by the LLM 41 as the investigation result (step S27). The investigation processing unit 36 ​​transmits the investigation result acquired in step S27 to the user terminal device 1 via the communication unit 313 (step S28), and ends the process.

[0070] <Summary> In the information processing system according to this embodiment having the above configuration, the debugging server device 3 acquires trace information and log information relating to the execution of the application program 21 to be tested, acquires the source code 112c of the application program 21, and acquires investigation content information indicating the details to be investigated for an error that has occurred during the execution of the application program 21. The debugging server device 3 inputs the acquired trace information, log information, source code 112c, and investigation content information to the LLM 41, acquires the investigation result information output by the LLM 41, and transmits (outputs) the acquired investigation result information to the user terminal device 1. As a result, the information processing system according to this embodiment is expected to assist the user in debugging the application program 21.

[0071] Furthermore, in the information processing system according to this embodiment, the debugging server device 3 generates summary information of the acquired source code 112c using the LLM 41, based on the acquired source code 112c. The debugging server device 3 inputs the trace information, log information, source code 112c, investigation content information, and the generated summary information to the LLM 41, and acquires the investigation result information output by the LLM 41. In this way, the information processing system according to this embodiment is expected to improve the accuracy of the investigation of the cause of an error by the LLM 41, by including the summary information of the source code 112c in the information input to the LLM 41.

[0072] Furthermore, in the information processing system according to this embodiment, the debugging server device 3 acquires a trace ID that identifies a test case used in the execution of the application program 21, and stores trace information and log information in association with the acquired trace ID in the trace DB 42. The debugging server device 3 reads trace information and log information from the trace DB 42 based on the trace ID of the test case in which an error occurred, and inputs the read trace information and log information to the LLM 41. As a result, the information processing system according to this embodiment accumulates trace information and log information collected in conjunction with the execution of the application program 21 to be tested in the trace DB 42, and is expected to effectively utilize the accumulated information in investigating the cause of the error.

[0073] Furthermore, in the information processing system according to this embodiment, the debugging server device 3 stores the acquired source code 112c in the source DB 43 and stores the generated summary information in the summary DB 44. The debugging server device 3 reads the source code 112c related to the test case in which an error occurred from the source DB 43, reads the summary information related to the test case in which the error occurred from the summary DB 44, and inputs the read source code 112c and summary information to the LLM 41. As a result, the information processing system according to this embodiment stores the source code 112c of the application program 21 to be tested and its summary information in a database, and is expected to effectively utilize the stored information in investigating the cause of the error.

[0074] Furthermore, in the information processing system according to this embodiment, the summary DB 44 of the debugging server device 3 stores vectorized summary information. As a result, in the information processing system according to this embodiment, it is expected that the summary information will be stored in a format that is easy for the LLM 41 to handle.

[0075] Furthermore, in the information processing system according to this embodiment, the debugging server device 3 generates a prompt including an instruction to investigate an error in the application program 21 based on the trace ID and the investigation content information, and inputs the generated prompt together with the trace information, log information, source code 112c, and summary information to the LLM 41. As a result, the information processing system according to this embodiment is expected to enable the LLM 41 to accurately investigate the cause of an error in the application program 21 to be tested.

[0076] The embodiments disclosed herein are to be considered in all respects as illustrative and not restrictive. The scope of the present disclosure is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims.

[0077] The matters described in each embodiment can be combined with each other. In addition, the independent claims and dependent claims described in the claims can be combined with each other in any combination, regardless of the reference format. Furthermore, although the claims use a format in which a claim references two or more other claims (multiple claim format), this is not limited to this format. A multiple claim (multi-multi claim) that references at least one other multiple claim may also be used. [Explanation of symbols]

[0078] 1. User terminal device 2 Test server equipment 3. Debugging server device 11 Browser 12 Testing Department 13 Display processing section 14. Survey Content Reception Department 15 Source code transmission processing unit 21 Application Programs 22 Information Gathering Department 31 Information Storage Unit 32 Summary generator 33 Trace Reference 34 Overview reference section 35 Source Reference Section 36 Investigation and Processing Department 41 LLM 42 Trace DB 43 Source DB 44 Summary DB 97, 98, 99 Recording media 111 Processing section 112 Storage section 112a Program 112c source code 113 Communications Department 114 Display section 115 Operation section 211 Processing section 212 Storage section 212a Program 213 Communications Department 311 Processing section 312 Storage section 312a Program 312b Model information storage unit 313 Communications Department N Network

Claims

1. The information processing device Acquire trace information and log information related to the execution of the application program to be tested; obtaining the source code of the application program; Acquire summary information of the acquired source code; Obtaining investigation content information indicating the content to be investigated for an error that has occurred during execution of the application program; inputting the acquired trace information, log information, source code, summary information, and investigation content information into a language model; Obtaining survey result information output by the language model; outputting the obtained survey result information; Information processing methods.

2. The acquired source code summary information is vectorized and stored in a database. Based on the acquired trace information and log information, summary information is read from the database; inputting the acquired trace information, the log information, the source code, and the investigation content information, and the read summary information into a language model; Obtaining the survey result information output by the language model; The information processing method according to claim 1 .

3. Based on the acquired source code, summary information of the source code is generated using a language model; inputting the acquired trace information, the log information, the source code, and the investigation content information, and the generated summary information, into a language model; obtaining the survey result information output by the language model; The information processing method according to claim 1 .

4. acquiring identification information of a test case used in executing the application program; storing the acquired trace information and log information in a first database in association with the acquired identification information; reading the trace information and log information from the first database based on identification information of the test case in which the error occurred; inputting the read trace information and log information into the language model; The information processing method according to claim 3 .

5. storing the obtained source code in a second database; storing the generated summary information in a third database; reading from the second database source code associated with the test case in which the error occurred; retrieving summary information relating to the test case in which the error occurred from the third database; inputting the read source code and the summary information into the language model; The information processing method according to claim 4.

6. the third database stores vectorized information of the summary information; The information processing method according to claim 5 .

7. generating a prompt including an instruction to investigate an error in the application program based on the identification information and the investigation content information; inputting the generated prompt into the language model; The information processing method according to claim 4.

8. a first information processing device, Execute the application program; acquiring the trace information and log information in association with the execution of the application program; transmitting the acquired trace information and log information to a second information processing device; The second information processing device, receiving the trace information and log information transmitted from the first information processing device; storing the received trace information and log information in the first database; The information processing method according to claim 4.

9. On the computer, Acquire trace information and log information related to the execution of the application program to be tested; obtaining the source code of the application program; Acquire summary information of the acquired source code; Obtaining investigation content information indicating the content to be investigated for an error that has occurred during execution of the application program; inputting the acquired trace information, log information, source code, summary information, and investigation content information into a language model; Obtaining survey result information output by the language model; Output the acquired survey result information A computer program that executes a process.

10. a processing unit; The processing unit Acquire trace information and log information related to the execution of the application program to be tested; obtaining the source code of the application program; Acquire summary information of the acquired source code; Obtaining investigation content information indicating the content to be investigated for an error that has occurred during execution of the application program; inputting the acquired trace information, log information, source code, summary information, and investigation content information into a language model; Obtaining survey result information output by the language model; outputting the obtained survey result information; Information processing device.

Images