System

Abstract

A system is provided.SOLUTION: A system, comprising: means for obtaining code differences on a web page by an extended function installed on a user equipment; means for sending the obtained code differences to an analysis server; means for analyzing the code differences by a AI model and generating code summaries and improvements at the analysis server; means for returning the generated analysis results to the user equipment; and means for displaying the analysis results on the web page at the user equipment.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Code reviews within a team are an important process for improving product quality, but reviews can take a long time depending on the availability of engineers, leading to a need for more efficient code reviews and consistent quality. Furthermore, when a large number of pull requests are received, the review burden increases, potentially affecting product delivery times. This invention aims to solve these problems, improve the efficiency of code reviews, and ensure consistent review quality. [Means for solving the problem]

[0005] The present invention is a system that provides a means for acquiring code differences on a web page using an extension installed on a user's device, a means for transmitting the acquired code difference data to an analysis server, a means for the analysis server to analyze the code difference data using an AI model and generate a code overview and improvements, a means for returning the generated analysis results to the user's device, and a means for displaying the analysis results on a web page on the user's device.The present invention enables the efficiency and quality of code reviews to be improved, reducing the burden on engineers and contributing to meeting product deadlines.

[0006] A "user terminal" is a computing device through which a user operates an interface and accesses software and web services.

[0007] An "extension" is a plug-in that is added to a web browser to extend or customize the browser's functionality.

[0008] "Code differences" are information indicating changes to source code in a version control system, and include code additions, deletions, and changes between different commits.

[0009] An "analysis server" is a computer device for processing received data, and is a server for performing advanced data analysis and executing machine learning models in particular.

[0010] An "AI model" is a mathematical and computational model that uses artificial intelligence techniques to analyze data and perform pattern recognition and predictions.

[0011] A "code overview" is information that briefly describes the overall purpose and functionality of the source code.

[0012] "Improvements" are specific suggestions or suggestions for improving the quality or functionality of the source code.

[0013] "Analysis results" refers to information including conclusions and recommendations generated by the AI ​​model after analyzing data.

[0014] A "web page" is a document consisting of one or more HTML documents and related resources that are accessible over the Internet.

[0015] A "pull request" is a request in a version control system to propose code changes to be merged into a repository, which can then be reviewed by other developers and either approved or rejected. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a system that uses an extension installed on a user's device to acquire code differences on a web page and transmits the difference data to an analysis server, thereby enabling efficient code review using an AI model.

[0038] Specific embodiments

[0039] User terminal

[0040] The user terminal is a computing device connected to the Internet and capable of accessing websites using a web browser. The user terminal has installed thereon the extension described in the patent claims. This extension is activated when the user opens a pull request page in a version control system such as GitHub.

[0041] Get code diff

[0042] The extension installed on the user's device analyzes the DOM structure of the web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text. The code difference data is then compiled as data for later analysis on the server.

[0043] Sending to the analysis server

[0044] The code difference data obtained on the user's device is sent to the server's API endpoint, which is designed to receive POST requests via the HTTP protocol and analyze the difference data.

[0045] Analysis on the server

[0046] The analysis server inputs the received code difference data into the AI ​​model, which uses pre-trained parameters to generate an outline of the code and suggest improvements. For example, it can detect specific patterns of errors or suggest optimal methods for improving code efficiency.

[0047] Return and display of analysis results

[0048] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately on a web page by the extension. Specifically, explanations and suggestions are displayed in a prominent position on the pull request page, allowing the user to immediately check the analysis results and make any necessary corrections.

[0049] Specific examples

[0050] When a user opens a pull request

[0051] When a user opens a GitHub pull request page, the extension activates and automatically retrieves the code diffs within the page, which are then sent to a server for analysis by an AI model.

[0052] Displaying analysis results

[0053] The analysis results include specific suggestions, such as "This code lacks error handling in the part that establishes a database connection. Please add error handling." Users can use this as a reference to correct their code.

[0054] In this way, the system according to the present invention can improve the efficiency of code reviews and enhance the quality of reviews, allowing engineers to receive immediate feedback and contributing to improving the quality of products.

[0055] The processing flow will be explained below.

[0056] Step 1:

[0057] A user opens the pull request page.

[0058] When a user opens a pull request page in a version control system, the extension installed in the browser is activated.

[0059] Step 2:

[0060] The device will get the code diff.

[0061] The extension installed on the user's device analyzes the DOM structure of the web page, identifies the code difference elements, and retrieves them in text format. Specifically, it uses the class name or ID of the element to find the code difference parts and extracts their contents as text.

[0062] Step 3:

[0063] The terminal transmits the code difference data to the analysis server.

[0064] The device's background script sends the acquired code difference data to the server's API endpoint as a POST request. The request includes the difference data in JSON format.

[0065] Step 4:

[0066] The server receives the differential data.

[0067] The analysis server receives the POST request and extracts the code difference data from the request body.

[0068] Step 5:

[0069] The server analyzes the differential data using an AI model.

[0070] The server passes the extracted differential data to the AI ​​model to begin analysis. The AI ​​model generates an overview of the code and points to improve. For example, it might generate a message like, "This code has the ability to establish a database connection, but it lacks error handling."

[0071] Step 6:

[0072] The server returns the analysis results.

[0073] The server compiles the generated analysis results into JSON format and returns them to the user's device.

[0074] Step 7:

[0075] The terminal receives the analysis results.

[0076] The user's device receives the analysis results returned from the server and forwards them to the content script of the extension.

[0077] Step 8:

[0078] The device displays the analysis results.

[0079] The extension's content script prominently displays the analysis results on the pull request page, either at the top of the page or in the comments section, allowing the user to review the results and make code changes.

[0080] Example 1

[0081] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0082] Conventional code reviews are often performed manually, which can be time-consuming and labor-intensive. Furthermore, the quality of the review depends on the reviewer's experience and skill, making it difficult to maintain a consistent level of quality. The present invention aims to solve these problems and provide a system that improves the efficiency and quality of code reviews.

[0083] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0084] In this invention, the server includes: means for acquiring code differences on a web page using an extension installed on a user terminal; means for transmitting the acquired code difference data to an analysis server; means for analyzing the code difference data using a generative AI model in the analysis server and generating a code overview and improvements; means for returning the generated analysis results to the user terminal; means for using a web browser interconnected to the user terminal via the Internet; means for sending the acquired difference data via an HTTP POST request; and means for returning the analysis results in JSON format. This enables automated and efficient code reviews and rapid, high-quality feedback.

[0085] A "user terminal" is a computing device that can connect to the Internet and access web pages using a web browser.

[0086] An "extension" is plug-in software that is added to a web browser to enhance specific functions.

[0087] A "code difference" is a difference that occurs when comparing code before and after a change.

[0088] An "analysis server" is a computer device that processes and analyzes received data.

[0089] A "generative AI model" is an artificial intelligence model that is pre-trained using machine learning algorithms to output optimal results for a specific task.

[0090] An "HTTP POST request" is a request format for sending data from a client to a server using the HTTP protocol.

[0091] The "JSON format" is a data exchange format that expresses data in object format and can be transferred as text.

[0092] The "DOM structure of a web page" refers to a document object model that represents the hierarchical structure of an HTML or XML document on a web page.

[0093] A "prompt" is an input sentence that gives instructions to an AI model and is used to specify the conditions and requirements for analysis.

[0094] "User needs" are the requirements and desires of the user, and are the criteria for the system to provide customization in response to these requirements.

[0095] The present invention is a system that realizes efficient code review using a generative AI model by using an extension installed on a user's device to obtain code differences on a web page and send the difference data to an analysis server.

[0096] User terminal

[0097] A user device is a computing device connected to the Internet and capable of accessing websites using a web browser (e.g., Google Chrome). A specific extension is installed on the user device. This extension is activated when the user opens a pull request page in a version control system (e.g., GitHub).

[0098] Get code diff

[0099] The extension installed on the user's device analyzes the DOM structure of the web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text. The obtained code differences are then compiled as data for later analysis on the server.

[0100] Sending to the analysis server

[0101] The code difference data acquired on the user's device is sent to the analysis server's API endpoint using an HTTP POST request, which can receive POST requests via the HTTP protocol.

[0102] Analysis on the server

[0103] The analysis server deserializes the received code diff data and feeds it into a generative AI model, which uses pre-trained parameters to generate an outline of the code and suggest improvements, such as detecting specific patterns of errors or suggesting optimal techniques for improving code efficiency.

[0104] Return and display of analysis results

[0105] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately on a web page by the extension. Specifically, explanations and suggestions are displayed in a prominent position on the pull request page, allowing the user to immediately check the analysis results and make any necessary corrections.

[0106] Specific examples

[0107] When a user opens a pull request

[0108] When a user opens a pull request page in a version control system, the extension activates and automatically retrieves the code diffs within the page, which are then sent to a server for analysis by a generative AI model.

[0109] Example prompt:

[0110] "What code diff process happens when a user opens a pull request page?"

[0111] Displaying analysis results

[0112] The analysis results include specific suggestions, such as "This code lacks error handling in the part that establishes a database connection. Please add error handling." Users can use this feedback to correct their code.

[0113] Example prompt:

[0114] "Please explain the process by which the analysis server inputs the code difference data received into the AI ​​model for analysis."

[0115] The present invention can significantly improve the efficiency and quality of code reviews, allowing engineers to receive immediate feedback and contributing to improved product quality.

[0116] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0117] Step 1:

[0118] Installing and launching the extension

[0119] A user installs Google Chrome on their computer and adds a specific extension that runs when they open a pull request page in a version control system (e.g., GitHub).

[0120] Input: Web browser, extension file

[0121] Output: Installed extensions

[0122] What happens: A user searches for an extension in the Chrome Web Store and clicks the "Install" button. After installation, the extension's icon appears in the browser toolbar.

[0123] Step 2:

[0124] Get code diff

[0125] When a user opens a pull request page in a version control system, the extension installed on the device analyzes the DOM structure of the page and extracts text data from multiple code difference elements.

[0126] Input: URL of the pull request page

[0127] Output: Extracted code diff data (text format)

[0128] What it does: The extension looks for HTML elements with specific class names or IDs on the pull request page, extracts the text data of the code diffs from those elements, and converts it to JSON format.

[0129] Step 3:

[0130] Sending differential data to the server

[0131] The device sends the acquired code difference data to the analysis server's API endpoint using an HTTP POST request.

[0132] Input: Code diff data (JSON format)

[0133] Output: POST request to the server

[0134] Specific operation: The differential data obtained on the terminal is serialized in JSON format, included in the body of the HTTP POST request, and sent to the API endpoint.

[0135] Step 4:

[0136] Data analysis on the server

[0137] The server deserializes the received code diff data and feeds it into a generative AI model, which uses pre-trained parameters to generate an outline of the code and suggests improvements.

[0138] Input: Code diff data (JSON format)

[0139] Output: Analysis results (code summary and improvements)

[0140] Specific operation: When the server receives a POST request, it parses the JSON data in the request body and converts it into an input format for the AI ​​model. The model then analyzes it and generates a recommendation result.

[0141] Step 5:

[0142] Return and display of analysis results

[0143] The server returns the generated analysis results in JSON format to the user's device, which then deserializes the results and displays them on the pull request page.

[0144] Input: Analysis result (JSON format)

[0145] Output: Analysis results displayed on the pull request page

[0146] Specific operation: Once the analysis process on the server is complete, the results are formatted as JSON and sent as an HTTP response to the terminal. The terminal parses the JSON data and embeds the results in the HTML of the pull request page.

[0147] This step provides a concrete understanding of the overall system behavior and allows users to receive a fast, high-quality code review.

[0148] (Application example 1)

[0149] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0150] Robot operation and software updates are frequently performed in factories. However, the code reviews and error detection that accompany these updates require a great deal of time and effort, creating a need for greater efficiency. Conventional methods require personnel to manually review the code and identify problems, which can lead to human error and oversight. Real-time feedback is also difficult, and suggestions for improvements are often delayed. This risks reducing the efficiency and safety of robot operation. A system that can resolve these issues and streamline software management for factory robots is needed.

[0151] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0152] In this invention, the server includes means for acquiring code differences on a web page using an extension installed on a user terminal, means for transmitting the acquired code difference data to an analysis server, means for analyzing the code difference data using an AI model in the analysis server and generating a code overview and improvements, means for returning the generated analysis results to the user terminal, means for displaying the analysis results on a web page in the user terminal, means for acquiring code differences in factory robot software and transmitting them to the analysis server, means for analyzing the acquired code differences using an AI model and proposing improvements to the efficiency and safety of robot operation, and means for returning the generated proposal results to the user terminal and displaying them. This enables efficient and accurate code reviews of factory robot software and enables real-time feedback and suggestions for improvements.

[0153] A "user terminal" is a computing device that can connect to the Internet using a web browser and that has advanced features installed.

[0154] An "extension" is a software program that is added to a web browser on a user's device to provide specific functionality.

[0155] "Code differences" are text data of parts of the software version history where changes or additions have been made.

[0156] The "analysis server" is a computer system that receives code difference data sent from a user terminal and analyzes it using an AI model.

[0157] An "AI model" is an artificial intelligence system that uses pre-trained algorithms to analyze input data and perform specific tasks.

[0158] The "generated analysis results" are information about the code overview and areas for improvement obtained after the AI ​​model analyzes the code difference data.

[0159] A "factory robot" is a mechanical device used to perform automated tasks in a factory.

[0160] "Software management" refers to the overall process of software version control, updates, maintenance, and review.

[0161] The "DOM structure of a web page" is an object model that shows the HTML elements of a web page and their hierarchical structure.

[0162] "Text data" is data that is expressed in the form of a string in a computer system.

[0163] This invention is a system for improving the efficiency of software management for factory robots, and is composed of a user terminal, an extension function, an analysis server, and an AI model. Specific embodiments using various hardware and software are described below.

[0164] User terminal

[0165] The user device is a computing device such as a smartphone or PC that can connect to the Internet. An extension is installed in the web browser on the user device to perform software updates and code reviews for factory robots. When a user opens a page containing code differences, the extension runs, analyzes the DOM structure of the web page, and obtains the code difference data.

[0166] Extensions

[0167] The extension is a software program added to the user's web browser. The extension uses the class names and IDs of HTML elements to identify code differences and extracts text data from multiple code difference elements. The code difference data is then sent to the server.

[0168] Analysis Server

[0169] The analysis server is a computer system located on the cloud or on-premise. To receive code difference data sent from user terminals, an API endpoint is provided via the HTTP protocol. The server inputs the received code difference data into an AI model for analysis.

[0170] AI model

[0171] The AI ​​model is an AI system that uses pre-trained algorithms to generate code summaries and improvements. The server uses the AI ​​model to analyze code diff data, detect specific patterns of errors, and suggest optimal methods for improving efficiency. The analysis results are returned to the user's device in JSON format.

[0172] Displaying analysis results

[0173] The analysis results are returned to the user's device and displayed on a web page by the extension, providing feedback on improvements to the software code and detecting errors in a user-friendly format.

[0174] Examples of specific examples and prompts

[0175] For example, the AI ​​model may generate suggestions like the following for code differences obtained when updating a robot's firmware.

[0176] For example: "This code is missing error handling where it establishes a database connection. Please add error handling."

[0177] An example of a prompt sentence to input to the generative AI model is as follows:

[0178] Please analyze the code diff below and suggest improvements:

[0179] Diff: {code_diff}

[0180] example:

[0181] Suggestion: Deprecated API is being used. Please use the latest API.

[0182] This invention enables efficient and accurate software management of factory robots, facilitating early error detection and correction. Real-time feedback and specific improvement suggestions improve the safety and efficiency of robot operations.

[0183] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0184] Step 1:

[0185] When a user opens the factory robot software update page, an extension on the user's device runs and automatically retrieves the code differences within the page. Specifically, the extension analyzes the DOM structure of the web page and identifies the code differences based on the class names and IDs of HTML elements. This analysis retrieves the code differences as text data.

[0186] Input: DOM structure of the web page

[0187] Output: Text data of code diff

[0188] Step 2:

[0189] The code difference data acquired on the user device is sent to the analysis server via the HTTP protocol. Specifically, the extension on the user device sends a POST request to the API endpoint and sends the acquired code difference data to the server in JSON format.

[0190] Input: Text data of code diff

[0191] Output: POST request to the server

[0192] Step 3:

[0193] The analysis server inputs the received code difference data into an AI model. The server uses the pre-trained AI model to analyze the code overview and areas for improvement. For example, it detects errors of specific patterns and suggests optimal methods for improving efficiency.

[0194] Input: Text data of the code diff sent to the server

[0195] Output: Analysis results by the AI ​​model (code summary and improvements)

[0196] Step 4:

[0197] The generated analysis results are returned to the user terminal in JSON format. The analysis server returns the analysis results as a response using the HTTP protocol.

[0198] Input: Analysis results from AI model

[0199] Output: JSON response to the user's device

[0200] Step 5:

[0201] The analysis results returned to the user's device are displayed on the web page by the extension. Specifically, the extension analyzes the analysis results and displays details of improvements and errors in a prominent location on the web page. This display allows the user to immediately check the analysis results and make any necessary corrections.

[0202] Input: Parsed result in JSON format

[0203] Output: Analysis results displayed on a web page

[0204] Step 6:

[0205] Users can then make modifications to the software code based on the analysis results displayed on the webpage. For example, they can follow suggestions from the AI ​​model to fix specific errors or make code changes to improve efficiency. These modifications will make the factory robots operate more safely and efficiently.

[0206] Input: Analysis results displayed on a web page

[0207] Output: User modifications to the software code

[0208] These steps ensure efficient and accurate software management of factory robots, and real-time feedback and specific AI-driven improvement suggestions can be obtained to enhance the safety and efficiency of robot operations.

[0209] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0210] This invention is a system that realizes efficient code review using an AI model by acquiring code differences on a web page using an extension installed on a user's device and sending the difference data to an analysis server. Furthermore, this invention includes a configuration that combines an emotion engine that recognizes the user's emotions, making it possible to adjust the way the analysis results are displayed based on the user's emotions.

[0211] Specific embodiments

[0212] User terminal

[0213] The user terminal is a computing device connected to the Internet and capable of accessing websites using a web browser. The claimed extension and emotion engine are installed on the user terminal. The extension is activated when the user opens a pull request page in a version control system such as GitHub.

[0214] Get code diff

[0215] The extension installed on the user's device analyzes the DOM structure of the web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text. The code difference data is then compiled as data for later analysis on the server.

[0216] Sending to the analysis server

[0217] The code difference data obtained on the user's device is sent to the server's API endpoint, which is designed to receive POST requests via the HTTP protocol and analyze the difference data.

[0218] Analysis on the server

[0219] The analysis server inputs the received code difference data into the AI ​​model, which uses pre-trained parameters to generate an overview of the code and suggestions for improvement. For example, it can detect specific patterns of errors or suggest optimal methods for improving code efficiency.

[0220] Recognizing user emotions with an emotion engine

[0221] The emotion engine analyzes data obtained from the user's input device and camera footage to recognize the user's emotions. The emotion engine can capture the user's emotional state in real time while viewing the pull request page. Emotions include joy, anger, sadness, surprise, etc.

[0222] Return and display of analysis results

[0223] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately on a web page by the extension's content script. The display method is adjusted based on the user's emotional state, as recognized by the emotion engine. For example, if the user is in a state of high stress, the analysis results can be presented in a soft tone, guiding the user into a relaxed state.

[0224] Specific examples

[0225] When a user opens a pull request

[0226] When a user opens a GitHub pull request page, the extension activates and automatically retrieves the code diffs within the page, which are then sent to a server for analysis by an AI model.

[0227] Emotion Engine Operation

[0228] While the user is reviewing the pull request, the emotion engine analyzes the user's facial expressions from the camera footage and recognizes their emotions. For example, if the user is feeling anxious about the code review, the analysis results will be displayed in a reassuring tone to alleviate the user's anxiety.

[0229] Displaying analysis results

[0230] The analysis results include specific suggestions, such as "This code lacks error handling in the part that establishes a database connection. Please add error handling," as well as comments that take the user's feelings into consideration. Users can refer to these and make corrections to their code.

[0231] In this way, the system according to the present invention can improve the efficiency of code reviews and enhance the quality of reviews. Furthermore, the introduction of an emotion engine can improve the user experience by providing feedback that takes into account the user's emotional state.

[0232] The processing flow will be explained below.

[0233] Step 1:

[0234] A user opens the pull request page.

[0235] When a user opens a pull request page in a version control system, the extension installed in the browser is activated.

[0236] Step 2:

[0237] The user terminal acquires the code difference.

[0238] The extension installed on the user's device analyzes the DOM structure of the web page, identifies the code differences, and retrieves them in text format. Specifically, it uses the class names and IDs of the elements to find the code differences and extracts their contents as text.

[0239] Step 3:

[0240] The user terminal transmits the code difference data to the analysis server.

[0241] The device's background script sends the acquired code difference data to the server's API endpoint as a POST request. The request includes the difference data in JSON format.

[0242] Step 4:

[0243] The server receives the differential data.

[0244] The analysis server receives the POST request and extracts the code difference data from the request body.

[0245] Step 5:

[0246] The server analyzes the differential data using an AI model.

[0247] The server inputs the extracted differential data into the AI ​​model and begins analysis. The AI ​​model generates an overview of the code and points to improve. For example, it might generate a message like, "This code has the ability to establish a database connection, but error handling is lacking."

[0248] Step 6:

[0249] The emotion engine of the user terminal recognizes the user's emotion.

[0250] The emotion engine analyzes data obtained from the user's input devices and camera footage to recognize the user's emotions. It identifies emotions such as joy, anger, sadness, and surprise.

[0251] Step 7:

[0252] The server returns the analysis results.

[0253] The server then compiles the generated analysis results in JSON format and sends them back to the user's device. The analysis results include an overview of the code, suggestions for improvement, and comments appropriate to the user's feelings.

[0254] Step 8:

[0255] The user terminal receives the analysis results.

[0256] A background script on the user terminal receives the analysis results returned from the server and transfers the results to the content script.

[0257] Step 9:

[0258] The user terminal displays the analysis results.

[0259] The content script displays the received analysis results in a prominent location on the pull request page. For example, it displays the analysis results for each code change so that users can understand them immediately. This display method takes user emotions into consideration, allowing users to review in a relaxed atmosphere.

[0260] Example 2

[0261] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0262] Conventional code review systems are limited to analyzing code and suggesting improvements, and are unable to provide feedback that takes into account the user's emotional state. This can lead to stress and make it difficult for users to accept the review results. Furthermore, many systems have difficulty acquiring and analyzing code differences, requiring significant effort to achieve efficient code reviews. Therefore, there is a need for a system that provides appropriate feedback based on the user's emotional state and allows for efficient and effective code reviews.

[0263] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring code differences on a web page using an extension installed on a user terminal, means for transmitting the acquired code difference data to an analysis server, means for analyzing the code difference data using a generative AI model in the analysis server and generating a code outline and improvements, means for returning the generated analysis results to the user terminal, means for detecting the user's emotional state using an emotion recognition engine, means for adjusting the display method of the analysis results depending on the emotional state, and means for displaying the analysis results on a web page in the user terminal. This enables feedback that takes the user's emotional state into consideration, resulting in efficient and effective code reviews.

[0264] "User terminal" refers to a computing device that can connect to the Internet and that can access websites using a web browser.

[0265] An "extension" refers to a program module that is installed in a particular web browser and provides specific additional functionality on a web page.

[0266] "Code differences" are data that indicate changes to source code in a version control system.

[0267] The term "analysis server" refers to a computer device that has the function of processing and analyzing received code difference data.

[0268] A "generative AI model" is an artificial intelligence model that is trained in advance using large amounts of data, and then analyzes code and suggests improvements.

[0269] An "emotion recognition engine" refers to software or hardware that analyzes a user's input device or camera footage to detect the user's emotional state.

[0270] "Analysis results" refers to data that includes an overview of the code generated by the generative AI model and areas for improvement.

[0271] "Adjusting the display method" refers to changing the way the analysis results are presented depending on the user's emotional state.

[0272] User terminal

[0273] The user terminal is a computing device that can connect to the Internet and can access web pages using a web browser. The user terminal includes an extension and an emotion recognition engine installed in a specific web browser. The extension is activated, for example, when the user opens a pull request page in a version control system.

[0274] Get code diff

[0275] The extension analyzes the DOM structure of a web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text data.

[0276] Analysis Server

[0277] The code difference data acquired by the user's device is sent to the analysis server via HTTP protocol. The analysis server inputs the received data into a generative AI model to generate an outline of the code and improvements. This generative AI model is a model that has been trained in advance on a large code dataset.

[0278] Generative AI Models

[0279] Based on the input code difference data, the generative AI model detects specific error patterns and suggests improvements to improve code efficiency, such as "This function lacks error handling when connecting to a database."

[0280] Emotion Recognition Engine

[0281] The emotion recognition engine on the user device analyzes the user's input device and camera footage to recognize the user's emotional state in real time. Types of emotions include joy, anger, sadness, surprise, etc., and the way the analysis results are displayed is adjusted based on these.

[0282] Return and display of analysis results

[0283] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately by the extension's content script. The display method is adjusted based on the user's emotional state detected by the emotion recognition engine. For example, if the user is under stress, the analysis results will be displayed in a softer tone to reduce the user's psychological burden.

[0284] Specific examples

[0285] When a user opens a pull request

[0286] When a user opens a GitHub pull request page, the extension activates and automatically retrieves the code diffs within the webpage, which are then sent to an analytics server for analysis by the generative AI model.

[0287] Emotion Engine Operation

[0288] While the user is reviewing the pull request, the emotion engine analyzes the user's facial expressions from the camera footage and recognizes their emotions. For example, if the user is feeling anxious about the code review, the analysis results will be displayed in a reassuring tone to alleviate the user's anxiety.

[0289] Displaying analysis results

[0290] The analysis results include specific suggestions such as "This code lacks error handling in the part that establishes a database connection. Please add error handling," as well as comments that take the user's feelings into consideration. Users can use this feedback to make corrections to their code.

[0291] Prompt Sentence Examples

[0292] "Please analyze this code diff and suggest the best improvements."

[0293] "If the user is under high stress, please display the analysis results in a relaxing tone."

[0294] If you find a specific error pattern, please suggest ways to improve it.

[0295] In this way, the system according to the present invention can improve the efficiency of code reviews and enhance the quality of reviews. Furthermore, by utilizing an emotion recognition engine, feedback that takes into account the user's emotional state can be provided, thereby improving the user experience.

[0296] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0297] Step 1:

[0298] When a user opens a pull request page in a version control system using a web browser, the extension is automatically activated and monitors the page load. The input is the user accessing the page, and the output is the extension's launch. Specifically, the extension waits for the page's DOM to be fully loaded.

[0299] Step 2:

[0300] The extension analyzes the DOM structure of a web page to identify code differences. This involves using the class names and IDs of HTML elements to find elements that contain code differences. The input is the DOM structure of the page, and the output is the text data of the code differences. Specifically, the extension uses JavaScript methods such as querySelector to extract specific elements.

[0301] Step 3:

[0302] The user device converts the acquired code difference text data into JSON format and sends it to the analysis server as an HTTP POST request. The input is the code difference text data, and the output is a request sent to the analysis server. Specifically, the user device uses the fetch API to send the data to the specified URL.

[0303] Step 4:

[0304] The analysis server analyzes the received code difference data. First, it inputs the data into a generative AI model to analyze the code. The input is JSON data of the code difference, and the output is the generation of analysis results. Specifically, the server invokes an AI model (for example, a model using Python's TensorFlow or PyTorch) to generate an overview of the code and areas for improvement.

[0305] Step 5:

[0306] The emotion recognition engine analyzes the camera images and input device data on the user's device to recognize the user's emotional state in real time. The input is the camera images and input device data, and the output is data on the user's emotional state. Specifically, the emotion recognition engine uses OpenCV and deep learning frameworks to analyze facial expressions.

[0307] Step 6:

[0308] The analysis server returns the generated analysis results in JSON format to the user device. At the same time, it also returns instructions to adjust the display method based on the user's emotional state. The input is the analysis results and the user's emotional state data, and the output is response data in JSON format. Specifically, the server returns the analysis results as an HTTP response.

[0309] Step 7:

[0310] The user device displays the received analysis results on a web page. The display method is adjusted according to the user's emotional state. The input is the JSON data of the analysis results, and the output is the user's display screen. Specifically, the extension performs DOM manipulation to appropriately display the analysis results. For example, it uses the innerHTML method to insert the analysis results into HTML elements and adjust the display style.

[0311] In this way, the system acts automatically in response to user actions, improving the efficiency of code review and the user experience.

[0312] (Application example 2)

[0313] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0314] Conventional code review systems have limited means for efficiently analyzing code difference data, making it difficult to provide appropriate feedback based on the user's emotional state. Furthermore, code reviews can sometimes cause stress, so measures to improve the user experience are needed.

[0315] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring code differences on a web page using an extension installed on a user terminal, means for transmitting the acquired code difference data to an analysis server, means for analyzing the code difference data using a generative AI model and generating a code outline and improvements in the analysis server, means for adjusting the display method of the analysis results using an emotion engine that recognizes the user's emotional state, means for returning the generated analysis results to the user terminal, and means for displaying the analysis results on a web page in the user terminal. This makes it possible to improve the efficiency of code review while providing appropriate feedback tailored to the user's emotional state.

[0316] A "user terminal" is a computing device connected to the Internet that can access websites using a web browser.

[0317] An "extension" is a software component installed in a web browser to perform specific tasks in order to extend the functionality of the browser.

[0318] A "web page" is a page of information accessible via the Internet, the content of which is written in a standard language such as HTML.

[0319] A "code delta" refers to a change in software code, typically representing the difference between old and new source code in a version control system.

[0320] An "analysis server" is a remote computing device that processes and analyzes received data to generate results.

[0321] A "generative AI model" is an artificial intelligence model that has been pre-trained using machine learning and deep learning techniques to perform analysis and predictions based on input data.

[0322] The "emotion engine" is a system that analyzes data obtained from the user's input device and camera footage to recognize the user's emotions.

[0323] "User's emotional state" refers to the user's mental and emotional state, including emotions such as joy, anger, sadness, surprise, etc.

[0324] "Adjusting the display method" refers to changing the way the analysis results are presented based on the user's emotional state.

[0325] The "DOM structure of a web page" is the document object model of a web page, which is a tree-like representation of the logical structure of an HTML or XML document.

[0326] "Text data" refers to data in the form of code or strings that can be analyzed and processed.

[0327] User terminal

[0328] The user terminal is a computing device connected to the Internet and capable of accessing websites using a web browser. The user terminal has installed thereon the extension and emotion engine described in the claims. The extension operates when the user opens a pull request page in the version control system.

[0329] Get code diff

[0330] The extension installed on the user's device analyzes the DOM structure of the web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text. The code difference data is then compiled as data for later analysis on the server.

[0331] Sending to the analysis server

[0332] The code diff data is sent to a server API endpoint that is designed to accept POST requests over the HTTP protocol and parse the diff data.

[0333] Analysis on the server

[0334] The analysis server inputs the received code difference data into a generative AI model. The generative AI model uses pre-trained parameters to generate an outline of the code and suggestions for improvement. For example, it can detect specific patterns of errors or suggest optimal methods for improving code efficiency.

[0335] Recognizing user emotions with an emotion engine

[0336] The emotion engine analyzes data obtained from the user's input device and camera footage to recognize the user's emotions. The emotion engine can capture the user's emotional state in real time while viewing the pull request page. Emotions include joy, anger, sadness, surprise, etc.

[0337] Return and display of analysis results

[0338] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately on a web page by the extension's content script. The display method is adjusted based on the user's emotional state, as recognized by the emotion engine. For example, if the user is in a state of high stress, the analysis results can be presented in a soft tone, guiding the user into a relaxed state.

[0339] Specific examples

[0340] Example 1: Recommendations based on product page emotional state

[0341] If the emotion engine detects excitement while a user is viewing a particular product page:

[0342] "Want to know more about this product? Click here for more information!"

[0343] Input prompt example

[0344] "Show me a Python script that recognizes a user's emotional state while browsing a product page and serves them a message tailored to that emotion."

[0345] In this way, each component, such as the server, user terminal, and emotion engine, works in cooperation to improve the user experience while enabling efficient code review.

[0346] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0347] Step 1:

[0348] When a user opens a pull request page, an extension installed on the user's device is activated. The extension analyzes the DOM structure of the web page and identifies code differences. The HTML content of the web page is used as input, and the output is text data containing the code differences. This text data is then compiled in a format that can be sent to the server.

[0349] Step 2:

[0350] The user device sends the acquired code difference data to the analysis server's API endpoint via an HTTP POST request. The input is the code difference text data obtained in step 1, and the output is the code difference data in JSON format that the server receives.

[0351] Step 3:

[0352] The analysis server processes the received code difference data and inputs it into the generative AI model. The input is the code difference data sent in step 2, and the output is the analysis results from the generative AI model. The generative AI model generates a code outline and improvements using pre-trained parameters.

[0353] Step 4:

[0354] The server returns the analysis results in JSON format to the user device. The input is the analysis results from the generative AI model, and the output is the JSON data of the analysis results received by the user device. This data includes improvement suggestions and error detection information.

[0355] Step 5:

[0356] The user device receives the analysis results and simultaneously activates the emotion engine to recognize the user's emotional state in real time. The input is camera footage capturing the user's facial expressions and data from input devices, and the output is the user's emotional state (joy, anger, sadness, surprise, etc.).

[0357] Step 6:

[0358] Based on the user's emotional state output by the emotion engine, the system selects appropriate templates and tones to adjust the display method. The input is the user's emotional state and the analysis results from the server, and the output is the display content of the adjusted analysis results.

[0359] Step 7:

[0360] The user terminal displays the adjusted analysis results on a web page. The input is the display content adjusted in step 6, and the output is feedback that is easy for the user to see and causes little stress to the user.

[0361] Through these steps, users can receive efficient code reviews and receive feedback tailored to their emotional state, improving the user experience.

[0362] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0363] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0364] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0365] [Second embodiment]

[0366] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0367] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0368] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0369] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0370] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0371] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0372] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0373] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0374] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0375] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0376] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0377] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0378] This invention is a system that uses an extension installed on a user's device to acquire code differences on a web page and transmits the difference data to an analysis server, thereby enabling efficient code review using an AI model.

[0379] Specific embodiments

[0380] User terminal

[0381] The user terminal is a computing device connected to the Internet and capable of accessing websites using a web browser. The user terminal has installed thereon the extension described in the patent claims. This extension is activated when the user opens a pull request page in a version control system such as GitHub.

[0382] Get code diff

[0383] The extension installed on the user's device analyzes the DOM structure of the web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text. The code difference data is then compiled as data for later analysis on the server.

[0384] Sending to the analysis server

[0385] The code difference data obtained on the user's device is sent to the server's API endpoint, which is designed to receive POST requests via the HTTP protocol and analyze the difference data.

[0386] Analysis on the server

[0387] The analysis server inputs the received code difference data into the AI ​​model, which uses pre-trained parameters to generate an outline of the code and suggest improvements. For example, it can detect specific patterns of errors or suggest optimal methods for improving code efficiency.

[0388] Return and display of analysis results

[0389] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately on a web page by the extension. Specifically, explanations and suggestions are displayed in a prominent position on the pull request page, allowing the user to immediately check the analysis results and make any necessary corrections.

[0390] Specific examples

[0391] When a user opens a pull request

[0392] When a user opens a GitHub pull request page, the extension activates and automatically retrieves the code diffs within the page, which are then sent to a server for analysis by an AI model.

[0393] Displaying analysis results

[0394] The analysis results include specific suggestions, such as "This code lacks error handling in the part that establishes a database connection. Please add error handling." Users can use this as a reference to correct their code.

[0395] In this way, the system according to the present invention can improve the efficiency of code reviews and enhance the quality of reviews, allowing engineers to receive immediate feedback and contributing to improving the quality of products.

[0396] The processing flow will be explained below.

[0397] Step 1:

[0398] A user opens the pull request page.

[0399] When a user opens a pull request page in a version control system, the extension installed in the browser is activated.

[0400] Step 2:

[0401] The device will get the code diff.

[0402] The extension installed on the user's device analyzes the DOM structure of the web page, identifies the code difference elements, and retrieves them in text format. Specifically, it uses the class name or ID of the element to find the code difference parts and extracts their contents as text.

[0403] Step 3:

[0404] The terminal transmits the code difference data to the analysis server.

[0405] The device's background script sends the acquired code difference data to the server's API endpoint as a POST request. The request includes the difference data in JSON format.

[0406] Step 4:

[0407] The server receives the differential data.

[0408] The analysis server receives the POST request and extracts the code difference data from the request body.

[0409] Step 5:

[0410] The server analyzes the differential data using an AI model.

[0411] The server passes the extracted differential data to the AI ​​model to begin analysis. The AI ​​model generates an overview of the code and points to improve. For example, it might generate a message like, "This code has the ability to establish a database connection, but it lacks error handling."

[0412] Step 6:

[0413] The server returns the analysis results.

[0414] The server compiles the generated analysis results into JSON format and returns them to the user's device.

[0415] Step 7:

[0416] The terminal receives the analysis results.

[0417] The user's device receives the analysis results returned from the server and forwards them to the content script of the extension.

[0418] Step 8:

[0419] The device displays the analysis results.

[0420] The extension's content script prominently displays the analysis results on the pull request page, either at the top of the page or in the comments section, allowing the user to review the results and make code changes.

[0421] Example 1

[0422] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0423] Conventional code reviews are often performed manually, which can be time-consuming and labor-intensive. Furthermore, the quality of the review depends on the reviewer's experience and skill, making it difficult to maintain a consistent level of quality. The present invention aims to solve these problems and provide a system that improves the efficiency and quality of code reviews.

[0424] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0425] In this invention, the server includes: means for acquiring code differences on a web page using an extension installed on a user terminal; means for transmitting the acquired code difference data to an analysis server; means for analyzing the code difference data using a generative AI model in the analysis server and generating a code overview and improvements; means for returning the generated analysis results to the user terminal; means for using a web browser interconnected to the user terminal via the Internet; means for sending the acquired difference data via an HTTP POST request; and means for returning the analysis results in JSON format. This enables automated and efficient code reviews and rapid, high-quality feedback.

[0426] A "user terminal" is a computing device that can connect to the Internet and access web pages using a web browser.

[0427] An "extension" is plug-in software that is added to a web browser to enhance specific functions.

[0428] A "code difference" is a difference that occurs when comparing code before and after a change.

[0429] An "analysis server" is a computer device that processes and analyzes received data.

[0430] A "generative AI model" is an artificial intelligence model that is pre-trained using machine learning algorithms to output optimal results for a specific task.

[0431] An "HTTP POST request" is a request format for sending data from a client to a server using the HTTP protocol.

[0432] The "JSON format" is a data exchange format that expresses data in object format and can be transferred as text.

[0433] The "DOM structure of a web page" refers to a document object model that represents the hierarchical structure of an HTML or XML document on a web page.

[0434] A "prompt" is an input sentence that gives instructions to an AI model and is used to specify the conditions and requirements for analysis.

[0435] "User needs" are the requirements and desires of the user, and are the criteria for the system to provide customization in response to these requirements.

[0436] The present invention is a system that realizes efficient code review using a generative AI model by using an extension installed on a user's device to obtain code differences on a web page and send the difference data to an analysis server.

[0437] User terminal

[0438] A user device is a computing device connected to the Internet and capable of accessing websites using a web browser (e.g., Google Chrome). A specific extension is installed on the user device. This extension is activated when the user opens a pull request page in a version control system (e.g., GitHub).

[0439] Get code diff

[0440] The extension installed on the user's device analyzes the DOM structure of the web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text. The obtained code differences are then compiled as data for later analysis on the server.

[0441] Sending to the analysis server

[0442] The code difference data acquired on the user's device is sent to the analysis server's API endpoint using an HTTP POST request, which can receive POST requests via the HTTP protocol.

[0443] Analysis on the server

[0444] The analysis server deserializes the received code diff data and feeds it into a generative AI model, which uses pre-trained parameters to generate an outline of the code and suggest improvements, such as detecting specific patterns of errors or suggesting optimal techniques for improving code efficiency.

[0445] Return and display of analysis results

[0446] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately on a web page by the extension. Specifically, explanations and suggestions are displayed in a prominent position on the pull request page, allowing the user to immediately check the analysis results and make any necessary corrections.

[0447] Specific examples

[0448] When a user opens a pull request

[0449] When a user opens a pull request page in a version control system, the extension activates and automatically retrieves the code diffs within the page, which are then sent to a server for analysis by a generative AI model.

[0450] Example prompt:

[0451] "What code diff process happens when a user opens a pull request page?"

[0452] Displaying analysis results

[0453] The analysis results include specific suggestions, such as "This code lacks error handling in the part that establishes a database connection. Please add error handling." Users can use this feedback to correct their code.

[0454] Example prompt:

[0455] "Please explain the process by which the analysis server inputs the code difference data received into the AI ​​model for analysis."

[0456] The present invention can significantly improve the efficiency and quality of code reviews, allowing engineers to receive immediate feedback and contributing to improved product quality.

[0457] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0458] Step 1:

[0459] Installing and launching the extension

[0460] A user installs Google Chrome on their computer and adds a specific extension that runs when they open a pull request page in a version control system (e.g., GitHub).

[0461] Input: Web browser, extension file

[0462] Output: Installed extensions

[0463] What happens: A user searches for an extension in the Chrome Web Store and clicks the "Install" button. After installation, the extension's icon appears in the browser toolbar.

[0464] Step 2:

[0465] Get code diff

[0466] When a user opens a pull request page in a version control system, the extension installed on the device analyzes the DOM structure of the page and extracts text data from multiple code difference elements.

[0467] Input: URL of the pull request page

[0468] Output: Extracted code diff data (text format)

[0469] What it does: The extension looks for HTML elements with specific class names or IDs on the pull request page, extracts the text data of the code diffs from those elements, and converts it to JSON format.

[0470] Step 3:

[0471] Sending differential data to the server

[0472] The device sends the acquired code difference data to the analysis server's API endpoint using an HTTP POST request.

[0473] Input: Code diff data (JSON format)

[0474] Output: POST request to the server

[0475] Specific operation: The differential data obtained on the terminal is serialized in JSON format, included in the body of the HTTP POST request, and sent to the API endpoint.

[0476] Step 4:

[0477] Data analysis on the server

[0478] The server deserializes the received code diff data and feeds it into a generative AI model, which uses pre-trained parameters to generate an outline of the code and suggests improvements.

[0479] Input: Code diff data (JSON format)

[0480] Output: Analysis results (code summary and improvements)

[0481] Specific operation: When the server receives a POST request, it parses the JSON data in the request body and converts it into an input format for the AI ​​model. The model then analyzes it and generates a recommendation result.

[0482] Step 5:

[0483] Return and display of analysis results

[0484] The server returns the generated analysis results in JSON format to the user's device, which then deserializes the results and displays them on the pull request page.

[0485] Input: Analysis result (JSON format)

[0486] Output: Analysis results displayed on the pull request page

[0487] Specific operation: Once the analysis process on the server is complete, the results are formatted as JSON and sent as an HTTP response to the terminal. The terminal parses the JSON data and embeds the results in the HTML of the pull request page.

[0488] This step provides a concrete understanding of the overall system behavior and allows users to receive a fast, high-quality code review.

[0489] (Application example 1)

[0490] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0491] Robot operation and software updates are frequently performed in factories. However, the code reviews and error detection that accompany these updates require a great deal of time and effort, creating a need for greater efficiency. Conventional methods require personnel to manually review the code and identify problems, which can lead to human error and oversight. Real-time feedback is also difficult, and suggestions for improvements are often delayed. This risks reducing the efficiency and safety of robot operation. A system that can resolve these issues and streamline software management for factory robots is needed.

[0492] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0493] In this invention, the server includes means for acquiring code differences on a web page using an extension installed on a user terminal, means for transmitting the acquired code difference data to an analysis server, means for analyzing the code difference data using an AI model in the analysis server and generating a code overview and improvements, means for returning the generated analysis results to the user terminal, means for displaying the analysis results on a web page in the user terminal, means for acquiring code differences in factory robot software and transmitting them to the analysis server, means for analyzing the acquired code differences using an AI model and proposing improvements to the efficiency and safety of robot operation, and means for returning the generated proposal results to the user terminal and displaying them. This enables efficient and accurate code reviews of factory robot software and enables real-time feedback and suggestions for improvements.

[0494] A "user terminal" is a computing device that can connect to the Internet using a web browser and that has advanced features installed.

[0495] An "extension" is a software program that is added to a web browser on a user's device to provide specific functionality.

[0496] "Code differences" are text data of parts of the software version history where changes or additions have been made.

[0497] The "analysis server" is a computer system that receives code difference data sent from a user terminal and analyzes it using an AI model.

[0498] An "AI model" is an artificial intelligence system that uses pre-trained algorithms to analyze input data and perform specific tasks.

[0499] The "generated analysis results" are information about the code overview and areas for improvement obtained after the AI ​​model analyzes the code difference data.

[0500] A "factory robot" is a mechanical device used to perform automated tasks in a factory.

[0501] "Software management" refers to the overall process of software version control, updates, maintenance, and review.

[0502] The "DOM structure of a web page" is an object model that shows the HTML elements of a web page and their hierarchical structure.

[0503] "Text data" is data that is expressed in the form of a string in a computer system.

[0504] This invention is a system for improving the efficiency of software management for factory robots, and is composed of a user terminal, an extension function, an analysis server, and an AI model. Specific embodiments using various hardware and software are described below.

[0505] User terminal

[0506] The user device is a computing device such as a smartphone or PC that can connect to the Internet. An extension is installed in the web browser on the user device to perform software updates and code reviews for factory robots. When a user opens a page containing code differences, the extension runs, analyzes the DOM structure of the web page, and obtains the code difference data.

[0507] Extensions

[0508] The extension is a software program added to the user's web browser. The extension uses the class names and IDs of HTML elements to identify code differences and extracts text data from multiple code difference elements. The code difference data is then sent to the server.

[0509] Analysis Server

[0510] The analysis server is a computer system located on the cloud or on-premise. To receive code difference data sent from user terminals, an API endpoint is provided via the HTTP protocol. The server inputs the received code difference data into an AI model for analysis.

[0511] AI model

[0512] The AI ​​model is an AI system that uses pre-trained algorithms to generate code summaries and improvements. The server uses the AI ​​model to analyze code diff data, detect specific patterns of errors, and suggest optimal methods for improving efficiency. The analysis results are returned to the user's device in JSON format.

[0513] Displaying analysis results

[0514] The analysis results are returned to the user's device and displayed on a web page by the extension, providing feedback on improvements to the software code and detecting errors in a user-friendly format.

[0515] Examples of specific examples and prompts

[0516] For example, the AI ​​model may generate suggestions like the following for code differences obtained when updating a robot's firmware.

[0517] For example: "This code is missing error handling where it establishes a database connection. Please add error handling."

[0518] An example of a prompt sentence to input to the generative AI model is as follows:

[0519] Please analyze the code diff below and suggest improvements:

[0520] Diff: {code_diff}

[0521] example:

[0522] Suggestion: Deprecated API is being used. Please use the latest API.

[0523] This invention enables efficient and accurate software management of factory robots, facilitating early error detection and correction. Real-time feedback and specific improvement suggestions improve the safety and efficiency of robot operations.

[0524] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0525] Step 1:

[0526] When a user opens the factory robot software update page, an extension on the user's device runs and automatically retrieves the code differences within the page. Specifically, the extension analyzes the DOM structure of the web page and identifies the code differences based on the class names and IDs of HTML elements. This analysis retrieves the code differences as text data.

[0527] Input: DOM structure of the web page

[0528] Output: Text data of code diff

[0529] Step 2:

[0530] The code difference data acquired on the user device is sent to the analysis server via the HTTP protocol. Specifically, the extension on the user device sends a POST request to the API endpoint and sends the acquired code difference data to the server in JSON format.

[0531] Input: Text data of code diff

[0532] Output: POST request to the server

[0533] Step 3:

[0534] The analysis server inputs the received code difference data into an AI model. The server uses the pre-trained AI model to analyze the code overview and areas for improvement. For example, it detects errors of specific patterns and suggests optimal methods for improving efficiency.

[0535] Input: Text data of the code diff sent to the server

[0536] Output: Analysis results by the AI ​​model (code summary and improvements)

[0537] Step 4:

[0538] The generated analysis results are returned to the user terminal in JSON format. The analysis server returns the analysis results as a response using the HTTP protocol.

[0539] Input: Analysis results from AI model

[0540] Output: JSON response to the user's device

[0541] Step 5:

[0542] The analysis results returned to the user's device are displayed on the web page by the extension. Specifically, the extension analyzes the analysis results and displays details of improvements and errors in a prominent location on the web page. This display allows the user to immediately check the analysis results and make any necessary corrections.

[0543] Input: Parsed result in JSON format

[0544] Output: Analysis results displayed on a web page

[0545] Step 6:

[0546] Users can then make modifications to the software code based on the analysis results displayed on the webpage. For example, they can follow suggestions from the AI ​​model to fix specific errors or make code changes to improve efficiency. These modifications will make the factory robots operate more safely and efficiently.

[0547] Input: Analysis results displayed on a web page

[0548] Output: User modifications to the software code

[0549] These steps ensure efficient and accurate software management of factory robots, and real-time feedback and specific AI-driven improvement suggestions can be obtained to enhance the safety and efficiency of robot operations.

[0550] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0551] This invention is a system that realizes efficient code review using an AI model by acquiring code differences on a web page using an extension installed on a user's device and sending the difference data to an analysis server. Furthermore, this invention includes a configuration that combines an emotion engine that recognizes the user's emotions, making it possible to adjust the way the analysis results are displayed based on the user's emotions.

[0552] Specific embodiments

[0553] User terminal

[0554] The user terminal is a computing device connected to the Internet and capable of accessing websites using a web browser. The claimed extension and emotion engine are installed on the user terminal. The extension is activated when the user opens a pull request page in a version control system such as GitHub.

[0555] Get code diff

[0556] The extension installed on the user's device analyzes the DOM structure of the web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text. The code difference data is then compiled as data for later analysis on the server.

[0557] Sending to the analysis server

[0558] The code difference data obtained on the user's device is sent to the server's API endpoint, which is designed to receive POST requests via the HTTP protocol and analyze the difference data.

[0559] Analysis on the server

[0560] The analysis server inputs the received code difference data into the AI ​​model, which uses pre-trained parameters to generate an overview of the code and suggestions for improvement. For example, it can detect specific patterns of errors or suggest optimal methods for improving code efficiency.

[0561] Recognizing user emotions with an emotion engine

[0562] The emotion engine analyzes data obtained from the user's input device and camera footage to recognize the user's emotions. The emotion engine can capture the user's emotional state in real time while viewing the pull request page. Emotions include joy, anger, sadness, surprise, etc.

[0563] Return and display of analysis results

[0564] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately on a web page by the extension's content script. The display method is adjusted based on the user's emotional state, as recognized by the emotion engine. For example, if the user is in a state of high stress, the analysis results can be presented in a soft tone, guiding the user into a relaxed state.

[0565] Specific examples

[0566] When a user opens a pull request

[0567] When a user opens a GitHub pull request page, the extension activates and automatically retrieves the code diffs within the page, which are then sent to a server for analysis by an AI model.

[0568] Emotion Engine Operation

[0569] While the user is reviewing the pull request, the emotion engine analyzes the user's facial expressions from the camera footage and recognizes their emotions. For example, if the user is feeling anxious about the code review, the analysis results will be displayed in a reassuring tone to alleviate the user's anxiety.

[0570] Displaying analysis results

[0571] The analysis results include specific suggestions, such as "This code lacks error handling in the part that establishes a database connection. Please add error handling," as well as comments that take the user's feelings into consideration. Users can refer to these and make corrections to their code.

[0572] In this way, the system according to the present invention can improve the efficiency of code reviews and enhance the quality of reviews. Furthermore, the introduction of an emotion engine can improve the user experience by providing feedback that takes into account the user's emotional state.

[0573] The processing flow will be explained below.

[0574] Step 1:

[0575] A user opens the pull request page.

[0576] When a user opens a pull request page in a version control system, the extension installed in the browser is activated.

[0577] Step 2:

[0578] The user terminal acquires the code difference.

[0579] The extension installed on the user's device analyzes the DOM structure of the web page, identifies the code differences, and retrieves them in text format. Specifically, it uses the class names and IDs of the elements to find the code differences and extracts their contents as text.

[0580] Step 3:

[0581] The user terminal transmits the code difference data to the analysis server.

[0582] The device's background script sends the acquired code difference data to the server's API endpoint as a POST request. The request includes the difference data in JSON format.

[0583] Step 4:

[0584] The server receives the differential data.

[0585] The analysis server receives the POST request and extracts the code difference data from the request body.

[0586] Step 5:

[0587] The server analyzes the differential data using an AI model.

[0588] The server inputs the extracted differential data into the AI ​​model and begins analysis. The AI ​​model generates an overview of the code and points to improve. For example, it might generate a message like, "This code has the ability to establish a database connection, but error handling is lacking."

[0589] Step 6:

[0590] The emotion engine of the user terminal recognizes the user's emotion.

[0591] The emotion engine analyzes data obtained from the user's input devices and camera footage to recognize the user's emotions. It identifies emotions such as joy, anger, sadness, and surprise.

[0592] Step 7:

[0593] The server returns the analysis results.

[0594] The server then compiles the generated analysis results in JSON format and sends them back to the user's device. The analysis results include an overview of the code, suggestions for improvement, and comments appropriate to the user's feelings.

[0595] Step 8:

[0596] The user terminal receives the analysis results.

[0597] A background script on the user terminal receives the analysis results returned from the server and transfers the results to the content script.

[0598] Step 9:

[0599] The user terminal displays the analysis results.

[0600] The content script displays the received analysis results in a prominent location on the pull request page. For example, it displays the analysis results for each code change so that users can understand them immediately. This display method takes user emotions into consideration, allowing users to review in a relaxed atmosphere.

[0601] Example 2

[0602] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0603] Conventional code review systems are limited to analyzing code and suggesting improvements, and are unable to provide feedback that takes into account the user's emotional state. This can lead to stress and make it difficult for users to accept the review results. Furthermore, many systems have difficulty acquiring and analyzing code differences, requiring significant effort to achieve efficient code reviews. Therefore, there is a need for a system that provides appropriate feedback based on the user's emotional state and allows for efficient and effective code reviews.

[0604] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring code differences on a web page using an extension installed on a user terminal, means for transmitting the acquired code difference data to an analysis server, means for analyzing the code difference data using a generative AI model in the analysis server and generating a code outline and improvements, means for returning the generated analysis results to the user terminal, means for detecting the user's emotional state using an emotion recognition engine, means for adjusting the display method of the analysis results depending on the emotional state, and means for displaying the analysis results on a web page in the user terminal. This enables feedback that takes the user's emotional state into consideration, resulting in efficient and effective code reviews.

[0605] "User terminal" refers to a computing device that can connect to the Internet and that can access websites using a web browser.

[0606] An "extension" refers to a program module that is installed in a particular web browser and provides specific additional functionality on a web page.

[0607] "Code differences" are data that indicate changes to source code in a version control system.

[0608] The term "analysis server" refers to a computer device that has the function of processing and analyzing received code difference data.

[0609] A "generative AI model" is an artificial intelligence model that is trained in advance using large amounts of data, and then analyzes code and suggests improvements.

[0610] An "emotion recognition engine" refers to software or hardware that analyzes a user's input device or camera footage to detect the user's emotional state.

[0611] "Analysis results" refers to data that includes an overview of the code generated by the generative AI model and areas for improvement.

[0612] "Adjusting the display method" refers to changing the way the analysis results are presented depending on the user's emotional state.

[0613] User terminal

[0614] The user terminal is a computing device that can connect to the Internet and can access web pages using a web browser. The user terminal includes an extension and an emotion recognition engine installed in a specific web browser. The extension is activated, for example, when the user opens a pull request page in a version control system.

[0615] Get code diff

[0616] The extension analyzes the DOM structure of a web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text data.

[0617] Analysis Server

[0618] The code difference data acquired by the user's device is sent to the analysis server via HTTP protocol. The analysis server inputs the received data into a generative AI model to generate an outline of the code and improvements. This generative AI model is a model that has been trained in advance on a large code dataset.

[0619] Generative AI Models

[0620] Based on the input code difference data, the generative AI model detects specific error patterns and suggests improvements to improve code efficiency, such as "This function lacks error handling when connecting to a database."

[0621] Emotion Recognition Engine

[0622] The emotion recognition engine on the user device analyzes the user's input device and camera footage to recognize the user's emotional state in real time. Types of emotions include joy, anger, sadness, surprise, etc., and the way the analysis results are displayed is adjusted based on these.

[0623] Return and display of analysis results

[0624] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately by the extension's content script. The display method is adjusted based on the user's emotional state detected by the emotion recognition engine. For example, if the user is under stress, the analysis results will be displayed in a softer tone to reduce the user's psychological burden.

[0625] Specific examples

[0626] When a user opens a pull request

[0627] When a user opens a GitHub pull request page, the extension activates and automatically retrieves the code diffs within the webpage, which are then sent to an analytics server for analysis by the generative AI model.

[0628] Emotion Engine Operation

[0629] While the user is reviewing the pull request, the emotion engine analyzes the user's facial expressions from the camera footage and recognizes their emotions. For example, if the user is feeling anxious about the code review, the analysis results will be displayed in a reassuring tone to alleviate the user's anxiety.

[0630] Displaying analysis results

[0631] The analysis results include specific suggestions such as "This code lacks error handling in the part that establishes a database connection. Please add error handling," as well as comments that take the user's feelings into consideration. Users can use this feedback to make corrections to their code.

[0632] Prompt Sentence Examples

[0633] "Please analyze this code diff and suggest the best improvements."

[0634] "If the user is under high stress, please display the analysis results in a relaxing tone."

[0635] If you find a specific error pattern, please suggest ways to improve it.

[0636] In this way, the system according to the present invention can improve the efficiency of code reviews and enhance the quality of reviews. Furthermore, by utilizing an emotion recognition engine, feedback that takes into account the user's emotional state can be provided, thereby improving the user experience.

[0637] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0638] Step 1:

[0639] When a user opens a pull request page in a version control system using a web browser, the extension is automatically activated and monitors the page load. The input is the user accessing the page, and the output is the extension's launch. Specifically, the extension waits for the page's DOM to be fully loaded.

[0640] Step 2:

[0641] The extension analyzes the DOM structure of a web page to identify code differences. This involves using the class names and IDs of HTML elements to find elements that contain code differences. The input is the DOM structure of the page, and the output is the text data of the code differences. Specifically, the extension uses JavaScript methods such as querySelector to extract specific elements.

[0642] Step 3:

[0643] The user device converts the acquired code difference text data into JSON format and sends it to the analysis server as an HTTP POST request. The input is the code difference text data, and the output is a request sent to the analysis server. Specifically, the user device uses the fetch API to send the data to the specified URL.

[0644] Step 4:

[0645] The analysis server analyzes the received code difference data. First, it inputs the data into a generative AI model to analyze the code. The input is JSON data of the code difference, and the output is the generation of analysis results. Specifically, the server invokes an AI model (for example, a model using Python's TensorFlow or PyTorch) to generate an overview of the code and areas for improvement.

[0646] Step 5:

[0647] The emotion recognition engine analyzes the camera images and input device data on the user's device to recognize the user's emotional state in real time. The input is the camera images and input device data, and the output is data on the user's emotional state. Specifically, the emotion recognition engine uses OpenCV and deep learning frameworks to analyze facial expressions.

[0648] Step 6:

[0649] The analysis server returns the generated analysis results in JSON format to the user device. At the same time, it also returns instructions to adjust the display method based on the user's emotional state. The input is the analysis results and the user's emotional state data, and the output is response data in JSON format. Specifically, the server returns the analysis results as an HTTP response.

[0650] Step 7:

[0651] The user device displays the received analysis results on a web page. The display method is adjusted according to the user's emotional state. The input is the JSON data of the analysis results, and the output is the user's display screen. Specifically, the extension performs DOM manipulation to appropriately display the analysis results. For example, it uses the innerHTML method to insert the analysis results into HTML elements and adjust the display style.

[0652] In this way, the system acts automatically in response to user actions, improving the efficiency of code review and the user experience.

[0653] (Application example 2)

[0654] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0655] Conventional code review systems have limited means for efficiently analyzing code difference data, making it difficult to provide appropriate feedback based on the user's emotional state. Furthermore, code reviews can sometimes cause stress, so measures to improve the user experience are needed.

[0656] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring code differences on a web page using an extension installed on a user terminal, means for transmitting the acquired code difference data to an analysis server, means for analyzing the code difference data using a generative AI model and generating a code outline and improvements in the analysis server, means for adjusting the display method of the analysis results using an emotion engine that recognizes the user's emotional state, means for returning the generated analysis results to the user terminal, and means for displaying the analysis results on a web page in the user terminal. This makes it possible to improve the efficiency of code review while providing appropriate feedback tailored to the user's emotional state.

[0657] A "user terminal" is a computing device connected to the Internet that can access websites using a web browser.

[0658] An "extension" is a software component installed in a web browser to perform specific tasks in order to extend the functionality of the browser.

[0659] A "web page" is a page of information accessible via the Internet, the content of which is written in a standard language such as HTML.

[0660] A "code delta" refers to a change in software code, typically representing the difference between old and new source code in a version control system.

[0661] An "analysis server" is a remote computing device that processes and analyzes received data to generate results.

[0662] A "generative AI model" is an artificial intelligence model that has been pre-trained using machine learning and deep learning techniques to perform analysis and predictions based on input data.

[0663] The "emotion engine" is a system that analyzes data obtained from the user's input device and camera footage to recognize the user's emotions.

[0664] "User's emotional state" refers to the user's mental and emotional state, including emotions such as joy, anger, sadness, surprise, etc.

[0665] "Adjusting the display method" refers to changing the way the analysis results are presented based on the user's emotional state.

[0666] The "DOM structure of a web page" is the document object model of a web page, which is a tree-like representation of the logical structure of an HTML or XML document.

[0667] "Text data" refers to data in the form of code or strings that can be analyzed and processed.

[0668] User terminal

[0669] The user terminal is a computing device connected to the Internet and capable of accessing websites using a web browser. The user terminal has installed thereon the extension and emotion engine described in the claims. The extension operates when the user opens a pull request page in the version control system.

[0670] Get code diff

[0671] The extension installed on the user's device analyzes the DOM structure of the web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text. The code difference data is then compiled as data for later analysis on the server.

[0672] Sending to the analysis server

[0673] The code diff data is sent to a server API endpoint that is designed to accept POST requests over the HTTP protocol and parse the diff data.

[0674] Analysis on the server

[0675] The analysis server inputs the received code difference data into a generative AI model. The generative AI model uses pre-trained parameters to generate an outline of the code and suggestions for improvement. For example, it can detect specific patterns of errors or suggest optimal methods for improving code efficiency.

[0676] Recognizing user emotions with an emotion engine

[0677] The emotion engine analyzes data obtained from the user's input device and camera footage to recognize the user's emotions. The emotion engine can capture the user's emotional state in real time while viewing the pull request page. Emotions include joy, anger, sadness, surprise, etc.

[0678] Return and display of analysis results

[0679] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately on a web page by the extension's content script. The display method is adjusted based on the user's emotional state, as recognized by the emotion engine. For example, if the user is in a state of high stress, the analysis results can be presented in a soft tone, guiding the user into a relaxed state.

[0680] Specific examples

[0681] Example 1: Recommendations based on product page emotional state

[0682] If the emotion engine detects excitement while a user is viewing a particular product page:

[0683] "Want to know more about this product? Click here for more information!"

[0684] Input prompt example

[0685] "Show me a Python script that recognizes a user's emotional state while browsing a product page and serves them a message tailored to that emotion."

[0686] In this way, each component, such as the server, user terminal, and emotion engine, works in cooperation to improve the user experience while enabling efficient code review.

[0687] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0688] Step 1:

[0689] When a user opens a pull request page, an extension installed on the user's device is activated. The extension analyzes the DOM structure of the web page and identifies code differences. The HTML content of the web page is used as input, and the output is text data containing the code differences. This text data is then compiled in a format that can be sent to the server.

[0690] Step 2:

[0691] The user device sends the acquired code difference data to the analysis server's API endpoint via an HTTP POST request. The input is the code difference text data obtained in step 1, and the output is the code difference data in JSON format that the server receives.

[0692] Step 3:

[0693] The analysis server processes the received code difference data and inputs it into the generative AI model. The input is the code difference data sent in step 2, and the output is the analysis results from the generative AI model. The generative AI model generates a code outline and improvements using pre-trained parameters.

[0694] Step 4:

[0695] The server returns the analysis results in JSON format to the user device. The input is the analysis results from the generative AI model, and the output is the JSON data of the analysis results received by the user device. This data includes improvement suggestions and error detection information.

[0696] Step 5:

[0697] The user device receives the analysis results and simultaneously activates the emotion engine to recognize the user's emotional state in real time. The input is camera footage capturing the user's facial expressions and data from input devices, and the output is the user's emotional state (joy, anger, sadness, surprise, etc.).

[0698] Step 6:

[0699] Based on the user's emotional state output by the emotion engine, the system selects appropriate templates and tones to adjust the display method. The input is the user's emotional state and the analysis results from the server, and the output is the display content of the adjusted analysis results.

[0700] Step 7:

[0701] The user terminal displays the adjusted analysis results on a web page. The input is the display content adjusted in step 6, and the output is feedback that is easy for the user to see and causes little stress to the user.

[0702] Through these steps, users can receive efficient code reviews and receive feedback tailored to their emotional state, improving the user experience.

[0703] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0704] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0705] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0706] [Third embodiment]

[0707] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0708] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0709] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0710] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0711] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0712] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0713] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0714] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0715] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0716] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0717] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0718] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0719] This invention is a system that uses an extension installed on a user's device to acquire code differences on a web page and transmits the difference data to an analysis server, thereby enabling efficient code review using an AI model.

[0720] Specific embodiments

[0721] User terminal

[0722] The user terminal is a computing device connected to the Internet and capable of accessing websites using a web browser. The user terminal has installed thereon the extension described in the patent claims. This extension is activated when the user opens a pull request page in a version control system such as GitHub.

[0723] Get code diff

[0724] The extension installed on the user's device analyzes the DOM structure of the web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text. The code difference data is then compiled as data for later analysis on the server.

[0725] Sending to the analysis server

[0726] The code difference data obtained on the user's device is sent to the server's API endpoint, which is designed to receive POST requests via the HTTP protocol and analyze the difference data.

[0727] Analysis on the server

[0728] The analysis server inputs the received code difference data into the AI ​​model, which uses pre-trained parameters to generate an outline of the code and suggest improvements. For example, it can detect specific patterns of errors or suggest optimal methods for improving code efficiency.

[0729] Return and display of analysis results

[0730] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately on a web page by the extension. Specifically, explanations and suggestions are displayed in a prominent position on the pull request page, allowing the user to immediately check the analysis results and make any necessary corrections.

[0731] Specific examples

[0732] When a user opens a pull request

[0733] When a user opens a GitHub pull request page, the extension activates and automatically retrieves the code diffs within the page, which are then sent to a server for analysis by an AI model.

[0734] Displaying analysis results

[0735] The analysis results include specific suggestions, such as "This code lacks error handling in the part that establishes a database connection. Please add error handling." Users can use this as a reference to correct their code.

[0736] In this way, the system according to the present invention can improve the efficiency of code reviews and enhance the quality of reviews, allowing engineers to receive immediate feedback and contributing to improving the quality of products.

[0737] The processing flow will be explained below.

[0738] Step 1:

[0739] A user opens the pull request page.

[0740] When a user opens a pull request page in a version control system, the extension installed in the browser is activated.

[0741] Step 2:

[0742] The device will get the code diff.

[0743] The extension installed on the user's device analyzes the DOM structure of the web page, identifies the code difference elements, and retrieves them in text format. Specifically, it uses the class name or ID of the element to find the code difference parts and extracts their contents as text.

[0744] Step 3:

[0745] The terminal transmits the code difference data to the analysis server.

[0746] The device's background script sends the acquired code difference data to the server's API endpoint as a POST request. The request includes the difference data in JSON format.

[0747] Step 4:

[0748] The server receives the differential data.

[0749] The analysis server receives the POST request and extracts the code difference data from the request body.

[0750] Step 5:

[0751] The server analyzes the differential data using an AI model.

[0752] The server passes the extracted differential data to the AI ​​model to begin analysis. The AI ​​model generates an overview of the code and points to improve. For example, it might generate a message like, "This code has the ability to establish a database connection, but it lacks error handling."

[0753] Step 6:

[0754] The server returns the analysis results.

[0755] The server compiles the generated analysis results into JSON format and returns them to the user's device.

[0756] Step 7:

[0757] The terminal receives the analysis results.

[0758] The user's device receives the analysis results returned from the server and forwards them to the content script of the extension.

[0759] Step 8:

[0760] The device displays the analysis results.

[0761] The extension's content script prominently displays the analysis results on the pull request page, either at the top of the page or in the comments section, allowing the user to review the results and make code changes.

[0762] Example 1

[0763] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0764] Conventional code reviews are often performed manually, which can be time-consuming and labor-intensive. Furthermore, the quality of the review depends on the reviewer's experience and skill, making it difficult to maintain a consistent level of quality. The present invention aims to solve these problems and provide a system that improves the efficiency and quality of code reviews.

[0765] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0766] In this invention, the server includes: means for acquiring code differences on a web page using an extension installed on a user terminal; means for transmitting the acquired code difference data to an analysis server; means for analyzing the code difference data using a generative AI model in the analysis server and generating a code overview and improvements; means for returning the generated analysis results to the user terminal; means for using a web browser interconnected to the user terminal via the Internet; means for sending the acquired difference data via an HTTP POST request; and means for returning the analysis results in JSON format. This enables automated and efficient code reviews and rapid, high-quality feedback.

[0767] A "user terminal" is a computing device that can connect to the Internet and access web pages using a web browser.

[0768] An "extension" is plug-in software that is added to a web browser to enhance specific functions.

[0769] A "code difference" is a difference that occurs when comparing code before and after a change.

[0770] An "analysis server" is a computer device that processes and analyzes received data.

[0771] A "generative AI model" is an artificial intelligence model that is pre-trained using machine learning algorithms to output optimal results for a specific task.

[0772] An "HTTP POST request" is a request format for sending data from a client to a server using the HTTP protocol.

[0773] The "JSON format" is a data exchange format that expresses data in object format and can be transferred as text.

[0774] The "DOM structure of a web page" refers to a document object model that represents the hierarchical structure of an HTML or XML document on a web page.

[0775] A "prompt" is an input sentence that gives instructions to an AI model and is used to specify the conditions and requirements for analysis.

[0776] "User needs" are the requirements and desires of the user, and are the criteria for the system to provide customization in response to these requirements.

[0777] The present invention is a system that realizes efficient code review using a generative AI model by using an extension installed on a user's device to obtain code differences on a web page and send the difference data to an analysis server.

[0778] User terminal

[0779] A user device is a computing device connected to the Internet and capable of accessing websites using a web browser (e.g., Google Chrome). A specific extension is installed on the user device. This extension is activated when the user opens a pull request page in a version control system (e.g., GitHub).

[0780] Get code diff

[0781] The extension installed on the user's device analyzes the DOM structure of the web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text. The obtained code differences are then compiled as data for later analysis on the server.

[0782] Sending to the analysis server

[0783] The code difference data acquired on the user's device is sent to the analysis server's API endpoint using an HTTP POST request, which can receive POST requests via the HTTP protocol.

[0784] Analysis on the server

[0785] The analysis server deserializes the received code diff data and feeds it into a generative AI model, which uses pre-trained parameters to generate an outline of the code and suggest improvements, such as detecting specific patterns of errors or suggesting optimal techniques for improving code efficiency.

[0786] Return and display of analysis results

[0787] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately on a web page by the extension. Specifically, explanations and suggestions are displayed in a prominent position on the pull request page, allowing the user to immediately check the analysis results and make any necessary corrections.

[0788] Specific examples

[0789] When a user opens a pull request

[0790] When a user opens a pull request page in a version control system, the extension activates and automatically retrieves the code diffs within the page, which are then sent to a server for analysis by a generative AI model.

[0791] Example prompt:

[0792] "What code diff process happens when a user opens a pull request page?"

[0793] Displaying analysis results

[0794] The analysis results include specific suggestions, such as "This code lacks error handling in the part that establishes a database connection. Please add error handling." Users can use this feedback to correct their code.

[0795] Example prompt:

[0796] "Please explain the process by which the analysis server inputs the code difference data received into the AI ​​model for analysis."

[0797] The present invention can significantly improve the efficiency and quality of code reviews, allowing engineers to receive immediate feedback and contributing to improved product quality.

[0798] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0799] Step 1:

[0800] Installing and launching the extension

[0801] A user installs Google Chrome on their computer and adds a specific extension that runs when they open a pull request page in a version control system (e.g., GitHub).

[0802] Input: Web browser, extension file

[0803] Output: Installed extensions

[0804] What happens: A user searches for an extension in the Chrome Web Store and clicks the "Install" button. After installation, the extension's icon appears in the browser toolbar.

[0805] Step 2:

[0806] Get code diff

[0807] When a user opens a pull request page in a version control system, the extension installed on the device analyzes the DOM structure of the page and extracts text data from multiple code difference elements.

[0808] Input: URL of the pull request page

[0809] Output: Extracted code diff data (text format)

[0810] What it does: The extension looks for HTML elements with specific class names or IDs on the pull request page, extracts the text data of the code diffs from those elements, and converts it to JSON format.

[0811] Step 3:

[0812] Sending differential data to the server

[0813] The device sends the acquired code difference data to the analysis server's API endpoint using an HTTP POST request.

[0814] Input: Code diff data (JSON format)

[0815] Output: POST request to the server

[0816] Specific operation: The differential data obtained on the terminal is serialized in JSON format, included in the body of the HTTP POST request, and sent to the API endpoint.

[0817] Step 4:

[0818] Data analysis on the server

[0819] The server deserializes the received code diff data and feeds it into a generative AI model, which uses pre-trained parameters to generate an outline of the code and suggests improvements.

[0820] Input: Code diff data (JSON format)

[0821] Output: Analysis results (code summary and improvements)

[0822] Specific operation: When the server receives a POST request, it parses the JSON data in the request body and converts it into an input format for the AI ​​model. The model then analyzes it and generates a recommendation result.

[0823] Step 5:

[0824] Return and display of analysis results

[0825] The server returns the generated analysis results in JSON format to the user's device, which then deserializes the results and displays them on the pull request page.

[0826] Input: Analysis result (JSON format)

[0827] Output: Analysis results displayed on the pull request page

[0828] Specific operation: Once the analysis process on the server is complete, the results are formatted as JSON and sent as an HTTP response to the terminal. The terminal parses the JSON data and embeds the results in the HTML of the pull request page.

[0829] This step provides a concrete understanding of the overall system behavior and allows users to receive a fast, high-quality code review.

[0830] (Application example 1)

[0831] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0832] Robot operation and software updates are frequently performed in factories. However, the code reviews and error detection that accompany these updates require a great deal of time and effort, creating a need for greater efficiency. Conventional methods require personnel to manually review the code and identify problems, which can lead to human error and oversight. Real-time feedback is also difficult, and suggestions for improvements are often delayed. This risks reducing the efficiency and safety of robot operation. A system that can resolve these issues and streamline software management for factory robots is needed.

[0833] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0834] In this invention, the server includes means for acquiring code differences on a web page using an extension installed on a user terminal, means for transmitting the acquired code difference data to an analysis server, means for analyzing the code difference data using an AI model in the analysis server and generating a code overview and improvements, means for returning the generated analysis results to the user terminal, means for displaying the analysis results on a web page in the user terminal, means for acquiring code differences in factory robot software and transmitting them to the analysis server, means for analyzing the acquired code differences using an AI model and proposing improvements to the efficiency and safety of robot operation, and means for returning the generated proposal results to the user terminal and displaying them. This enables efficient and accurate code reviews of factory robot software and enables real-time feedback and suggestions for improvements.

[0835] A "user terminal" is a computing device that can connect to the Internet using a web browser and that has advanced features installed.

[0836] An "extension" is a software program that is added to a web browser on a user's device to provide specific functionality.

[0837] "Code differences" are text data of parts of the software version history where changes or additions have been made.

[0838] The "analysis server" is a computer system that receives code difference data sent from a user terminal and analyzes it using an AI model.

[0839] An "AI model" is an artificial intelligence system that uses pre-trained algorithms to analyze input data and perform specific tasks.

[0840] The "generated analysis results" are information about the code overview and areas for improvement obtained after the AI ​​model analyzes the code difference data.

[0841] A "factory robot" is a mechanical device used to perform automated tasks in a factory.

[0842] "Software management" refers to the overall process of software version control, updates, maintenance, and review.

[0843] The "DOM structure of a web page" is an object model that shows the HTML elements of a web page and their hierarchical structure.

[0844] "Text data" is data that is expressed in the form of a string in a computer system.

[0845] This invention is a system for improving the efficiency of software management for factory robots, and is composed of a user terminal, an extension function, an analysis server, and an AI model. Specific embodiments using various hardware and software are described below.

[0846] User terminal

[0847] The user device is a computing device such as a smartphone or PC that can connect to the Internet. An extension is installed in the web browser on the user device to perform software updates and code reviews for factory robots. When a user opens a page containing code differences, the extension runs, analyzes the DOM structure of the web page, and obtains the code difference data.

[0848] Extensions

[0849] The extension is a software program added to the user's web browser. The extension uses the class names and IDs of HTML elements to identify code differences and extracts text data from multiple code difference elements. The code difference data is then sent to the server.

[0850] Analysis Server

[0851] The analysis server is a computer system located on the cloud or on-premise. To receive code difference data sent from user terminals, an API endpoint is provided via the HTTP protocol. The server inputs the received code difference data into an AI model for analysis.

[0852] AI model

[0853] The AI ​​model is an AI system that uses pre-trained algorithms to generate code summaries and improvements. The server uses the AI ​​model to analyze code diff data, detect specific patterns of errors, and suggest optimal methods for improving efficiency. The analysis results are returned to the user's device in JSON format.

[0854] Displaying analysis results

[0855] The analysis results are returned to the user's device and displayed on a web page by the extension, providing feedback on improvements to the software code and detecting errors in a user-friendly format.

[0856] Examples of specific examples and prompts

[0857] For example, the AI ​​model may generate suggestions like the following for code differences obtained when updating a robot's firmware.

[0858] For example: "This code is missing error handling where it establishes a database connection. Please add error handling."

[0859] An example of a prompt sentence to input to the generative AI model is as follows:

[0860] Please analyze the code diff below and suggest improvements:

[0861] Diff: {code_diff}

[0862] example:

[0863] Suggestion: Deprecated API is being used. Please use the latest API.

[0864] This invention enables efficient and accurate software management of factory robots, facilitating early error detection and correction. Real-time feedback and specific improvement suggestions improve the safety and efficiency of robot operations.

[0865] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0866] Step 1:

[0867] When a user opens the factory robot software update page, an extension on the user's device runs and automatically retrieves the code differences within the page. Specifically, the extension analyzes the DOM structure of the web page and identifies the code differences based on the class names and IDs of HTML elements. This analysis retrieves the code differences as text data.

[0868] Input: DOM structure of the web page

[0869] Output: Text data of code diff

[0870] Step 2:

[0871] The code difference data acquired on the user device is sent to the analysis server via the HTTP protocol. Specifically, the extension on the user device sends a POST request to the API endpoint and sends the acquired code difference data to the server in JSON format.

[0872] Input: Text data of code diff

[0873] Output: POST request to the server

[0874] Step 3:

[0875] The analysis server inputs the received code difference data into an AI model. The server uses the pre-trained AI model to analyze the code overview and areas for improvement. For example, it detects errors of specific patterns and suggests optimal methods for improving efficiency.

[0876] Input: Text data of the code diff sent to the server

[0877] Output: Analysis results by the AI ​​model (code summary and improvements)

[0878] Step 4:

[0879] The generated analysis results are returned to the user terminal in JSON format. The analysis server returns the analysis results as a response using the HTTP protocol.

[0880] Input: Analysis results from AI model

[0881] Output: JSON response to the user's device

[0882] Step 5:

[0883] The analysis results returned to the user's device are displayed on the web page by the extension. Specifically, the extension analyzes the analysis results and displays details of improvements and errors in a prominent location on the web page. This display allows the user to immediately check the analysis results and make any necessary corrections.

[0884] Input: Parsed result in JSON format

[0885] Output: Analysis results displayed on a web page

[0886] Step 6:

[0887] Users can then make modifications to the software code based on the analysis results displayed on the webpage. For example, they can follow suggestions from the AI ​​model to fix specific errors or make code changes to improve efficiency. These modifications will make the factory robots operate more safely and efficiently.

[0888] Input: Analysis results displayed on a web page

[0889] Output: User modifications to the software code

[0890] These steps ensure efficient and accurate software management of factory robots, and real-time feedback and specific AI-driven improvement suggestions can be obtained to enhance the safety and efficiency of robot operations.

[0891] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0892] This invention is a system that realizes efficient code review using an AI model by acquiring code differences on a web page using an extension installed on a user's device and sending the difference data to an analysis server. Furthermore, this invention includes a configuration that combines an emotion engine that recognizes the user's emotions, making it possible to adjust the way the analysis results are displayed based on the user's emotions.

[0893] Specific embodiments

[0894] User terminal

[0895] The user terminal is a computing device connected to the Internet and capable of accessing websites using a web browser. The claimed extension and emotion engine are installed on the user terminal. The extension is activated when the user opens a pull request page in a version control system such as GitHub.

[0896] Get code diff

[0897] The extension installed on the user's device analyzes the DOM structure of the web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text. The code difference data is then compiled as data for later analysis on the server.

[0898] Sending to the analysis server

[0899] The code difference data obtained on the user's device is sent to the server's API endpoint, which is designed to receive POST requests via the HTTP protocol and analyze the difference data.

[0900] Analysis on the server

[0901] The analysis server inputs the received code difference data into the AI ​​model, which uses pre-trained parameters to generate an overview of the code and suggestions for improvement. For example, it can detect specific patterns of errors or suggest optimal methods for improving code efficiency.

[0902] Recognizing user emotions with an emotion engine

[0903] The emotion engine analyzes data obtained from the user's input device and camera footage to recognize the user's emotions. The emotion engine can capture the user's emotional state in real time while viewing the pull request page. Emotions include joy, anger, sadness, surprise, etc.

[0904] Return and display of analysis results

[0905] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately on a web page by the extension's content script. The display method is adjusted based on the user's emotional state, as recognized by the emotion engine. For example, if the user is in a state of high stress, the analysis results can be presented in a soft tone, guiding the user into a relaxed state.

[0906] Specific examples

[0907] When a user opens a pull request

[0908] When a user opens a GitHub pull request page, the extension activates and automatically retrieves the code diffs within the page, which are then sent to a server for analysis by an AI model.

[0909] Emotion Engine Operation

[0910] While the user is reviewing the pull request, the emotion engine analyzes the user's facial expressions from the camera footage and recognizes their emotions. For example, if the user is feeling anxious about the code review, the analysis results will be displayed in a reassuring tone to alleviate the user's anxiety.

[0911] Displaying analysis results

[0912] The analysis results include specific suggestions, such as "This code lacks error handling in the part that establishes a database connection. Please add error handling," as well as comments that take the user's feelings into consideration. Users can refer to these and make corrections to their code.

[0913] In this way, the system according to the present invention can improve the efficiency of code reviews and enhance the quality of reviews. Furthermore, the introduction of an emotion engine can improve the user experience by providing feedback that takes into account the user's emotional state.

[0914] The processing flow will be explained below.

[0915] Step 1:

[0916] A user opens the pull request page.

[0917] When a user opens a pull request page in a version control system, the extension installed in the browser is activated.

[0918] Step 2:

[0919] The user terminal acquires the code difference.

[0920] The extension installed on the user's device analyzes the DOM structure of the web page, identifies the code differences, and retrieves them in text format. Specifically, it uses the class names and IDs of the elements to find the code differences and extracts their contents as text.

[0921] Step 3:

[0922] The user terminal transmits the code difference data to the analysis server.

[0923] The device's background script sends the acquired code difference data to the server's API endpoint as a POST request. The request includes the difference data in JSON format.

[0924] Step 4:

[0925] The server receives the differential data.

[0926] The analysis server receives the POST request and extracts the code difference data from the request body.

[0927] Step 5:

[0928] The server analyzes the differential data using an AI model.

[0929] The server inputs the extracted differential data into the AI ​​model and begins analysis. The AI ​​model generates an overview of the code and points to improve. For example, it might generate a message like, "This code has the ability to establish a database connection, but error handling is lacking."

[0930] Step 6:

[0931] The emotion engine of the user terminal recognizes the user's emotion.

[0932] The emotion engine analyzes data obtained from the user's input devices and camera footage to recognize the user's emotions. It identifies emotions such as joy, anger, sadness, and surprise.

[0933] Step 7:

[0934] The server returns the analysis results.

[0935] The server then compiles the generated analysis results in JSON format and sends them back to the user's device. The analysis results include an overview of the code, suggestions for improvement, and comments appropriate to the user's feelings.

[0936] Step 8:

[0937] The user terminal receives the analysis results.

[0938] A background script on the user terminal receives the analysis results returned from the server and transfers the results to the content script.

[0939] Step 9:

[0940] The user terminal displays the analysis results.

[0941] The content script displays the received analysis results in a prominent location on the pull request page. For example, it displays the analysis results for each code change so that users can understand them immediately. This display method takes user emotions into consideration, allowing users to review in a relaxed atmosphere.

[0942] Example 2

[0943] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0944] Conventional code review systems are limited to analyzing code and suggesting improvements, and are unable to provide feedback that takes into account the user's emotional state. This can lead to stress and make it difficult for users to accept the review results. Furthermore, many systems have difficulty acquiring and analyzing code differences, requiring significant effort to achieve efficient code reviews. Therefore, there is a need for a system that provides appropriate feedback based on the user's emotional state and allows for efficient and effective code reviews.

[0945] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring code differences on a web page using an extension installed on a user terminal, means for transmitting the acquired code difference data to an analysis server, means for analyzing the code difference data using a generative AI model in the analysis server and generating a code outline and improvements, means for returning the generated analysis results to the user terminal, means for detecting the user's emotional state using an emotion recognition engine, means for adjusting the display method of the analysis results depending on the emotional state, and means for displaying the analysis results on a web page in the user terminal. This enables feedback that takes the user's emotional state into consideration, resulting in efficient and effective code reviews.

[0946] "User terminal" refers to a computing device that can connect to the Internet and that can access websites using a web browser.

[0947] An "extension" refers to a program module that is installed in a particular web browser and provides specific additional functionality on a web page.

[0948] "Code differences" are data that indicate changes to source code in a version control system.

[0949] The term "analysis server" refers to a computer device that has the function of processing and analyzing received code difference data.

[0950] A "generative AI model" is an artificial intelligence model that is trained in advance using large amounts of data, and then analyzes code and suggests improvements.

[0951] An "emotion recognition engine" refers to software or hardware that analyzes a user's input device or camera footage to detect the user's emotional state.

[0952] "Analysis results" refers to data that includes an overview of the code generated by the generative AI model and areas for improvement.

[0953] "Adjusting the display method" refers to changing the way the analysis results are presented depending on the user's emotional state.

[0954] User terminal

[0955] The user terminal is a computing device that can connect to the Internet and can access web pages using a web browser. The user terminal includes an extension and an emotion recognition engine installed in a specific web browser. The extension is activated, for example, when the user opens a pull request page in a version control system.

[0956] Get code diff

[0957] The extension analyzes the DOM structure of a web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text data.

[0958] Analysis Server

[0959] The code difference data acquired by the user's device is sent to the analysis server via HTTP protocol. The analysis server inputs the received data into a generative AI model to generate an outline of the code and improvements. This generative AI model is a model that has been trained in advance on a large code dataset.

[0960] Generative AI Models

[0961] Based on the input code difference data, the generative AI model detects specific error patterns and suggests improvements to improve code efficiency, such as "This function lacks error handling when connecting to a database."

[0962] Emotion Recognition Engine

[0963] The emotion recognition engine on the user device analyzes the user's input device and camera footage to recognize the user's emotional state in real time. Types of emotions include joy, anger, sadness, surprise, etc., and the way the analysis results are displayed is adjusted based on these.

[0964] Return and display of analysis results

[0965] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately by the extension's content script. The display method is adjusted based on the user's emotional state detected by the emotion recognition engine. For example, if the user is under stress, the analysis results will be displayed in a softer tone to reduce the user's psychological burden.

[0966] Specific examples

[0967] When a user opens a pull request

[0968] When a user opens a GitHub pull request page, the extension activates and automatically retrieves the code diffs within the webpage, which are then sent to an analytics server for analysis by the generative AI model.

[0969] Emotion Engine Operation

[0970] While the user is reviewing the pull request, the emotion engine analyzes the user's facial expressions from the camera footage and recognizes their emotions. For example, if the user is feeling anxious about the code review, the analysis results will be displayed in a reassuring tone to alleviate the user's anxiety.

[0971] Displaying analysis results

[0972] The analysis results include specific suggestions such as "This code lacks error handling in the part that establishes a database connection. Please add error handling," as well as comments that take the user's feelings into consideration. Users can use this feedback to make corrections to their code.

[0973] Prompt Sentence Examples

[0974] "Please analyze this code diff and suggest the best improvements."

[0975] "If the user is under high stress, please display the analysis results in a relaxing tone."

[0976] If you find a specific error pattern, please suggest ways to improve it.

[0977] In this way, the system according to the present invention can improve the efficiency of code reviews and enhance the quality of reviews. Furthermore, by utilizing an emotion recognition engine, feedback that takes into account the user's emotional state can be provided, thereby improving the user experience.

[0978] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0979] Step 1:

[0980] When a user opens a pull request page in a version control system using a web browser, the extension is automatically activated and monitors the page load. The input is the user accessing the page, and the output is the extension's launch. Specifically, the extension waits for the page's DOM to be fully loaded.

[0981] Step 2:

[0982] The extension analyzes the DOM structure of a web page to identify code differences. This involves using the class names and IDs of HTML elements to find elements that contain code differences. The input is the DOM structure of the page, and the output is the text data of the code differences. Specifically, the extension uses JavaScript methods such as querySelector to extract specific elements.

[0983] Step 3:

[0984] The user device converts the acquired code difference text data into JSON format and sends it to the analysis server as an HTTP POST request. The input is the code difference text data, and the output is a request sent to the analysis server. Specifically, the user device uses the fetch API to send the data to the specified URL.

[0985] Step 4:

[0986] The analysis server analyzes the received code difference data. First, it inputs the data into a generative AI model to analyze the code. The input is JSON data of the code difference, and the output is the generation of analysis results. Specifically, the server invokes an AI model (for example, a model using Python's TensorFlow or PyTorch) to generate an overview of the code and areas for improvement.

[0987] Step 5:

[0988] The emotion recognition engine analyzes the camera images and input device data on the user's device to recognize the user's emotional state in real time. The input is the camera images and input device data, and the output is data on the user's emotional state. Specifically, the emotion recognition engine uses OpenCV and deep learning frameworks to analyze facial expressions.

[0989] Step 6:

[0990] The analysis server returns the generated analysis results in JSON format to the user device. At the same time, it also returns instructions to adjust the display method based on the user's emotional state. The input is the analysis results and the user's emotional state data, and the output is response data in JSON format. Specifically, the server returns the analysis results as an HTTP response.

[0991] Step 7:

[0992] The user device displays the received analysis results on a web page. The display method is adjusted according to the user's emotional state. The input is the JSON data of the analysis results, and the output is the user's display screen. Specifically, the extension performs DOM manipulation to appropriately display the analysis results. For example, it uses the innerHTML method to insert the analysis results into HTML elements and adjust the display style.

[0993] In this way, the system acts automatically in response to user actions, improving the efficiency of code review and the user experience.

[0994] (Application example 2)

[0995] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0996] Conventional code review systems have limited means for efficiently analyzing code difference data, making it difficult to provide appropriate feedback based on the user's emotional state. Furthermore, code reviews can sometimes cause stress, so measures to improve the user experience are needed.

[0997] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring code differences on a web page using an extension installed on a user terminal, means for transmitting the acquired code difference data to an analysis server, means for analyzing the code difference data using a generative AI model and generating a code outline and improvements in the analysis server, means for adjusting the display method of the analysis results using an emotion engine that recognizes the user's emotional state, means for returning the generated analysis results to the user terminal, and means for displaying the analysis results on a web page in the user terminal. This makes it possible to improve the efficiency of code review while providing appropriate feedback tailored to the user's emotional state.

[0998] A "user terminal" is a computing device connected to the Internet that can access websites using a web browser.

[0999] An "extension" is a software component installed in a web browser to perform specific tasks in order to extend the functionality of the browser.

[1000] A "web page" is a page of information accessible via the Internet, the content of which is written in a standard language such as HTML.

[1001] A "code delta" refers to a change in software code, typically representing the difference between old and new source code in a version control system.

[1002] An "analysis server" is a remote computing device that processes and analyzes received data to generate results.

[1003] A "generative AI model" is an artificial intelligence model that has been pre-trained using machine learning and deep learning techniques to perform analysis and predictions based on input data.

[1004] The "emotion engine" is a system that analyzes data obtained from the user's input device and camera footage to recognize the user's emotions.

[1005] "User's emotional state" refers to the user's mental and emotional state, including emotions such as joy, anger, sadness, surprise, etc.

[1006] "Adjusting the display method" refers to changing the way the analysis results are presented based on the user's emotional state.

[1007] The "DOM structure of a web page" is the document object model of a web page, which is a tree-like representation of the logical structure of an HTML or XML document.

[1008] "Text data" refers to data in the form of code or strings that can be analyzed and processed.

[1009] User terminal

[1010] The user terminal is a computing device connected to the Internet and capable of accessing websites using a web browser. The user terminal has installed thereon the extension and emotion engine described in the claims. The extension operates when the user opens a pull request page in the version control system.

[1011] Get code diff

[1012] The extension installed on the user's device analyzes the DOM structure of the web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text. The code difference data is then compiled as data for later analysis on the server.

[1013] Sending to the analysis server

[1014] The code diff data is sent to a server API endpoint that is designed to accept POST requests over the HTTP protocol and parse the diff data.

[1015] Analysis on the server

[1016] The analysis server inputs the received code difference data into a generative AI model. The generative AI model uses pre-trained parameters to generate an outline of the code and suggestions for improvement. For example, it can detect specific patterns of errors or suggest optimal methods for improving code efficiency.

[1017] Recognizing user emotions with an emotion engine

[1018] The emotion engine analyzes data obtained from the user's input device and camera footage to recognize the user's emotions. The emotion engine can capture the user's emotional state in real time while viewing the pull request page. Emotions include joy, anger, sadness, surprise, etc.

[1019] Return and display of analysis results

[1020] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately on a web page by the extension's content script. The display method is adjusted based on the user's emotional state, as recognized by the emotion engine. For example, if the user is in a state of high stress, the analysis results can be presented in a soft tone, guiding the user into a relaxed state.

[1021] Specific examples

[1022] Example 1: Recommendations based on product page emotional state

[1023] If the emotion engine detects excitement while a user is viewing a particular product page:

[1024] "Want to know more about this product? Click here for more information!"

[1025] Input prompt example

[1026] "Show me a Python script that recognizes a user's emotional state while browsing a product page and serves them a message tailored to that emotion."

[1027] In this way, each component, such as the server, user terminal, and emotion engine, works in cooperation to improve the user experience while enabling efficient code review.

[1028] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1029] Step 1:

[1030] When a user opens a pull request page, an extension installed on the user's device is activated. The extension analyzes the DOM structure of the web page and identifies code differences. The HTML content of the web page is used as input, and the output is text data containing the code differences. This text data is then compiled in a format that can be sent to the server.

[1031] Step 2:

[1032] The user device sends the acquired code difference data to the analysis server's API endpoint via an HTTP POST request. The input is the code difference text data obtained in step 1, and the output is the code difference data in JSON format that the server receives.

[1033] Step 3:

[1034] The analysis server processes the received code difference data and inputs it into the generative AI model. The input is the code difference data sent in step 2, and the output is the analysis results from the generative AI model. The generative AI model generates a code outline and improvements using pre-trained parameters.

[1035] Step 4:

[1036] The server returns the analysis results in JSON format to the user device. The input is the analysis results from the generative AI model, and the output is the JSON data of the analysis results received by the user device. This data includes improvement suggestions and error detection information.

[1037] Step 5:

[1038] The user device receives the analysis results and simultaneously activates the emotion engine to recognize the user's emotional state in real time. The input is camera footage capturing the user's facial expressions and data from input devices, and the output is the user's emotional state (joy, anger, sadness, surprise, etc.).

[1039] Step 6:

[1040] Based on the user's emotional state output by the emotion engine, the system selects appropriate templates and tones to adjust the display method. The input is the user's emotional state and the analysis results from the server, and the output is the display content of the adjusted analysis results.

[1041] Step 7:

[1042] The user terminal displays the adjusted analysis results on a web page. The input is the display content adjusted in step 6, and the output is feedback that is easy for the user to see and causes little stress to the user.

[1043] Through these steps, users can receive efficient code reviews and receive feedback tailored to their emotional state, improving the user experience.

[1044] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1045] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1046] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1047] [Fourth embodiment]

[1048] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1049] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1050] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1051] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1052] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1053] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1054] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1055] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1056] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1057] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1058] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1059] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1060] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1061] This invention is a system that uses an extension installed on a user's device to acquire code differences on a web page and transmits the difference data to an analysis server, thereby enabling efficient code review using an AI model.

[1062] Specific embodiments

[1063] User terminal

[1064] The user terminal is a computing device connected to the Internet and capable of accessing websites using a web browser. The user terminal has installed thereon the extension described in the patent claims. This extension is activated when the user opens a pull request page in a version control system such as GitHub.

[1065] Get code diff

[1066] The extension installed on the user's device analyzes the DOM structure of the web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text. The code difference data is then compiled as data for later analysis on the server.

[1067] Sending to the analysis server

[1068] The code difference data obtained on the user's device is sent to the server's API endpoint, which is designed to receive POST requests via the HTTP protocol and analyze the difference data.

[1069] Analysis on the server

[1070] The analysis server inputs the received code difference data into the AI ​​model, which uses pre-trained parameters to generate an outline of the code and suggest improvements. For example, it can detect specific patterns of errors or suggest optimal methods for improving code efficiency.

[1071] Return and display of analysis results

[1072] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately on a web page by the extension. Specifically, explanations and suggestions are displayed in a prominent position on the pull request page, allowing the user to immediately check the analysis results and make any necessary corrections.

[1073] Specific examples

[1074] When a user opens a pull request

[1075] When a user opens a GitHub pull request page, the extension activates and automatically retrieves the code diffs within the page, which are then sent to a server for analysis by an AI model.

[1076] Displaying analysis results

[1077] The analysis results include specific suggestions, such as "This code lacks error handling in the part that establishes a database connection. Please add error handling." Users can use this as a reference to correct their code.

[1078] In this way, the system according to the present invention can improve the efficiency of code reviews and enhance the quality of reviews, allowing engineers to receive immediate feedback and contributing to improving the quality of products.

[1079] The processing flow will be explained below.

[1080] Step 1:

[1081] A user opens the pull request page.

[1082] When a user opens a pull request page in a version control system, the extension installed in the browser is activated.

[1083] Step 2:

[1084] The device will get the code diff.

[1085] The extension installed on the user's device analyzes the DOM structure of the web page, identifies the code difference elements, and retrieves them in text format. Specifically, it uses the class name or ID of the element to find the code difference parts and extracts their contents as text.

[1086] Step 3:

[1087] The terminal transmits the code difference data to the analysis server.

[1088] The device's background script sends the acquired code difference data to the server's API endpoint as a POST request. The request includes the difference data in JSON format.

[1089] Step 4:

[1090] The server receives the differential data.

[1091] The analysis server receives the POST request and extracts the code difference data from the request body.

[1092] Step 5:

[1093] The server analyzes the differential data using an AI model.

[1094] The server passes the extracted differential data to the AI ​​model to begin analysis. The AI ​​model generates an overview of the code and points to improve. For example, it might generate a message like, "This code has the ability to establish a database connection, but it lacks error handling."

[1095] Step 6:

[1096] The server returns the analysis results.

[1097] The server compiles the generated analysis results into JSON format and returns them to the user's device.

[1098] Step 7:

[1099] The terminal receives the analysis results.

[1100] The user's device receives the analysis results returned from the server and forwards them to the content script of the extension.

[1101] Step 8:

[1102] The device displays the analysis results.

[1103] The extension's content script prominently displays the analysis results on the pull request page, either at the top of the page or in the comments section, allowing the user to review the results and make code changes.

[1104] Example 1

[1105] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1106] Conventional code reviews are often performed manually, which can be time-consuming and labor-intensive. Furthermore, the quality of the review depends on the reviewer's experience and skill, making it difficult to maintain a consistent level of quality. The present invention aims to solve these problems and provide a system that improves the efficiency and quality of code reviews.

[1107] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1108] In this invention, the server includes: means for acquiring code differences on a web page using an extension installed on a user terminal; means for transmitting the acquired code difference data to an analysis server; means for analyzing the code difference data using a generative AI model in the analysis server and generating a code overview and improvements; means for returning the generated analysis results to the user terminal; means for using a web browser interconnected to the user terminal via the Internet; means for sending the acquired difference data via an HTTP POST request; and means for returning the analysis results in JSON format. This enables automated and efficient code reviews and rapid, high-quality feedback.

[1109] A "user terminal" is a computing device that can connect to the Internet and access web pages using a web browser.

[1110] An "extension" is plug-in software that is added to a web browser to enhance specific functions.

[1111] A "code difference" is a difference that occurs when comparing code before and after a change.

[1112] An "analysis server" is a computer device that processes and analyzes received data.

[1113] A "generative AI model" is an artificial intelligence model that is pre-trained using machine learning algorithms to output optimal results for a specific task.

[1114] An "HTTP POST request" is a request format for sending data from a client to a server using the HTTP protocol.

[1115] The "JSON format" is a data exchange format that expresses data in object format and can be transferred as text.

[1116] The "DOM structure of a web page" refers to a document object model that represents the hierarchical structure of an HTML or XML document on a web page.

[1117] A "prompt" is an input sentence that gives instructions to an AI model and is used to specify the conditions and requirements for analysis.

[1118] "User needs" are the requirements and desires of the user, and are the criteria for the system to provide customization in response to these requirements.

[1119] The present invention is a system that realizes efficient code review using a generative AI model by using an extension installed on a user's device to obtain code differences on a web page and send the difference data to an analysis server.

[1120] User terminal

[1121] A user device is a computing device connected to the Internet and capable of accessing websites using a web browser (e.g., Google Chrome). A specific extension is installed on the user device. This extension is activated when the user opens a pull request page in a version control system (e.g., GitHub).

[1122] Get code diff

[1123] The extension installed on the user's device analyzes the DOM structure of the web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text. The obtained code differences are then compiled as data for later analysis on the server.

[1124] Sending to the analysis server

[1125] The code difference data acquired on the user's device is sent to the analysis server's API endpoint using an HTTP POST request, which can receive POST requests via the HTTP protocol.

[1126] Analysis on the server

[1127] The analysis server deserializes the received code diff data and feeds it into a generative AI model, which uses pre-trained parameters to generate an outline of the code and suggest improvements, such as detecting specific patterns of errors or suggesting optimal techniques for improving code efficiency.

[1128] Return and display of analysis results

[1129] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately on a web page by the extension. Specifically, explanations and suggestions are displayed in a prominent position on the pull request page, allowing the user to immediately check the analysis results and make any necessary corrections.

[1130] Specific examples

[1131] When a user opens a pull request

[1132] When a user opens a pull request page in a version control system, the extension activates and automatically retrieves the code diffs within the page, which are then sent to a server for analysis by a generative AI model.

[1133] Example prompt:

[1134] "What code diff process happens when a user opens a pull request page?"

[1135] Displaying analysis results

[1136] The analysis results include specific suggestions, such as "This code lacks error handling in the part that establishes a database connection. Please add error handling." Users can use this feedback to correct their code.

[1137] Example prompt:

[1138] "Please explain the process by which the analysis server inputs the code difference data received into the AI ​​model for analysis."

[1139] The present invention can significantly improve the efficiency and quality of code reviews, allowing engineers to receive immediate feedback and contributing to improved product quality.

[1140] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1141] Step 1:

[1142] Installing and launching the extension

[1143] A user installs Google Chrome on their computer and adds a specific extension that runs when they open a pull request page in a version control system (e.g., GitHub).

[1144] Input: Web browser, extension file

[1145] Output: Installed extensions

[1146] What happens: A user searches for an extension in the Chrome Web Store and clicks the "Install" button. After installation, the extension's icon appears in the browser toolbar.

[1147] Step 2:

[1148] Get code diff

[1149] When a user opens a pull request page in a version control system, the extension installed on the device analyzes the DOM structure of the page and extracts text data from multiple code difference elements.

[1150] Input: URL of the pull request page

[1151] Output: Extracted code diff data (text format)

[1152] What it does: The extension looks for HTML elements with specific class names or IDs on the pull request page, extracts the text data of the code diffs from those elements, and converts it to JSON format.

[1153] Step 3:

[1154] Sending differential data to the server

[1155] The device sends the acquired code difference data to the analysis server's API endpoint using an HTTP POST request.

[1156] Input: Code diff data (JSON format)

[1157] Output: POST request to the server

[1158] Specific operation: The differential data obtained on the terminal is serialized in JSON format, included in the body of the HTTP POST request, and sent to the API endpoint.

[1159] Step 4:

[1160] Data analysis on the server

[1161] The server deserializes the received code diff data and feeds it into a generative AI model, which uses pre-trained parameters to generate an outline of the code and suggests improvements.

[1162] Input: Code diff data (JSON format)

[1163] Output: Analysis results (code summary and improvements)

[1164] Specific operation: When the server receives a POST request, it parses the JSON data in the request body and converts it into an input format for the AI ​​model. The model then analyzes it and generates a recommendation result.

[1165] Step 5:

[1166] Return and display of analysis results

[1167] The server returns the generated analysis results in JSON format to the user's device, which then deserializes the results and displays them on the pull request page.

[1168] Input: Analysis result (JSON format)

[1169] Output: Analysis results displayed on the pull request page

[1170] Specific operation: Once the analysis process on the server is complete, the results are formatted as JSON and sent as an HTTP response to the terminal. The terminal parses the JSON data and embeds the results in the HTML of the pull request page.

[1171] This step provides a concrete understanding of the overall system behavior and allows users to receive a fast, high-quality code review.

[1172] (Application example 1)

[1173] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1174] Robot operation and software updates are frequently performed in factories. However, the code reviews and error detection that accompany these updates require a great deal of time and effort, creating a need for greater efficiency. Conventional methods require personnel to manually review the code and identify problems, which can lead to human error and oversight. Real-time feedback is also difficult, and suggestions for improvements are often delayed. This risks reducing the efficiency and safety of robot operation. A system that can resolve these issues and streamline software management for factory robots is needed.

[1175] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1176] In this invention, the server includes means for acquiring code differences on a web page using an extension installed on a user terminal, means for transmitting the acquired code difference data to an analysis server, means for analyzing the code difference data using an AI model in the analysis server and generating a code overview and improvements, means for returning the generated analysis results to the user terminal, means for displaying the analysis results on a web page in the user terminal, means for acquiring code differences in factory robot software and transmitting them to the analysis server, means for analyzing the acquired code differences using an AI model and proposing improvements to the efficiency and safety of robot operation, and means for returning the generated proposal results to the user terminal and displaying them. This enables efficient and accurate code reviews of factory robot software and enables real-time feedback and suggestions for improvements.

[1177] A "user terminal" is a computing device that can connect to the Internet using a web browser and that has advanced features installed.

[1178] An "extension" is a software program that is added to a web browser on a user's device to provide specific functionality.

[1179] "Code differences" are text data of parts of the software version history where changes or additions have been made.

[1180] The "analysis server" is a computer system that receives code difference data sent from a user terminal and analyzes it using an AI model.

[1181] An "AI model" is an artificial intelligence system that uses pre-trained algorithms to analyze input data and perform specific tasks.

[1182] The "generated analysis results" are information about the code overview and areas for improvement obtained after the AI ​​model analyzes the code difference data.

[1183] A "factory robot" is a mechanical device used to perform automated tasks in a factory.

[1184] "Software management" refers to the overall process of software version control, updates, maintenance, and review.

[1185] The "DOM structure of a web page" is an object model that shows the HTML elements of a web page and their hierarchical structure.

[1186] "Text data" is data that is expressed in the form of a string in a computer system.

[1187] This invention is a system for improving the efficiency of software management for factory robots, and is composed of a user terminal, an extension function, an analysis server, and an AI model. Specific embodiments using various hardware and software are described below.

[1188] User terminal

[1189] The user device is a computing device such as a smartphone or PC that can connect to the Internet. An extension is installed in the web browser on the user device to perform software updates and code reviews for factory robots. When a user opens a page containing code differences, the extension runs, analyzes the DOM structure of the web page, and obtains the code difference data.

[1190] Extensions

[1191] The extension is a software program added to the user's web browser. The extension uses the class names and IDs of HTML elements to identify code differences and extracts text data from multiple code difference elements. The code difference data is then sent to the server.

[1192] Analysis Server

[1193] The analysis server is a computer system located on the cloud or on-premise. To receive code difference data sent from user terminals, an API endpoint is provided via the HTTP protocol. The server inputs the received code difference data into an AI model for analysis.

[1194] AI model

[1195] The AI ​​model is an AI system that uses pre-trained algorithms to generate code summaries and improvements. The server uses the AI ​​model to analyze code diff data, detect specific patterns of errors, and suggest optimal methods for improving efficiency. The analysis results are returned to the user's device in JSON format.

[1196] Displaying analysis results

[1197] The analysis results are returned to the user's device and displayed on a web page by the extension, providing feedback on improvements to the software code and detecting errors in a user-friendly format.

[1198] Examples of specific examples and prompts

[1199] For example, the AI ​​model may generate suggestions like the following for code differences obtained when updating a robot's firmware.

[1200] For example: "This code is missing error handling where it establishes a database connection. Please add error handling."

[1201] An example of a prompt sentence to input to the generative AI model is as follows:

[1202] Please analyze the code diff below and suggest improvements:

[1203] Diff: {code_diff}

[1204] example:

[1205] Suggestion: Deprecated API is being used. Please use the latest API.

[1206] This invention enables efficient and accurate software management of factory robots, facilitating early error detection and correction. Real-time feedback and specific improvement suggestions improve the safety and efficiency of robot operations.

[1207] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1208] Step 1:

[1209] When a user opens the factory robot software update page, an extension on the user's device runs and automatically retrieves the code differences within the page. Specifically, the extension analyzes the DOM structure of the web page and identifies the code differences based on the class names and IDs of HTML elements. This analysis retrieves the code differences as text data.

[1210] Input: DOM structure of the web page

[1211] Output: Text data of code diff

[1212] Step 2:

[1213] The code difference data acquired on the user device is sent to the analysis server via the HTTP protocol. Specifically, the extension on the user device sends a POST request to the API endpoint and sends the acquired code difference data to the server in JSON format.

[1214] Input: Text data of code diff

[1215] Output: POST request to the server

[1216] Step 3:

[1217] The analysis server inputs the received code difference data into an AI model. The server uses the pre-trained AI model to analyze the code overview and areas for improvement. For example, it detects errors of specific patterns and suggests optimal methods for improving efficiency.

[1218] Input: Text data of the code diff sent to the server

[1219] Output: Analysis results by the AI ​​model (code summary and improvements)

[1220] Step 4:

[1221] The generated analysis results are returned to the user terminal in JSON format. The analysis server returns the analysis results as a response using the HTTP protocol.

[1222] Input: Analysis results from AI model

[1223] Output: JSON response to the user's device

[1224] Step 5:

[1225] The analysis results returned to the user's device are displayed on the web page by the extension. Specifically, the extension analyzes the analysis results and displays details of improvements and errors in a prominent location on the web page. This display allows the user to immediately check the analysis results and make any necessary corrections.

[1226] Input: Parsed result in JSON format

[1227] Output: Analysis results displayed on a web page

[1228] Step 6:

[1229] Users can then make modifications to the software code based on the analysis results displayed on the webpage. For example, they can follow suggestions from the AI ​​model to fix specific errors or make code changes to improve efficiency. These modifications will make the factory robots operate more safely and efficiently.

[1230] Input: Analysis results displayed on a web page

[1231] Output: User modifications to the software code

[1232] These steps ensure efficient and accurate software management of factory robots, and real-time feedback and specific AI-driven improvement suggestions can be obtained to enhance the safety and efficiency of robot operations.

[1233] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1234] This invention is a system that realizes efficient code review using an AI model by acquiring code differences on a web page using an extension installed on a user's device and sending the difference data to an analysis server. Furthermore, this invention includes a configuration that combines an emotion engine that recognizes the user's emotions, making it possible to adjust the way the analysis results are displayed based on the user's emotions.

[1235] Specific embodiments

[1236] User terminal

[1237] The user terminal is a computing device connected to the Internet and capable of accessing websites using a web browser. The claimed extension and emotion engine are installed on the user terminal. The extension is activated when the user opens a pull request page in a version control system such as GitHub.

[1238] Get code diff

[1239] The extension installed on the user's device analyzes the DOM structure of the web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text. The code difference data is then compiled as data for later analysis on the server.

[1240] Sending to the analysis server

[1241] The code difference data obtained on the user's device is sent to the server's API endpoint, which is designed to receive POST requests via the HTTP protocol and analyze the difference data.

[1242] Analysis on the server

[1243] The analysis server inputs the received code difference data into the AI ​​model, which uses pre-trained parameters to generate an overview of the code and suggestions for improvement. For example, it can detect specific patterns of errors or suggest optimal methods for improving code efficiency.

[1244] Recognizing user emotions with an emotion engine

[1245] The emotion engine analyzes data obtained from the user's input device and camera footage to recognize the user's emotions. The emotion engine can capture the user's emotional state in real time while viewing the pull request page. Emotions include joy, anger, sadness, surprise, etc.

[1246] Return and display of analysis results

[1247] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately on a web page by the extension's content script. The display method is adjusted based on the user's emotional state, as recognized by the emotion engine. For example, if the user is in a state of high stress, the analysis results can be presented in a soft tone, guiding the user into a relaxed state.

[1248] Specific examples

[1249] When a user opens a pull request

[1250] When a user opens a GitHub pull request page, the extension activates and automatically retrieves the code diffs within the page, which are then sent to a server for analysis by an AI model.

[1251] Emotion Engine Operation

[1252] While the user is reviewing the pull request, the emotion engine analyzes the user's facial expressions from the camera footage and recognizes their emotions. For example, if the user is feeling anxious about the code review, the analysis results will be displayed in a reassuring tone to alleviate the user's anxiety.

[1253] Displaying analysis results

[1254] The analysis results include specific suggestions, such as "This code lacks error handling in the part that establishes a database connection. Please add error handling," as well as comments that take the user's feelings into consideration. Users can refer to these and make corrections to their code.

[1255] In this way, the system according to the present invention can improve the efficiency of code reviews and enhance the quality of reviews. Furthermore, the introduction of an emotion engine can improve the user experience by providing feedback that takes into account the user's emotional state.

[1256] The processing flow will be explained below.

[1257] Step 1:

[1258] A user opens the pull request page.

[1259] When a user opens a pull request page in a version control system, the extension installed in the browser is activated.

[1260] Step 2:

[1261] The user terminal acquires the code difference.

[1262] The extension installed on the user's device analyzes the DOM structure of the web page, identifies the code differences, and retrieves them in text format. Specifically, it uses the class names and IDs of the elements to find the code differences and extracts their contents as text.

[1263] Step 3:

[1264] The user terminal transmits the code difference data to the analysis server.

[1265] The device's background script sends the acquired code difference data to the server's API endpoint as a POST request. The request includes the difference data in JSON format.

[1266] Step 4:

[1267] The server receives the differential data.

[1268] The analysis server receives the POST request and extracts the code difference data from the request body.

[1269] Step 5:

[1270] The server analyzes the differential data using an AI model.

[1271] The server inputs the extracted differential data into the AI ​​model and begins analysis. The AI ​​model generates an overview of the code and points to improve. For example, it might generate a message like, "This code has the ability to establish a database connection, but error handling is lacking."

[1272] Step 6:

[1273] The emotion engine of the user terminal recognizes the user's emotion.

[1274] The emotion engine analyzes data obtained from the user's input devices and camera footage to recognize the user's emotions. It identifies emotions such as joy, anger, sadness, and surprise.

[1275] Step 7:

[1276] The server returns the analysis results.

[1277] The server then compiles the generated analysis results in JSON format and sends them back to the user's device. The analysis results include an overview of the code, suggestions for improvement, and comments appropriate to the user's feelings.

[1278] Step 8:

[1279] The user terminal receives the analysis results.

[1280] A background script on the user terminal receives the analysis results returned from the server and transfers the results to the content script.

[1281] Step 9:

[1282] The user terminal displays the analysis results.

[1283] The content script displays the received analysis results in a prominent location on the pull request page. For example, it displays the analysis results for each code change so that users can understand them immediately. This display method takes user emotions into consideration, allowing users to review in a relaxed atmosphere.

[1284] Example 2

[1285] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1286] Conventional code review systems are limited to analyzing code and suggesting improvements, and are unable to provide feedback that takes into account the user's emotional state. This can lead to stress and make it difficult for users to accept the review results. Furthermore, many systems have difficulty acquiring and analyzing code differences, requiring significant effort to achieve efficient code reviews. Therefore, there is a need for a system that provides appropriate feedback based on the user's emotional state and allows for efficient and effective code reviews.

[1287] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring code differences on a web page using an extension installed on a user terminal, means for transmitting the acquired code difference data to an analysis server, means for analyzing the code difference data using a generative AI model in the analysis server and generating a code outline and improvements, means for returning the generated analysis results to the user terminal, means for detecting the user's emotional state using an emotion recognition engine, means for adjusting the display method of the analysis results depending on the emotional state, and means for displaying the analysis results on a web page in the user terminal. This enables feedback that takes the user's emotional state into consideration, resulting in efficient and effective code reviews.

[1288] "User terminal" refers to a computing device that can connect to the Internet and that can access websites using a web browser.

[1289] An "extension" refers to a program module that is installed in a particular web browser and provides specific additional functionality on a web page.

[1290] "Code differences" are data that indicate changes to source code in a version control system.

[1291] The term "analysis server" refers to a computer device that has the function of processing and analyzing received code difference data.

[1292] A "generative AI model" is an artificial intelligence model that is trained in advance using large amounts of data, and then analyzes code and suggests improvements.

[1293] An "emotion recognition engine" refers to software or hardware that analyzes a user's input device or camera footage to detect the user's emotional state.

[1294] "Analysis results" refers to data that includes an overview of the code generated by the generative AI model and areas for improvement.

[1295] "Adjusting the display method" refers to changing the way the analysis results are presented depending on the user's emotional state.

[1296] User terminal

[1297] The user terminal is a computing device that can connect to the Internet and can access web pages using a web browser. The user terminal includes an extension and an emotion recognition engine installed in a specific web browser. The extension is activated, for example, when the user opens a pull request page in a version control system.

[1298] Get code diff

[1299] The extension analyzes the DOM structure of a web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text data.

[1300] Analysis Server

[1301] The code difference data acquired by the user's device is sent to the analysis server via HTTP protocol. The analysis server inputs the received data into a generative AI model to generate an outline of the code and improvements. This generative AI model is a model that has been trained in advance on a large code dataset.

[1302] Generative AI Models

[1303] Based on the input code difference data, the generative AI model detects specific error patterns and suggests improvements to improve code efficiency, such as "This function lacks error handling when connecting to a database."

[1304] Emotion Recognition Engine

[1305] The emotion recognition engine on the user device analyzes the user's input device and camera footage to recognize the user's emotional state in real time. Types of emotions include joy, anger, sadness, surprise, etc., and the way the analysis results are displayed is adjusted based on these.

[1306] Return and display of analysis results

[1307] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately by the extension's content script. The display method is adjusted based on the user's emotional state detected by the emotion recognition engine. For example, if the user is under stress, the analysis results will be displayed in a softer tone to reduce the user's psychological burden.

[1308] Specific examples

[1309] When a user opens a pull request

[1310] When a user opens a GitHub pull request page, the extension activates and automatically retrieves the code diffs within the webpage, which are then sent to an analytics server for analysis by the generative AI model.

[1311] Emotion Engine Operation

[1312] While the user is reviewing the pull request, the emotion engine analyzes the user's facial expressions from the camera footage and recognizes their emotions. For example, if the user is feeling anxious about the code review, the analysis results will be displayed in a reassuring tone to alleviate the user's anxiety.

[1313] Displaying analysis results

[1314] The analysis results include specific suggestions such as "This code lacks error handling in the part that establishes a database connection. Please add error handling," as well as comments that take the user's feelings into consideration. Users can use this feedback to make corrections to their code.

[1315] Prompt Sentence Examples

[1316] "Please analyze this code diff and suggest the best improvements."

[1317] "If the user is under high stress, please display the analysis results in a relaxing tone."

[1318] If you find a specific error pattern, please suggest ways to improve it.

[1319] In this way, the system according to the present invention can improve the efficiency of code reviews and enhance the quality of reviews. Furthermore, by utilizing an emotion recognition engine, feedback that takes into account the user's emotional state can be provided, thereby improving the user experience.

[1320] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1321] Step 1:

[1322] When a user opens a pull request page in a version control system using a web browser, the extension is automatically activated and monitors the page load. The input is the user accessing the page, and the output is the extension's launch. Specifically, the extension waits for the page's DOM to be fully loaded.

[1323] Step 2:

[1324] The extension analyzes the DOM structure of a web page to identify code differences. This involves using the class names and IDs of HTML elements to find elements that contain code differences. The input is the DOM structure of the page, and the output is the text data of the code differences. Specifically, the extension uses JavaScript methods such as querySelector to extract specific elements.

[1325] Step 3:

[1326] The user device converts the acquired code difference text data into JSON format and sends it to the analysis server as an HTTP POST request. The input is the code difference text data, and the output is a request sent to the analysis server. Specifically, the user device uses the fetch API to send the data to the specified URL.

[1327] Step 4:

[1328] The analysis server analyzes the received code difference data. First, it inputs the data into a generative AI model to analyze the code. The input is JSON data of the code difference, and the output is the generation of analysis results. Specifically, the server invokes an AI model (for example, a model using Python's TensorFlow or PyTorch) to generate an overview of the code and areas for improvement.

[1329] Step 5:

[1330] The emotion recognition engine analyzes the camera images and input device data on the user's device to recognize the user's emotional state in real time. The input is the camera images and input device data, and the output is data on the user's emotional state. Specifically, the emotion recognition engine uses OpenCV and deep learning frameworks to analyze facial expressions.

[1331] Step 6:

[1332] The analysis server returns the generated analysis results in JSON format to the user device. At the same time, it also returns instructions to adjust the display method based on the user's emotional state. The input is the analysis results and the user's emotional state data, and the output is response data in JSON format. Specifically, the server returns the analysis results as an HTTP response.

[1333] Step 7:

[1334] The user device displays the received analysis results on a web page. The display method is adjusted according to the user's emotional state. The input is the JSON data of the analysis results, and the output is the user's display screen. Specifically, the extension performs DOM manipulation to appropriately display the analysis results. For example, it uses the innerHTML method to insert the analysis results into HTML elements and adjust the display style.

[1335] In this way, the system acts automatically in response to user actions, improving the efficiency of code review and the user experience.

[1336] (Application example 2)

[1337] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1338] Conventional code review systems have limited means for efficiently analyzing code difference data, making it difficult to provide appropriate feedback based on the user's emotional state. Furthermore, code reviews can sometimes cause stress, so measures to improve the user experience are needed.

[1339] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring code differences on a web page using an extension installed on a user terminal, means for transmitting the acquired code difference data to an analysis server, means for analyzing the code difference data using a generative AI model and generating a code outline and improvements in the analysis server, means for adjusting the display method of the analysis results using an emotion engine that recognizes the user's emotional state, means for returning the generated analysis results to the user terminal, and means for displaying the analysis results on a web page in the user terminal. This makes it possible to improve the efficiency of code review while providing appropriate feedback tailored to the user's emotional state.

[1340] A "user terminal" is a computing device connected to the Internet that can access websites using a web browser.

[1341] An "extension" is a software component installed in a web browser to perform specific tasks in order to extend the functionality of the browser.

[1342] A "web page" is a page of information accessible via the Internet, the content of which is written in a standard language such as HTML.

[1343] A "code delta" refers to a change in software code, typically representing the difference between old and new source code in a version control system.

[1344] An "analysis server" is a remote computing device that processes and analyzes received data to generate results.

[1345] A "generative AI model" is an artificial intelligence model that has been pre-trained using machine learning and deep learning techniques to perform analysis and predictions based on input data.

[1346] The "emotion engine" is a system that analyzes data obtained from the user's input device and camera footage to recognize the user's emotions.

[1347] "User's emotional state" refers to the user's mental and emotional state, including emotions such as joy, anger, sadness, surprise, etc.

[1348] "Adjusting the display method" refers to changing the way the analysis results are presented based on the user's emotional state.

[1349] The "DOM structure of a web page" is the document object model of a web page, which is a tree-like representation of the logical structure of an HTML or XML document.

[1350] "Text data" refers to data in the form of code or strings that can be analyzed and processed.

[1351] User terminal

[1352] The user terminal is a computing device connected to the Internet and capable of accessing websites using a web browser. The user terminal has installed thereon the extension and emotion engine described in the claims. The extension operates when the user opens a pull request page in the version control system.

[1353] Get code diff

[1354] The extension installed on the user's device analyzes the DOM structure of the web page and extracts text data from multiple code difference elements. Specifically, it identifies code difference locations using the class names and IDs of HTML elements and obtains their contents as text. The code difference data is then compiled as data for later analysis on the server.

[1355] Sending to the analysis server

[1356] The code diff data is sent to a server API endpoint that is designed to accept POST requests over the HTTP protocol and parse the diff data.

[1357] Analysis on the server

[1358] The analysis server inputs the received code difference data into a generative AI model. The generative AI model uses pre-trained parameters to generate an outline of the code and suggestions for improvement. For example, it can detect specific patterns of errors or suggest optimal methods for improving code efficiency.

[1359] Recognizing user emotions with an emotion engine

[1360] The emotion engine analyzes data obtained from the user's input device and camera footage to recognize the user's emotions. The emotion engine can capture the user's emotional state in real time while viewing the pull request page. Emotions include joy, anger, sadness, surprise, etc.

[1361] Return and display of analysis results

[1362] The generated analysis results are sent back to the user's device in JSON format. The analysis results returned to the user's device are then displayed appropriately on a web page by the extension's content script. The display method is adjusted based on the user's emotional state, as recognized by the emotion engine. For example, if the user is in a state of high stress, the analysis results can be presented in a soft tone, guiding the user into a relaxed state.

[1363] Specific examples

[1364] Example 1: Recommendations based on product page emotional state

[1365] If the emotion engine detects excitement while a user is viewing a particular product page:

[1366] "Want to know more about this product? Click here for more information!"

[1367] Input prompt example

[1368] "Show me a Python script that recognizes a user's emotional state while browsing a product page and serves them a message tailored to that emotion."

[1369] In this way, each component, such as the server, user terminal, and emotion engine, works in cooperation to improve the user experience while enabling efficient code review.

[1370] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1371] Step 1:

[1372] When a user opens a pull request page, an extension installed on the user's device is activated. The extension analyzes the DOM structure of the web page and identifies code differences. The HTML content of the web page is used as input, and the output is text data containing the code differences. This text data is then compiled in a format that can be sent to the server.

[1373] Step 2:

[1374] The user device sends the acquired code difference data to the analysis server's API endpoint via an HTTP POST request. The input is the code difference text data obtained in step 1, and the output is the code difference data in JSON format that the server receives.

[1375] Step 3:

[1376] The analysis server processes the received code difference data and inputs it into the generative AI model. The input is the code difference data sent in step 2, and the output is the analysis results from the generative AI model. The generative AI model generates a code outline and improvements using pre-trained parameters.

[1377] Step 4:

[1378] The server returns the analysis results in JSON format to the user device. The input is the analysis results from the generative AI model, and the output is the JSON data of the analysis results received by the user device. This data includes improvement suggestions and error detection information.

[1379] Step 5:

[1380] The user device receives the analysis results and simultaneously activates the emotion engine to recognize the user's emotional state in real time. The input is camera footage capturing the user's facial expressions and data from input devices, and the output is the user's emotional state (joy, anger, sadness, surprise, etc.).

[1381] Step 6:

[1382] Based on the user's emotional state output by the emotion engine, the system selects appropriate templates and tones to adjust the display method. The input is the user's emotional state and the analysis results from the server, and the output is the display content of the adjusted analysis results.

[1383] Step 7:

[1384] The user terminal displays the adjusted analysis results on a web page. The input is the display content adjusted in step 6, and the output is feedback that is easy for the user to see and causes little stress to the user.

[1385] Through these steps, users can receive efficient code reviews and receive feedback tailored to their emotional state, improving the user experience.

[1386] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1387] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1388] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1389] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1390] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1391] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1392] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1393] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1394] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1395] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1396] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1397] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1398] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1399] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1400] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1401] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1402] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1403] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1404] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1405] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1406] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1407] The following is further disclosed regarding the above embodiment.

[1408] (Claim 1)

[1409] The extension installed on the user's device

[1410] A means for obtaining code diffs on a web page;

[1411] a means for transmitting the acquired code difference data to an analysis server;

[1412] In the analysis server, a means for analyzing the code difference data using an AI model and generating an outline of the code and improvements;

[1413] means for returning the generated analysis results to a user terminal;

[1414] The system includes means for displaying the analysis results on a web page at a user terminal.

[1415] (Claim 2)

[1416] 2. The system according to claim 1, wherein the code difference data acquisition means analyzes the DOM structure of the web page and extracts text data from a plurality of code difference elements.

[1417] (Claim 3)

[1418] 2. The system of claim 1, wherein the means for generating analysis results includes a function for providing instructions to the AI ​​model using prompts that can be customized according to user needs.

[1419] "Example 1"

[1420] (Claim 1)

[1421] The extension installed on the user's device

[1422] A means for obtaining code diffs on a web page;

[1423] a means for transmitting the acquired code difference data to an analysis server;

[1424] In the analysis server, a means for analyzing the code difference data using a generation AI model and generating an outline of the code and improvements;

[1425] means for returning the generated analysis results to a user terminal;

[1426] means for displaying the analysis results on a web page in the user terminal;

[1427] means for using a web browser interconnected via the Internet on a user terminal;

[1428] A means to send the acquired differential data via an HTTP POST request;

[1429] A system that includes a means to return analysis results in JSON format.

[1430] (Claim 2)

[1431] 2. The system according to claim 1, wherein the code difference data acquisition means analyzes the DOM structure of the web page and extracts text data from a plurality of code difference elements.

[1432] (Claim 3)

[1433] 2. The system of claim 1, wherein the means for generating analysis results includes a function for providing instructions to the generating AI model using prompts that can be customized according to user needs.

[1434] "Application Example 1"

[1435] (Claim 1)

[1436] The extension installed on the user's device

[1437] A means for obtaining code diffs on a web page;

[1438] a means for transmitting the acquired code difference data to an analysis server;

[1439] In the analysis server, a means for analyzing the code difference data using an AI model and generating an outline of the code and improvements;

[1440] means for returning the generated analysis results to a user terminal;

[1441] means for displaying the analysis results on a web page in the user terminal;

[1442] A means for acquiring code differences of the factory robot software and transmitting them to an analysis server;

[1443] The acquired code differences are analyzed using an AI model to suggest improvements to the efficiency and safety of robot operation.

[1444] The system includes a means for returning the generated proposal results to the user terminal and displaying them.

[1445] (Claim 2)

[1446] 2. The system according to claim 1, wherein the code difference data acquisition means analyzes the DOM structure of the web page and extracts text data from a plurality of code difference elements.

[1447] (Claim 3)

[1448] 2. The system of claim 1, wherein the means for generating analysis results includes a function for providing instructions to the AI ​​model using prompts that can be customized according to user needs.

[1449] "Example 2: Combining Emotion Engines"

[1450] (Claim 1)

[1451] The extension installed on the user's device

[1452] A means for obtaining code diffs on a web page;

[1453] a means for transmitting the acquired code difference data to an analysis server;

[1454] In the analysis server, a means for analyzing the code difference data using a generation AI model and generating an outline of the code and improvements;

[1455] means for returning the generated analysis results to a user terminal;

[1456] means for detecting the emotional state of a user by an emotion recognition engine;

[1457] A means for adjusting the display method of the analysis results according to the emotional state;

[1458] The system includes means for displaying the analysis results on a web page at a user terminal.

[1459] (Claim 2)

[1460] 2. The system according to claim 1, wherein the code difference data acquisition means analyzes the DOM structure of the web page and extracts text data from a plurality of code difference elements.

[1461] (Claim 3)

[1462] 2. The system of claim 1, wherein the means for generating the analysis results includes a function for providing instructions to the generating AI model using prompts that can be customized according to the user's needs.

[1463] "Application example 2 when combining emotion engines"

[1464] (Claim 1)

[1465] The extension installed on the user's device

[1466] A means for obtaining code diffs on a web page;

[1467] a means for transmitting the acquired code difference data to an analysis server;

[1468] In the analysis server, a means for analyzing the code difference data using a generation AI model and generating an outline of the code and improvements;

[1469] a means for adjusting the display of the analysis results using an emotion engine that recognizes the user's emotional state;

[1470] means for returning the generated analysis results to a user terminal;

[1471] The system includes means for displaying the analysis results on a web page at a user terminal.

[1472] (Claim 2)

[1473] 2. The system according to claim 1, wherein the code difference data acquisition means analyzes the DOM structure of the web page and extracts text data from a plurality of code difference elements.

[1474] (Claim 3)

[1475] The system according to claim 1, characterized in that the means for generating analysis results includes a function for providing instructions to the generative AI model using prompt sentences that can be customized according to user needs. [Explanation of symbols]

[1476] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. The extension installed on the user's device A means for obtaining code diffs on a web page; means for transmitting the acquired code difference data to an analysis server; In the analysis server, a means for analyzing the code difference data using an AI model and generating an outline of the code and improvements; means for returning the generated analysis results to a user terminal; The system includes means for displaying the analysis results on a web page at a user terminal.

2. 2. The system according to claim 1, wherein the code difference data acquisition means analyzes a DOM structure of the web page and extracts text data from a plurality of code difference elements.

3. 2. The system of claim 1, wherein the means for generating the analysis results includes a function for providing instructions to the AI ​​model using prompts that can be customized according to user needs.

Images