System

Abstract

A system is provided.SOLUTION: A system, comprising: means for receiving and analyzing source code; means for generating verification patterns from the analyzed source code; means for performing automated tests based on the generated verification patterns; and means for recording and reporting test results.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In the verification process of systems and networks, the number of verification patterns is increasing exponentially due to the diversification of services and the development and modification of large-scale systems. This makes it practically difficult to comprehensively test all verification patterns manually, requiring significant resources and time. As a result, there is an unavoidable risk that some verifications will be overlooked, affecting the quality of the service. Therefore, there is a need for a system that can efficiently and accurately generate verification patterns and perform automated testing. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for receiving and analyzing source code, a means for generating verification patterns from the analyzed source code, a means for performing automated testing based on the generated verification patterns, and a means for recording and notifying test results. In particular, the accuracy of the source code analysis is improved by using natural language processing technology, and efficiency is enhanced by using a version control system to obtain source code differences. Furthermore, an automation tool is used to automatically operate the user interface, and the verification pattern generation means generates multiple verification patterns based on conditional branching and loop structures, achieving extensive test coverage. In this way, the efficiency and quality of the verification process are improved.

[0006] "Source code" is the program text written as a set of instructions or syntax that describes the functionality and behavior of software.

[0007] "Analysis" is the process of interpreting input data or information and finding specific meaning or structure.

[0008] A "verification pattern" is a set of specific conditions or scenarios for verifying the behavior of software or a system.

[0009] "Automated testing" is a technique in which a series of tests to check the operation of software is executed programmatically rather than manually.

[0010] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[0011] "Diff" is a term that refers to the changes or differences in content between two versions.

[0012] A "version control system" is a system for recording and managing the change history of source code and documents.

[0013] A "user interface" is the means or method by which a user and a system interact with each other.

[0014] An "automation tool" is a program or software designed to perform repetitive tasks automatically.

[0015] A "log" is a collection of data that provides a record of system operations and events.

[0016] A "conditional branch" is a control structure in which the flow of program processing changes depending on specific conditions.

[0017] A "loop structure" is a control structure that repeats the same series of operations until a specific condition is met. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9]1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention relates to a technique for automatically generating verification patterns and efficiently performing automatic testing in the process of verifying a system or network.

[0040] The system contains several key components, each performing a specific function. These components and their functions are described below.

[0041] Receiving and analyzing the source code

[0042] The server receives source code from the user. For example, the user uploads the source code through a web interface. The server analyzes the source code using natural language processing techniques. During the analysis, the structure of the source code (e.g., functions, conditional branches, loops, etc.) is represented as an abstract syntax tree (AST).

[0043] Generating validation patterns

[0044] The server automatically generates verification patterns based on each path of conditional branches and loops from the analyzed source code. These verification patterns cover a wide range of scenarios and conditions, preventing oversights in manual verification.

[0045] As a concrete example, if there is a conditional branch "if (userAge > 18)" in the source code, the server will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less."

[0046] Differential analysis and test pattern generation

[0047] The terminal clones or pulls the latest source code from a version control system (e.g., Git). The terminal then obtains the difference from the previous commit and transfers this difference to the server for analysis. The server analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[0048] As a specific example, if the difference includes "adding a new user login function," the server generates test patterns such as "when the user enters correct login information" and "when the user enters incorrect login information."

[0049] GUI-based verification and automated testing

[0050] Users configure automation tools (such as Selenium or Puppeteer) on the server, which then uses these automation tools to run test scripts, which can include filling out forms or pressing buttons on web applications.

[0051] As a concrete example, when a server automatically fills in a user registration form for a web application, it enters values ​​for each field (e.g., "user name," "email address," "password," etc.), clicks the submit button, and records the resulting message (success message or error message) that is displayed.

[0052] Recording and notifying test results

[0053] The server records the test execution results in a detailed log, including the success or failure of each test case and any error messages that occurred. This log is then notified to the user via email and a dashboard, where the user can view the results in detail.

[0054] As explained above, the system of the present invention can efficiently and effectively carry out a series of processes, from analyzing source code to generating verification patterns, analyzing differences, and conducting automated tests and notifying the results. This system eliminates the need for engineers to manually consider a huge number of verification patterns, significantly reducing the risk of missing verifications and improving the quality of system releases.

[0055] The processing flow will be explained below.

[0056] Step 1:

[0057] Users provide source code to the server, which can be uploaded through a web interface.

[0058] Step 2:

[0059] The server analyzes the received source code using natural language processing techniques to represent the code structure (functions, conditional branches, loops, etc.) as an abstract syntax tree (AST).

[0060] Step 3:

[0061] Based on the generated AST, the server traverses each path of the conditional branches and loops in the source code and automatically generates verification patterns, taking into account each scenario and condition in the source code.

[0062] Step 4:

[0063] The device clones or pulls the latest source code from the version control system (e.g. Git), which gives you the most up-to-date code base.

[0064] Step 5:

[0065] The terminal obtains the difference between the latest commit and the previous commit to identify what has been newly added or changed.

[0066] Step 6:

[0067] The device transfers the retrieved differences to the server, allowing the server to see the latest code changes.

[0068] Step 7:

[0069] The server analyzes the received differences, identifying any newly added or modified code sections.

[0070] Step 8:

[0071] Based on the analyzed differences, the server identifies existing verification patterns that may be affected and generates new test patterns.

[0072] Step 9:

[0073] Users configure the automation tools they want to use on their servers, for example by selecting tools such as Selenium or Puppeteer through a web interface.

[0074] Step 10:

[0075] The server runs the test scripts using configured automation tools, which may include filling out forms or pressing buttons on the web application.

[0076] Step 11:

[0077] The server records detailed test execution results in a log, including the success or failure of each test case and any error messages.

[0078] Step 12:

[0079] The server notifies the user of the recorded results via email and a dashboard, where the user can view the test results in detail.

[0080] Example 1

[0081] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0082] In today's complex software development, manual verification and testing not only requires a great deal of time and effort, but also carries a high risk of human error and missed verifications. Furthermore, because software is frequently changed due to program modifications and the implementation of additional features, a system is required to efficiently verify only the changed parts. However, it has been difficult to meet these requirements with conventional manual testing methods.

[0083] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0084] In this invention, the server includes means for receiving and analyzing software, means for generating a verification pattern from the analyzed software, means for performing an automated test based on the generated verification pattern, means for recording and notifying the test results, means for acquiring software differences from a version control system and generating a new verification pattern based on the differences, and means for uploading settings via a user interface and executing tests using an automated tool. This makes it possible to efficiently verify software changes, significantly reduce manual verification work, and reduce the risk of missing verification points.

[0085] "Software" is a set of instructions or programs designed to cause a computer to perform a specific function.

[0086] "Means of analysis" are methods or devices that analyze and interpret the contents of software in detail in order to understand its internal structure and operating logic.

[0087] A "verification pattern" is a test scenario based on specific situations or conditions designed to verify that software behaves as expected.

[0088] "Means for performing automated testing" are programs and tools that can be used to verify that software functions properly without human intervention.

[0089] "Means for recording and notifying test results" refers to a method or device for storing the results of an automated test and communicating that information to the user.

[0090] A "version control system" is a system for managing software change history and tracking and integrating different versions.

[0091] "Software differences" refer to the parts that have been changed or newly added compared to the previous version.

[0092] "User interface" is the collective term for the means and methods by which a user interacts with software or a system.

[0093] An "automation tool" is software or a program designed to perform routine tasks or processes automatically.

[0094] A "means of configuration upload" is a method by which a user enters or transmits specific parameters or configuration information into the system.

[0095] The present invention relates to a technology for automatically generating verification patterns and efficiently performing automated testing in the verification process of a system or network. This system includes important components, each of which performs a specific function.

[0096] Receiving and analyzing the source code

[0097] The server receives the software. Users upload the software through a web interface. The server analyzes the software using natural language processing techniques. During the analysis, the software structure (e.g., functions, conditionals, loops, etc.) is represented as an abstract syntax tree (AST).

[0098] As a concrete example, a user opens a browser, clicks the "Choose File" button in the interface of a web application, selects the file to upload, and clicks the "Submit" button.

[0099] Generating validation patterns

[0100] The server automatically generates verification patterns based on each path of conditional branches and loops from the analyzed software, allowing it to cover a wide range of scenarios and conditions.

[0101] For example, if the source code has a conditional branch if (userAge > 18), two validation patterns will be generated - "if userAge is greater than or equal to 19" and "if userAge is less than or equal to 18".

[0102] Differential analysis and test pattern generation

[0103] The terminal clones or pulls the latest software from a version control system (e.g., Git). The terminal then obtains the differences from the previous commit and transfers these differences to the server. The server analyzes the received differences, identifies newly added or modified code sections, and generates test patterns based on them.

[0104] As a specific example, if the difference includes "adding a new user login function," test patterns for "when the user enters correct login information" and "when the user enters incorrect login information" will be generated.

[0105] GUI-based verification and automated testing

[0106] Users configure automation tools (such as Selenium or Puppeteer) on the server, which then uses these automation tools to run test scripts, which can include filling out forms or pressing buttons on a web application.

[0107] As a concrete example, when a server automatically fills in a user registration form for a web application, it enters values ​​for each field (e.g., "user name," "email address," "password," etc.), clicks the submit button, and records the resulting message (success message or error message).

[0108] Recording and notifying test results

[0109] The server records the test execution results in a detailed log, including the success or failure of each test case and any error messages that occurred. This log is then notified to the user via email and a dashboard, where the user can view the results in detail.

[0110] For example, if the user has email notifications enabled, they will receive an email with the test result details attached.

[0111] Examples of prompt statements

[0112] Below are some example prompts for input to generative AI models for validation and automated testing systems:

[0113] In this system, users upload software, and the server analyzes the software and automatically generates verification and test patterns. It also retrieves differences from the version control system and generates test cases based on the added or modified parts. It runs tests using automated tools, records the results as detailed logs, and notifies the user. A specific example is the automatic filling of user registration forms and recording the submission results.

[0114] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0115] Step 1:

[0116] The user uploads the software through a web interface: the user opens a browser, clicks the "Choose File" button in the web application interface, selects the file to upload, and clicks the "Submit" button.

[0117] Input: User uploaded software file

[0118] Output: Software files sent to the server

[0119] Step 2:

[0120] The server receives the uploaded software and analyzes it using natural language processing techniques. Specifically, it converts the software into an abstract syntax tree (AST) using Python's ast module, and then uses this AST to analyze the code structure (e.g., function definitions, conditional branches, loops, etc.).

[0121] Input: Uploaded software file

[0122] Output: Software structure as an Abstract Syntax Tree (AST)

[0123] Step 3:

[0124] The server automatically generates verification patterns from the analyzed software. Specifically, it generates test cases that cover all possible execution paths based on conditional branching and loop structures. For example, if the condition is "if (userAge > 18)," it generates both scenarios: "when userAge is 19 or greater" and "when userAge is 18 or less."

[0125] Input: Abstract Syntax Tree (AST)

[0126] Output: Validation pattern

[0127] Step 4:

[0128] The terminal clones or pulls the latest software from a version control system (e.g., Git). Specifically, the terminal executes the git clone command or git pull command to obtain the latest code locally.

[0129] Input: Version control system repository information

[0130] Output: A local copy of the latest software code

[0131] Step 5:

[0132] The terminal obtains the difference from the previous commit and transfers this difference to the server. Specifically, it uses the git diff command to obtain the difference from the previous commit and transfers that data to the server.

[0133] Input: Locally cloned or pulled software code

[0134] Output: Differential data

[0135] Step 6:

[0136] The server analyzes the received differences to identify newly added or modified code parts. Specifically, it analyzes the difference data to identify new functions, modified conditional branches, etc.

[0137] Input: differential data

[0138] Output: Identified new or modified code sections

[0139] Step 7:

[0140] The server generates test patterns based on the identified code parts. Specifically, if the difference includes "adding a new user login function," test cases will be generated for "what happens when the user enters correct login information" and "what happens when the user enters incorrect login information."

[0141] Input: New or modified code fragment

[0142] Output: Test pattern

[0143] Step 8:

[0144] The user configures the automation tool (such as Selenium or Puppeteer) on the server. Specifically, the user uploads the automation tool's configuration file (e.g., config.json) from the web interface and enters the required settings (browser type, test script path, etc.).

[0145] Input: User configuration files and settings information

[0146] Output: Settings saved on the server

[0147] Step 9:

[0148] The server runs the test script using an automation tool, which includes filling out forms and pressing buttons in the web application. Specifically, the server uses the configured automation tool (e.g., Selenium WebDriver) to launch a browser, open a web application page, and run the test.

[0149] Input: Configured automation tools and test scripts

[0150] Output: Test execution results

[0151] Step 10:

[0152] The server records the test execution results as a detailed log. This log includes the success or failure of each test case, any error messages that occurred, etc. Specifically, the execution results of each test case (success, failure, error messages, etc.) are saved in a text file or database.

[0153] Input: Test execution results

[0154] Output: Detailed log data

[0155] Step 11:

[0156] The server notifies the user of the test results by sending the generated log file as an email attachment and / or displaying the results in a web interface dashboard.

[0157] Input: Detailed log data

[0158] Output: Notification to user (email, dashboard display, etc.)

[0159] (Application example 1)

[0160] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0161] Currently, there is a lack of efficient and highly accurate means for verifying the performance of robotic motion sequences and tasks in factory environments. This can lead to unexpected malfunctions, resulting in reduced operational efficiency and safety issues. Furthermore, manual verification is time-consuming and labor-intensive, and carries the risk of missing verification steps. Therefore, a more efficient and automated verification system is needed.

[0162] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0163] In this invention, the server includes means for receiving and analyzing source code, means for generating verification patterns from the analyzed source code, means for performing automated testing based on the generated verification patterns, means for recording and notifying test results, means for analyzing the operation sequences and task performance of a robot in a factory environment and generating verification patterns, and means for performing automated testing of the robot based on the generated verification patterns. This makes it possible to automatically verify the operations and tasks of factory robots, thereby improving operational efficiency and reliability.

[0164] "Source code" is a collection of textual instructions written to direct the operation of a program.

[0165] "Means for analyzing" refers to a method or device for analyzing input data or information and understanding its structure and content.

[0166] A "verification pattern" refers to a combination of various test scenarios and conditions to be performed, which serves as a basis for evaluating functionality and performance.

[0167] "Means for performing automated testing" refers to an automated method or device for performing testing without human intervention.

[0168] "Means for recording and notifying test results" refers to a method or device for storing the results of a test after it has been performed and notifying relevant parties.

[0169] "Factory environment" refers to the physical location where manufacturing or processing takes place and the systems and equipment used therein.

[0170] "Robot movement sequence" refers to the order and timing of movements performed by a robot.

[0171] "Task performance" is an indicator of how effectively a robot performs a specific task.

[0172] The "means for generating verification patterns" refers to a method or device that automatically creates appropriate verification methods or scenarios based on analyzed information.

[0173] "Means for performing automated testing of a robot" refers to a method or device for automatically verifying the operation or functionality of a robot.

[0174] "Natural language processing technology" refers to technology that enables computers to understand, interpret, and generate human language.

[0175] "Difference" is a concept that indicates changes between different versions of data or files.

[0176] A system for realizing the present invention includes the following configuration and processing procedures.

[0177] Receiving and analyzing the source code

[0178] The server receives source code from users. Users upload the source code through a web interface. The server analyzes the source code using natural language processing techniques. During the analysis, the structure of the source code (functions, conditional branches, loops, etc.) is represented as an abstract syntax tree (AST).

[0179] Generating validation patterns

[0180] The server automatically generates verification patterns based on each path of conditional branches and loops from the analyzed source code. These verification patterns cover a wide range of scenarios and conditions, preventing oversights in manual verification.

[0181] Differential analysis and test pattern generation

[0182] The terminal clones or pulls the latest source code from a version control system (e.g., Git). The terminal then obtains the difference from the previous commit and transfers this difference to the server for analysis. The server analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[0183] GUI-based verification and automated testing

[0184] Users configure automation tools (such as Selenium or Puppeteer) on the server, which then uses these automation tools to run test scripts, which can include filling out forms or pressing buttons on web applications.

[0185] Recording and notifying test results

[0186] The server records the test execution results in a detailed log, including the success or failure of each test case and any error messages that occurred. This log is then notified to the user via email and a dashboard, where the user can view the results in detail.

[0187] Additional processes when applied to factory robots

[0188] In a factory environment, a means is added to analyze the robot's operation sequence and task performance and generate verification patterns. The server analyzes the robot's operation log and sensor input data and derives the optimal verification pattern from the results. Then, automatic testing of the robot is performed based on the generated verification pattern.

[0189] Hardware and software used

[0190] The following hardware and software are used to implement this system:

[0191] Hardware: Servers, terminals, robots

[0192] Software: Git (version control system), Selenium (automation tool), Puppeteer (automation tool), natural language processing libraries (e.g., NLTK)

[0193] Examples of concrete examples and prompts

[0194] As a concrete example, when a server automatically fills in a user registration form for a web application, it enters values ​​for each field (e.g., "user name," "email address," "password," etc.), clicks the submit button, and records the resulting message (success message or error message) that is displayed.

[0195] An example of a prompt sentence to be input to the generative AI model is written as follows:

[0196] "Detect conditional branches and loops in the source code of factory robots and generate the necessary verification patterns based on them. Also, run automated tests based on the generated patterns and log the results."

[0197] With the above configuration, the present invention can automatically verify the operations and tasks of factory robots, thereby improving operational efficiency and reliability.

[0198] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0199] Step 1:

[0200] The server receives source code from the user. The user uploads the source code through a web interface. The server receives the uploaded source code and prepares the data for analysis. Specifically, the server saves the source code as a text file and prepares it for analysis. The input is the text data of the source code provided by the user, and the output is a source code file stored internally on the server.

[0201] Step 2:

[0202] The server analyzes the received source code. This analysis uses natural language processing techniques to represent the source code structure (functions, conditional branches, loops, etc.) as an abstract syntax tree (AST). Specifically, it runs the source code through an AST library to generate an AST object. The input is the source code file saved in step 1, and the output is the parsed abstract syntax tree object.

[0203] Step 3:

[0204] The server generates verification patterns from the parsed abstract syntax tree. It automatically generates verification patterns based on each path of conditional branches and loops. Specifically, it traverses each node of the AST and adds verification scenarios at points where conditional branches or loops are detected. The input is the abstract syntax tree object generated in step 2, and the output is a list of generated verification patterns.

[0205] Step 4:

[0206] The terminal clones or pulls the latest source code from a version control system (e.g., Git). Then, it gets the diff against the previous commit in the repository you are using. Specifically, the terminal runs Git commands to get the latest commit and extract the diff. The input is the remote Git repository, and the output is the diff data between the old and new commits.

[0207] Step 5:

[0208] The terminal transfers the acquired differential data to the server. The server analyzes the received differential data, identifies newly added or modified code parts, and generates a new verification pattern based on them. The input is the differential data acquired in step 4, and the output is the newly generated verification pattern.

[0209] Step 6:

[0210] The user configures automation tools (such as Selenium or Puppeteer) on the server. The server then uses these automation tools to execute test scripts. Specifically, the server starts the automation tools and automatically simulates the operation of web applications or robots based on the configuration. The input is the automation script configured by the user, and the output is the test execution results.

[0211] Step 7:

[0212] The server records the test execution results as a detailed log. This log includes the success or failure of each test case, any error messages that occurred, etc. The server notifies the user of this record. Specifically, the server generates log data and notifies the user via email or the dashboard. The input is the test execution results obtained in step 6, and the output is the generated log data and notification message.

[0213] Step 8:

[0214] In a factory environment, a server analyzes the robot's operation log and sensor input data, and derives the optimal verification pattern from the results. Based on the generated verification pattern, an automatic test of the robot is performed. The input is the robot's operation log and sensor input data, and the output is the results of the automatic test. A specific example includes a test to confirm whether the robot operates accurately based on an operation scenario.

[0215] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0216] This invention relates to an automatic testing system that combines an emotion engine in the verification process of systems and networks. This system efficiently and effectively performs a series of processes that incorporates an emotion engine, from analyzing source code to generating verification patterns, conducting automatic tests, and recording and notifying test results.

[0217] The system includes the following main components:

[0218] Receiving and analyzing the source code

[0219] The server receives source code from users. Users upload the source code through a web interface. The server analyzes the source code using natural language processing techniques. During the analysis process, the structure of the source code (functions, conditional branches, loops, etc.) is represented as an abstract syntax tree (AST).

[0220] Generating validation patterns

[0221] Based on the generated AST, the server traverses each path of the conditional branches and loops in the source code and automatically generates verification patterns, taking into account each scenario and condition in the source code.

[0222] As a concrete example, if there is a conditional branch "if (userAge > 18)" in the source code, the server will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less."

[0223] Differential analysis and test pattern generation

[0224] The terminal clones or pulls the latest source code from a version control system (e.g., Git). The terminal obtains the difference from the previous commit and transfers this difference to the server for analysis. The server analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[0225] As a specific example, if the difference includes "adding a new user login function," the server generates test patterns such as "when the user enters correct login information" and "when the user enters incorrect login information."

[0226] Emotion engine integration

[0227] The server incorporates an emotion engine into its operation. The emotion engine recognizes and monitors the user's emotions in real time while operating the user interface. This emotion data is used to generate verification patterns and conduct tests.

[0228] As a specific example, if a user experiences stress or anxiety when filling out a form, additional validation patterns are generated to verify the impact of specific operations on the user.

[0229] Automated testing

[0230] Users configure automation tools on the server, for example by selecting tools such as Selenium or Puppeteer through a web interface. The server then uses these automation tools to run test scripts, which can include filling out forms or pressing buttons on web applications.

[0231] Recording and notifying test results

[0232] The server records the test execution results in a detailed log, including the success or failure of each test case, any error messages that occurred, etc. This log is then sent to the user via email or a dashboard.

[0233] As explained above, the system of the present invention can achieve more comprehensive and high-quality verification by analyzing source code, generating verification patterns, analyzing differences, conducting automated tests, and notifying the results, as well as taking into account the user's psychological reactions using an emotion engine. This system eliminates the need for engineers to manually consider a huge number of verification patterns, significantly reducing the risk of missing verifications and improving the quality of system releases.

[0234] The processing flow will be explained below.

[0235] Step 1:

[0236] Users provide source code to the server, which can be uploaded through a web interface.

[0237] Step 2:

[0238] The server analyzes the received source code using natural language processing techniques to represent the code structure (functions, conditional branches, loops, etc.) as an abstract syntax tree (AST).

[0239] Step 3:

[0240] Based on the generated AST, the server traverses each path of the conditional branches and loops in the source code and automatically generates verification patterns that cover a wide range of scenarios and conditions.

[0241] Step 4:

[0242] The device clones or pulls the latest source code from the version control system (e.g. Git), which gives you the most up-to-date code base.

[0243] Step 5:

[0244] The terminal obtains the difference between the latest commit and the previous commit, which identifies what has been added or changed.

[0245] Step 6:

[0246] The device transfers the retrieved differences to the server, allowing the server to see the latest code changes.

[0247] Step 7:

[0248] The server analyzes the received differences, identifying any newly added or modified code sections.

[0249] Step 8:

[0250] Based on the analyzed differences, the server identifies existing verification patterns that may be affected and generates new test patterns.

[0251] Step 9:

[0252] Users configure the server with the automation tools they want to use, for example, by selecting tools such as Selenium or Puppeteer through a web interface.

[0253] Step 10:

[0254] The server runs the test scripts using configured automation tools, which may include filling out forms or pressing buttons on the web application.

[0255] Step 11:

[0256] The server activates an emotion engine to recognize the user's emotions in real time. The emotion engine monitors the user's facial expressions and tone of voice while operating the user interface.

[0257] Step 12:

[0258] The server analyzes the recognized emotion data and generates additional verification patterns based on that data. By providing special verification for operations that cause stress or confusion to the user, the user experience is improved.

[0259] Step 13:

[0260] The server records detailed logs of the test execution results, including the success or failure of each test case, error messages, and the user's emotional state.

[0261] Step 14:

[0262] The server notifies the user of the recorded results via email and a dashboard, where the user can view the test results in detail.

[0263] Through these steps, this system effectively and efficiently verifies source code, and by taking into account user feelings in particular, it supports the development of higher quality, user-friendly software.

[0264] Example 2

[0265] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0266] Conventional automated testing systems focus on recognizing and analyzing source code, but no systems have taken into account user emotions and psychological responses. As a result, improvements to the user experience and the generation of emotion-based verification patterns have not been fully realized. Furthermore, the generation of verification patterns based on differential analysis of source code changes and additions has sometimes been inefficient. A solution to these problems was needed.

[0267] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving and analyzing source code, means for generating a verification pattern from the analyzed source code, means for performing an automated test based on the generated verification pattern, means for recognizing a user's emotions in real time while operating a user interface and generating a verification pattern based on the emotions, means for recording and notifying the test results, and means for acquiring differences in the source code and generating a new verification pattern based on the differences. This enables high-quality automated testing that takes into account the user's emotions and psychological reactions, and also realizes efficient generation of verification patterns for modified source code.

[0268] "Source code" is the set of instructions or statements that make up a program or application, usually written as a text file.

[0269] "Analysis" is the operation of breaking down given information or data into detail and clarifying its structure and relationships.

[0270] A "verification pattern" refers to a series of test cases or scenarios that should be verified by automated testing based on source code conditions or scenarios.

[0271] "Automated testing" is the process of verifying the functionality and performance of software or applications using automated tools and scripts.

[0272] "Test Results" refers to the information and data obtained after an automated test is performed, including success / failure status and error messages.

[0273] "Notifying" is the action or process of making certain information or results known to others.

[0274] "User interface" refers to the interaction means and screen layout that allow a user to interact with a computer system or application.

[0275] "Emotion" refers to the psychological state or reaction that a user feels during operation, and includes states such as joy, sadness, and surprise.

[0276] "Natural language processing technology" is a technology that allows computers to understand human language and perform analysis and generation.

[0277] "Delta" refers to changes or differences between different versions of data or code.

[0278] The present invention is a system that efficiently performs everything from analyzing source code to generating verification patterns, conducting automated tests, and recording and notifying test results. It realizes higher quality testing by recognizing user emotions in real time and utilizing them in generating verification patterns. The main components and operating procedures for implementing this invention are described in detail below.

[0279] System Configuration

[0280] This system mainly includes the following hardware and software:

[0281] Server: The central component that analyzes source code, generates verification patterns, manages automated tests, records and notifies test results, etc.

[0282] Terminal: Retrieves source code from a version control system (e.g., Git), analyzes the differences, and sends them to the server.

[0283] Emotion engine: Recognizes the user's emotions in real time while operating the user interface and sends the data to the server.

[0284] The specific software technologies used include:

[0285] Natural language processing technology: Technology for analyzing source code (e.g., Python's ast module).

[0286] Automation tools: Tools for performing automated testing (e.g., Selenium, Puppeteer).

[0287] Receiving and analyzing the source code

[0288] Users upload source code to the server through a web interface. Once the source code is uploaded, the server receives it and analyzes it using natural language processing technology. Specifically, the source code structure (functions, conditional branches, loops, etc.) is represented as an abstract syntax tree (AST) and the analysis results are obtained.

[0289] Generating validation patterns

[0290] Based on the generated AST, the server follows each path of conditional branches and loops in the source code and automatically generates validation patterns. For example, if the condition "if (userAge > 18)" is included, the server will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less."

[0291] Differential analysis and test pattern generation

[0292] The terminal clones or pulls the latest source code from the version control system, obtains the difference from the previous commit, and transfers this difference to the server. The server analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[0293] Emotion engine integration

[0294] The server integrates an emotion engine into the system. The emotion engine recognizes the user's emotions in real time while operating the user interface and sends the data to the server. For example, if the user feels stress or anxiety while filling out a form, the emotion engine detects this and the server generates additional verification patterns based on the emotion data.

[0295] Running automated tests and recording results

[0296] Users configure automation tools using a web interface, for example by selecting a tool such as Selenium or Puppeteer, and saving the automated test configuration. The server then executes test scripts using the selected automation tool, including form entries and button presses in web applications. After the tests are executed, the server records detailed test results in a log and notifies the user via email or a dashboard.

[0297] Examples of concrete examples and prompts

[0298] For example, if the source code contains a conditional branch "if (userAge > 18)", the server will generate two validation patterns - "if userAge is 19 or greater" and "if userAge is 18 or less".

[0299] Example prompt sentence:

[0300] We have added a new user login feature. Please generate test patterns for this feature that show correct and incorrect login information.

[0301] As described above, this embodiment makes it possible to carry out an automatic test that takes into account the user's emotions, and realizes a system that can obtain high-quality test results.

[0302] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0303] Step 1:

[0304] The user uploads source code to the server through a web interface. The input is a source code file, and the output is the source code saved on the server. Specifically, the user selects the target source code file using a file selection dialog and presses the "Upload" button.

[0305] Step 2:

[0306] After the server receives the uploaded source code, it analyzes it using natural language processing technology. The input is the source code file uploaded by the user, and the output is a parsed abstract syntax tree (AST). Specifically, it uses the Python ast module to analyze the structure of the source code and extract components such as functions, conditional branches, and loops.

[0307] Step 3:

[0308] Based on the generated AST, the server follows each path of conditional branches and loops in the source code and automatically generates verification patterns. The input is the AST, and the output is a list of verification patterns. Specifically, test cases are generated by dividing the scenarios into conditions for conditional branches (e.g., if (userAge > 18)). For example, two patterns are generated: "if userAge is 19 or greater" and "if userAge is 18 or less."

[0309] Step 4:

[0310] The terminal clones or pulls the latest source code from the version control system. The input is the repository URL and authentication information, and the output is the latest source code stored locally. Next, the terminal obtains the difference from the previous commit and sends that difference information to the server. Specifically, it uses Git commands to check the difference and sends the difference file to the server.

[0311] Step 5:

[0312] The server analyzes the received differences and identifies newly added or modified code sections. The input is the difference file, and the output is a list of changed sections. It then generates new test patterns based on these changes. Specifically, if the change includes "adding a new user login function," it generates test patterns that cover various scenarios for the login function (correct information, incorrect information).

[0313] Step 6:

[0314] The server integrates an emotion engine into the system and recognizes the user's emotions in real time while operating the user interface. The input is the user's operation data, and the output is the recognized emotion data. Based on this emotion data, the server generates additional verification patterns. For example, if the user feels stressed while filling out a form, the server detects that emotion and creates additional verification patterns accordingly.

[0315] Step 7:

[0316] The user configures the automation tool using a web interface. The input is the selection information for the tool, such as Selenium or Puppeteer, and the output is the configuration information saved on the server. Specifically, the user selects the automation tool on the configuration screen and presses the "Save" button.

[0317] Step 8:

[0318] The server runs the test script using an automation tool configured by the user. The input is the verification pattern and the automation tool's configuration information, and the output is the test execution results. Specifically, Puppeteer is used to automatically operate the browser and perform form input and button presses in the web application.

[0319] Step 9:

[0320] The server records the test execution results as a detailed log and notifies the user via email or dashboard. The input is the test execution results, and the output is notification information. Specifically, the success or failure of test cases, any error messages that occur, etc. are recorded in detail and notified to the user.

[0321] (Application example 2)

[0322] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0323] Current automated testing systems do not take into account the user's emotions and psychological state, making it difficult to reduce the stress and anxiety of operators, especially in real-world environments such as logistics centers. This leads to increased frustration and errors in system operation, hindering efficient business operations. Furthermore, analysis of test results does not include emotional data, resulting in a lack of information useful for improving the user experience.

[0324] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing source code, means for generating a verification pattern from the analyzed source code, means for recognizing the worker's emotions, means for performing an automatic test based on the generated verification pattern, and means for recording and notifying the test results and emotional data. This makes it possible to recognize the worker's emotions in real time and generate more realistic and effective verification patterns based on that data. Furthermore, by performing post-processing analysis based on the emotional data, it is possible to improve the user experience and reduce operational stress.

[0325] "Source code" is a set of textual instructions that make up a program or application.

[0326] "Analysis" is the process of understanding specific data or information and clarifying its meaning and structure.

[0327] A "verification pattern" is a specific condition or scenario designed to test the behavior of source code or a system.

[0328] "Automated testing" is the process of executing test cases using automated tools and checking the results.

[0329] "Means for recognizing emotions" refers to technologies and devices that can identify a worker's psychological state and emotions in real time.

[0330] "Test results" refers to information such as the success or failure of the tests performed and error messages.

[0331] "Emotional data" is information about the psychological reactions and emotions of workers measured in real time.

[0332] "Means for recording and notifying" refers to technology for storing test results and emotional data and notifying them to users and other systems.

[0333] "Natural language processing technology" is a technology in the field of computer science that analyzes human language (natural language) and understands its meaning.

[0334] "Difference" refers to changes or differences between versions of source code, etc.

[0335] "Real-time" means that processing occurs as fast as real time.

[0336] This invention relates to a system that efficiently and effectively performs a series of processes: analyzing source code, generating verification patterns, conducting automated testing based on those patterns, and recognizing, recording, and notifying workers' emotional data in real time.

[0337] Receiving and analyzing the source code

[0338] The server has the function of receiving source code from users through a web interface. The source code is analyzed by the server using natural language processing technology and is expressed as an abstract syntax tree (AST) in the process.

[0339] Generating validation patterns

[0340] Based on the generated abstract syntax tree (AST), the server follows each path of the conditional branches and loops in the source code and automatically generates validation patterns. This takes into account each scenario and condition in the source code. For example, if the source code contains a conditional branch "if (userAge > 18)", the server will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less".

[0341] Differential analysis and test pattern generation

[0342] The terminal clones or pulls the latest source code from the version control system and obtains the difference from the previous commit. This difference is then transferred to the server, which analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[0343] Emotion engine integration

[0344] The server connects to the device used by the worker and integrates the emotion engine into the operation. The emotion engine recognizes and monitors the worker's emotional data in real time using devices such as smart glasses or head-mounted displays. This emotional data is used to generate verification patterns and conduct tests. For example, if a worker feels stressed or anxious while filling out a form, additional verification patterns are generated accordingly to verify how a specific operation affects the worker.

[0345] Automated testing

[0346] Users configure the server with automation tools, such as Selenium, through a web interface. The server then uses these automation tools to execute test scripts, which can include filling out forms and pressing buttons on web applications.

[0347] Recording and notifying test results

[0348] The server records test execution results and emotional data in a detailed log, including the success or failure of each test case, any error messages that occurred, and the emotional data of the tester. This record is then notified to the user via email or a dashboard.

[0349] Specific examples

[0350] For example, consider the case where a new product registration function is added to the web interface of a distribution center. The source code may contain the following new functions:

[0351] Input source code:

[0352] javascript

[0353] function addNewProduct(product) { if (user.isAuthenticated) { / Add product logic / } else { / Show error / }}

[0354] Example prompt sentence:

[0355] "I want to test whether the new product addition function works properly. I also want to make sure that the operation is not stressful for the workers."

[0356] The present invention makes it possible to improve the overall user experience, including the impact that system operation has on workers at a logistics center.

[0357] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0358] Step 1:

[0359] Receive source code from the user.

[0360] Users upload source code to the server through a web interface, and the server receives the source code and stores it in storage, providing the source code data to be analyzed as input.

[0361] Step 2:

[0362] It analyzes the source code and generates an abstract syntax tree (AST).

[0363] The server analyzes the stored source code using natural language processing technology (e.g., SpaCy). During this analysis, the structure of the source code is understood and represented as an abstract syntax tree (AST). The input is the source code, and the output is the analyzed AST.

[0364] Step 3:

[0365] Generates verification patterns based on an abstract syntax tree (AST).

[0366] The server detects conditional branches and loops in the source code based on the generated AST and generates validation patterns. For example, if there is a conditional branch "if (userAge > 18)", it will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less". The input is the AST, and the output is multiple validation patterns.

[0367] Step 4:

[0368] Recognize worker emotions in real time using an emotion engine.

[0369] Using smart glasses or a head-mounted display connected to a terminal, the worker's emotions are recognized in real time and data is collected. An emotion engine (e.g., OpenFace) analyzes the input from this device and obtains the worker's emotional state (e.g., stress, anxiety, satisfaction) as output.

[0370] Step 5:

[0371] Conduct automated testing and leverage sentiment data.

[0372] The server executes the test script based on the verification pattern generated using an automation tool (e.g., Selenium) selected by the user. During the test, the device acquires emotion data using an emotion engine and sends it to the server. The input is the verification pattern and emotion data, and the test result and emotion data are obtained as output.

[0373] Step 6:

[0374] Test results and emotional data will be recorded and communicated.

[0375] The server records test results and emotional data as detailed logs (e.g., Elasticsearch). The recorded data is used for analysis and notifications. The server notifies the user of the results via email or a dashboard, which outputs information about test success or failure and the emotional state of the worker, which the user can review.

[0376] Each step involves the following specific actions:

[0377] Step 1: Upload source code, receive and save data.

[0378] Step 2: Code analysis and AST generation using natural language processing techniques.

[0379] Step 3: Detect conditional branches and loops and generate verification patterns.

[0380] Step 4: Collecting emotion data using smart devices and analyzing it in real time.

[0381] Step 5: Configure and run the automated testing tool and integrate the sentiment data.

[0382] Step 6: Logging the results, saving them to a database, and implementing notifications.

[0383] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0384] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0385] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0386] [Second embodiment]

[0387] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0388] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0389] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0390] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0391] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0392] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0393] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0394] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0395] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0396] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0397] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0398] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0399] The present invention relates to a technique for automatically generating verification patterns and efficiently performing automatic testing in the process of verifying a system or network.

[0400] The system contains several key components, each performing a specific function. These components and their functions are described below.

[0401] Receiving and analyzing the source code

[0402] The server receives source code from the user. For example, the user uploads the source code through a web interface. The server analyzes the source code using natural language processing techniques. During the analysis, the structure of the source code (e.g., functions, conditional branches, loops, etc.) is represented as an abstract syntax tree (AST).

[0403] Generating validation patterns

[0404] The server automatically generates verification patterns based on each path of conditional branches and loops from the analyzed source code. These verification patterns cover a wide range of scenarios and conditions, preventing oversights in manual verification.

[0405] As a concrete example, if there is a conditional branch "if (userAge > 18)" in the source code, the server will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less."

[0406] Differential analysis and test pattern generation

[0407] The terminal clones or pulls the latest source code from a version control system (e.g., Git). The terminal then obtains the difference from the previous commit and transfers this difference to the server for analysis. The server analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[0408] As a specific example, if the difference includes "adding a new user login function," the server generates test patterns such as "when the user enters correct login information" and "when the user enters incorrect login information."

[0409] GUI-based verification and automated testing

[0410] Users configure automation tools (such as Selenium or Puppeteer) on the server, which then uses these automation tools to run test scripts, which can include filling out forms or pressing buttons on web applications.

[0411] As a concrete example, when a server automatically fills in a user registration form for a web application, it enters values ​​for each field (e.g., "user name," "email address," "password," etc.), clicks the submit button, and records the resulting message (success message or error message) that is displayed.

[0412] Recording and notifying test results

[0413] The server records the test execution results in a detailed log, including the success or failure of each test case and any error messages that occurred. This log is then notified to the user via email and a dashboard, where the user can view the results in detail.

[0414] As explained above, the system of the present invention can efficiently and effectively carry out a series of processes, from analyzing source code to generating verification patterns, analyzing differences, and conducting automated tests and notifying the results. This system eliminates the need for engineers to manually consider a huge number of verification patterns, significantly reducing the risk of missing verifications and improving the quality of system releases.

[0415] The processing flow will be explained below.

[0416] Step 1:

[0417] Users provide source code to the server, which can be uploaded through a web interface.

[0418] Step 2:

[0419] The server analyzes the received source code using natural language processing techniques to represent the code structure (functions, conditional branches, loops, etc.) as an abstract syntax tree (AST).

[0420] Step 3:

[0421] Based on the generated AST, the server traverses each path of the conditional branches and loops in the source code and automatically generates verification patterns, taking into account each scenario and condition in the source code.

[0422] Step 4:

[0423] The device clones or pulls the latest source code from the version control system (e.g. Git), which gives you the most up-to-date code base.

[0424] Step 5:

[0425] The terminal obtains the difference between the latest commit and the previous commit to identify what has been newly added or changed.

[0426] Step 6:

[0427] The device transfers the retrieved differences to the server, allowing the server to see the latest code changes.

[0428] Step 7:

[0429] The server analyzes the received differences, identifying any newly added or modified code sections.

[0430] Step 8:

[0431] Based on the analyzed differences, the server identifies existing verification patterns that may be affected and generates new test patterns.

[0432] Step 9:

[0433] Users configure the automation tools they want to use on their servers, for example by selecting tools such as Selenium or Puppeteer through a web interface.

[0434] Step 10:

[0435] The server runs the test scripts using configured automation tools, which may include filling out forms or pressing buttons on the web application.

[0436] Step 11:

[0437] The server records detailed test execution results in a log, including the success or failure of each test case and any error messages.

[0438] Step 12:

[0439] The server notifies the user of the recorded results via email and a dashboard, where the user can view the test results in detail.

[0440] Example 1

[0441] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0442] In today's complex software development, manual verification and testing not only requires a great deal of time and effort, but also carries a high risk of human error and missed verifications. Furthermore, because software is frequently changed due to program modifications and the implementation of additional features, a system is required to efficiently verify only the changed parts. However, it has been difficult to meet these requirements with conventional manual testing methods.

[0443] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0444] In this invention, the server includes means for receiving and analyzing software, means for generating a verification pattern from the analyzed software, means for performing an automated test based on the generated verification pattern, means for recording and notifying the test results, means for acquiring software differences from a version control system and generating a new verification pattern based on the differences, and means for uploading settings via a user interface and executing tests using an automated tool. This makes it possible to efficiently verify software changes, significantly reduce manual verification work, and reduce the risk of missing verification points.

[0445] "Software" is a set of instructions or programs designed to cause a computer to perform a specific function.

[0446] "Means of analysis" are methods or devices that analyze and interpret the contents of software in detail in order to understand its internal structure and operating logic.

[0447] A "verification pattern" is a test scenario based on specific situations or conditions designed to verify that software behaves as expected.

[0448] "Means for performing automated testing" are programs and tools that can be used to verify that software functions properly without human intervention.

[0449] "Means for recording and notifying test results" refers to a method or device for storing the results of an automated test and communicating that information to the user.

[0450] A "version control system" is a system for managing software change history and tracking and integrating different versions.

[0451] "Software differences" refer to the parts that have been changed or newly added compared to the previous version.

[0452] "User interface" is the collective term for the means and methods by which a user interacts with software or a system.

[0453] An "automation tool" is software or a program designed to perform routine tasks or processes automatically.

[0454] A "means of configuration upload" is a method by which a user enters or transmits specific parameters or configuration information into the system.

[0455] The present invention relates to a technology for automatically generating verification patterns and efficiently performing automated testing in the verification process of a system or network. This system includes important components, each of which performs a specific function.

[0456] Receiving and analyzing the source code

[0457] The server receives the software. Users upload the software through a web interface. The server analyzes the software using natural language processing techniques. During the analysis, the software structure (e.g., functions, conditionals, loops, etc.) is represented as an abstract syntax tree (AST).

[0458] As a concrete example, a user opens a browser, clicks the "Choose File" button in the interface of a web application, selects the file to upload, and clicks the "Submit" button.

[0459] Generating validation patterns

[0460] The server automatically generates verification patterns based on each path of conditional branches and loops from the analyzed software, allowing it to cover a wide range of scenarios and conditions.

[0461] For example, if the source code has a conditional branch if (userAge > 18), two validation patterns will be generated - "if userAge is greater than or equal to 19" and "if userAge is less than or equal to 18".

[0462] Differential analysis and test pattern generation

[0463] The terminal clones or pulls the latest software from a version control system (e.g., Git). The terminal then obtains the differences from the previous commit and transfers these differences to the server. The server analyzes the received differences, identifies newly added or modified code sections, and generates test patterns based on them.

[0464] As a specific example, if the difference includes "adding a new user login function," test patterns for "when the user enters correct login information" and "when the user enters incorrect login information" will be generated.

[0465] GUI-based verification and automated testing

[0466] Users configure automation tools (such as Selenium or Puppeteer) on the server, which then uses these automation tools to run test scripts, which can include filling out forms or pressing buttons on a web application.

[0467] As a concrete example, when a server automatically fills in a user registration form for a web application, it enters values ​​for each field (e.g., "user name," "email address," "password," etc.), clicks the submit button, and records the resulting message (success message or error message).

[0468] Recording and notifying test results

[0469] The server records the test execution results in a detailed log, including the success or failure of each test case and any error messages that occurred. This log is then notified to the user via email and a dashboard, where the user can view the results in detail.

[0470] For example, if the user has email notifications enabled, they will receive an email with the test result details attached.

[0471] Examples of prompt statements

[0472] Below are some example prompts for input to generative AI models for validation and automated testing systems:

[0473] In this system, users upload software, and the server analyzes the software and automatically generates verification and test patterns. It also retrieves differences from the version control system and generates test cases based on the added or modified parts. It runs tests using automated tools, records the results as detailed logs, and notifies the user. A specific example is the automatic filling of user registration forms and recording the submission results.

[0474] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0475] Step 1:

[0476] The user uploads the software through a web interface: the user opens a browser, clicks the "Choose File" button in the web application interface, selects the file to upload, and clicks the "Submit" button.

[0477] Input: User uploaded software file

[0478] Output: Software files sent to the server

[0479] Step 2:

[0480] The server receives the uploaded software and analyzes it using natural language processing techniques. Specifically, it converts the software into an abstract syntax tree (AST) using Python's ast module, and then uses this AST to analyze the code structure (e.g., function definitions, conditional branches, loops, etc.).

[0481] Input: Uploaded software file

[0482] Output: Software structure as an Abstract Syntax Tree (AST)

[0483] Step 3:

[0484] The server automatically generates verification patterns from the analyzed software. Specifically, it generates test cases that cover all possible execution paths based on conditional branching and loop structures. For example, if the condition is "if (userAge > 18)," it generates both scenarios: "when userAge is 19 or greater" and "when userAge is 18 or less."

[0485] Input: Abstract Syntax Tree (AST)

[0486] Output: Validation pattern

[0487] Step 4:

[0488] The terminal clones or pulls the latest software from a version control system (e.g., Git). Specifically, the terminal executes the git clone command or git pull command to obtain the latest code locally.

[0489] Input: Version control system repository information

[0490] Output: A local copy of the latest software code

[0491] Step 5:

[0492] The terminal obtains the difference from the previous commit and transfers this difference to the server. Specifically, it uses the git diff command to obtain the difference from the previous commit and transfers that data to the server.

[0493] Input: Locally cloned or pulled software code

[0494] Output: Differential data

[0495] Step 6:

[0496] The server analyzes the received differences to identify newly added or modified code parts. Specifically, it analyzes the difference data to identify new functions, modified conditional branches, etc.

[0497] Input: differential data

[0498] Output: Identified new or modified code sections

[0499] Step 7:

[0500] The server generates test patterns based on the identified code parts. Specifically, if the difference includes "adding a new user login function," test cases will be generated for "what happens when the user enters correct login information" and "what happens when the user enters incorrect login information."

[0501] Input: New or modified code fragment

[0502] Output: Test pattern

[0503] Step 8:

[0504] The user configures the automation tool (such as Selenium or Puppeteer) on the server. Specifically, the user uploads the automation tool's configuration file (e.g., config.json) from the web interface and enters the required settings (browser type, test script path, etc.).

[0505] Input: User configuration files and settings information

[0506] Output: Settings saved on the server

[0507] Step 9:

[0508] The server runs the test script using an automation tool, which includes filling out forms and pressing buttons in the web application. Specifically, the server uses the configured automation tool (e.g., Selenium WebDriver) to launch a browser, open a web application page, and run the test.

[0509] Input: Configured automation tools and test scripts

[0510] Output: Test execution results

[0511] Step 10:

[0512] The server records the test execution results as a detailed log. This log includes the success or failure of each test case, any error messages that occurred, etc. Specifically, the execution results of each test case (success, failure, error messages, etc.) are saved in a text file or database.

[0513] Input: Test execution results

[0514] Output: Detailed log data

[0515] Step 11:

[0516] The server notifies the user of the test results by sending the generated log file as an email attachment and / or displaying the results in a web interface dashboard.

[0517] Input: Detailed log data

[0518] Output: Notification to user (email, dashboard display, etc.)

[0519] (Application example 1)

[0520] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0521] Currently, there is a lack of efficient and highly accurate means for verifying the performance of robotic motion sequences and tasks in factory environments. This can lead to unexpected malfunctions, resulting in reduced operational efficiency and safety issues. Furthermore, manual verification is time-consuming and labor-intensive, and carries the risk of missing verification steps. Therefore, a more efficient and automated verification system is needed.

[0522] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0523] In this invention, the server includes means for receiving and analyzing source code, means for generating verification patterns from the analyzed source code, means for performing automated testing based on the generated verification patterns, means for recording and notifying test results, means for analyzing the operation sequences and task performance of a robot in a factory environment and generating verification patterns, and means for performing automated testing of the robot based on the generated verification patterns. This makes it possible to automatically verify the operations and tasks of factory robots, thereby improving operational efficiency and reliability.

[0524] "Source code" is a collection of textual instructions written to direct the operation of a program.

[0525] "Means for analyzing" refers to a method or device for analyzing input data or information and understanding its structure and content.

[0526] A "verification pattern" refers to a combination of various test scenarios and conditions to be performed, which serves as a basis for evaluating functionality and performance.

[0527] "Means for performing automated testing" refers to an automated method or device for performing testing without human intervention.

[0528] "Means for recording and notifying test results" refers to a method or device for storing the results of a test after it has been performed and notifying relevant parties.

[0529] "Factory environment" refers to the physical location where manufacturing or processing takes place and the systems and equipment used therein.

[0530] "Robot movement sequence" refers to the order and timing of movements performed by a robot.

[0531] "Task performance" is an indicator of how effectively a robot performs a specific task.

[0532] The "means for generating verification patterns" refers to a method or device that automatically creates appropriate verification methods or scenarios based on analyzed information.

[0533] "Means for performing automated testing of a robot" refers to a method or device for automatically verifying the operation or functionality of a robot.

[0534] "Natural language processing technology" refers to technology that enables computers to understand, interpret, and generate human language.

[0535] "Difference" is a concept that indicates changes between different versions of data or files.

[0536] A system for realizing the present invention includes the following configuration and processing procedures.

[0537] Receiving and analyzing the source code

[0538] The server receives source code from users. Users upload the source code through a web interface. The server analyzes the source code using natural language processing techniques. During the analysis, the structure of the source code (functions, conditional branches, loops, etc.) is represented as an abstract syntax tree (AST).

[0539] Generating validation patterns

[0540] The server automatically generates verification patterns based on each path of conditional branches and loops from the analyzed source code. These verification patterns cover a wide range of scenarios and conditions, preventing oversights in manual verification.

[0541] Differential analysis and test pattern generation

[0542] The terminal clones or pulls the latest source code from a version control system (e.g., Git). The terminal then obtains the difference from the previous commit and transfers this difference to the server for analysis. The server analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[0543] GUI-based verification and automated testing

[0544] Users configure automation tools (such as Selenium or Puppeteer) on the server, which then uses these automation tools to run test scripts, which can include filling out forms or pressing buttons on web applications.

[0545] Recording and notifying test results

[0546] The server records the test execution results in a detailed log, including the success or failure of each test case and any error messages that occurred. This log is then notified to the user via email and a dashboard, where the user can view the results in detail.

[0547] Additional processes when applied to factory robots

[0548] In a factory environment, a means is added to analyze the robot's operation sequence and task performance and generate verification patterns. The server analyzes the robot's operation log and sensor input data and derives the optimal verification pattern from the results. Then, automatic testing of the robot is performed based on the generated verification pattern.

[0549] Hardware and software used

[0550] The following hardware and software are used to implement this system:

[0551] Hardware: Servers, terminals, robots

[0552] Software: Git (version control system), Selenium (automation tool), Puppeteer (automation tool), natural language processing libraries (e.g., NLTK)

[0553] Examples of concrete examples and prompts

[0554] As a concrete example, when a server automatically fills in a user registration form for a web application, it enters values ​​for each field (e.g., "user name," "email address," "password," etc.), clicks the submit button, and records the resulting message (success message or error message) that is displayed.

[0555] An example of a prompt sentence to be input to the generative AI model is written as follows:

[0556] "Detect conditional branches and loops in the source code of factory robots and generate the necessary verification patterns based on them. Also, run automated tests based on the generated patterns and log the results."

[0557] With the above configuration, the present invention can automatically verify the operations and tasks of factory robots, thereby improving operational efficiency and reliability.

[0558] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0559] Step 1:

[0560] The server receives source code from the user. The user uploads the source code through a web interface. The server receives the uploaded source code and prepares the data for analysis. Specifically, the server saves the source code as a text file and prepares it for analysis. The input is the text data of the source code provided by the user, and the output is a source code file stored internally on the server.

[0561] Step 2:

[0562] The server analyzes the received source code. This analysis uses natural language processing techniques to represent the source code structure (functions, conditional branches, loops, etc.) as an abstract syntax tree (AST). Specifically, it runs the source code through an AST library to generate an AST object. The input is the source code file saved in step 1, and the output is the parsed abstract syntax tree object.

[0563] Step 3:

[0564] The server generates verification patterns from the parsed abstract syntax tree. It automatically generates verification patterns based on each path of conditional branches and loops. Specifically, it traverses each node of the AST and adds verification scenarios at points where conditional branches or loops are detected. The input is the abstract syntax tree object generated in step 2, and the output is a list of generated verification patterns.

[0565] Step 4:

[0566] The terminal clones or pulls the latest source code from a version control system (e.g., Git). Then, it gets the diff against the previous commit in the repository you are using. Specifically, the terminal runs Git commands to get the latest commit and extract the diff. The input is the remote Git repository, and the output is the diff data between the old and new commits.

[0567] Step 5:

[0568] The terminal transfers the acquired differential data to the server. The server analyzes the received differential data, identifies newly added or modified code parts, and generates a new verification pattern based on them. The input is the differential data acquired in step 4, and the output is the newly generated verification pattern.

[0569] Step 6:

[0570] The user configures automation tools (such as Selenium or Puppeteer) on the server. The server then uses these automation tools to execute test scripts. Specifically, the server starts the automation tools and automatically simulates the operation of web applications or robots based on the configuration. The input is the automation script configured by the user, and the output is the test execution results.

[0571] Step 7:

[0572] The server records the test execution results as a detailed log. This log includes the success or failure of each test case, any error messages that occurred, etc. The server notifies the user of this record. Specifically, the server generates log data and notifies the user via email or the dashboard. The input is the test execution results obtained in step 6, and the output is the generated log data and notification message.

[0573] Step 8:

[0574] In a factory environment, a server analyzes the robot's operation log and sensor input data, and derives the optimal verification pattern from the results. Based on the generated verification pattern, an automatic test of the robot is performed. The input is the robot's operation log and sensor input data, and the output is the results of the automatic test. A specific example includes a test to confirm whether the robot operates accurately based on an operation scenario.

[0575] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0576] This invention relates to an automatic testing system that combines an emotion engine in the verification process of systems and networks. This system efficiently and effectively performs a series of processes that incorporates an emotion engine, from analyzing source code to generating verification patterns, conducting automatic tests, and recording and notifying test results.

[0577] The system includes the following main components:

[0578] Receiving and analyzing the source code

[0579] The server receives source code from users. Users upload the source code through a web interface. The server analyzes the source code using natural language processing techniques. During the analysis process, the structure of the source code (functions, conditional branches, loops, etc.) is represented as an abstract syntax tree (AST).

[0580] Generating validation patterns

[0581] Based on the generated AST, the server traverses each path of the conditional branches and loops in the source code and automatically generates verification patterns, taking into account each scenario and condition in the source code.

[0582] As a concrete example, if there is a conditional branch "if (userAge > 18)" in the source code, the server will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less."

[0583] Differential analysis and test pattern generation

[0584] The terminal clones or pulls the latest source code from a version control system (e.g., Git). The terminal obtains the difference from the previous commit and transfers this difference to the server for analysis. The server analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[0585] As a specific example, if the difference includes "adding a new user login function," the server generates test patterns such as "when the user enters correct login information" and "when the user enters incorrect login information."

[0586] Emotion engine integration

[0587] The server incorporates an emotion engine into its operation. The emotion engine recognizes and monitors the user's emotions in real time while operating the user interface. This emotion data is used to generate verification patterns and conduct tests.

[0588] As a specific example, if a user experiences stress or anxiety when filling out a form, additional validation patterns are generated to verify the impact of specific operations on the user.

[0589] Automated testing

[0590] Users configure automation tools on the server, for example by selecting tools such as Selenium or Puppeteer through a web interface. The server then uses these automation tools to run test scripts, which can include filling out forms or pressing buttons on web applications.

[0591] Recording and notifying test results

[0592] The server records the test execution results in a detailed log, including the success or failure of each test case, any error messages that occurred, etc. This log is then sent to the user via email or a dashboard.

[0593] As explained above, the system of the present invention can achieve more comprehensive and high-quality verification by analyzing source code, generating verification patterns, analyzing differences, conducting automated tests, and notifying the results, as well as taking into account the user's psychological reactions using an emotion engine. This system eliminates the need for engineers to manually consider a huge number of verification patterns, significantly reducing the risk of missing verifications and improving the quality of system releases.

[0594] The processing flow will be explained below.

[0595] Step 1:

[0596] Users provide source code to the server, which can be uploaded through a web interface.

[0597] Step 2:

[0598] The server analyzes the received source code using natural language processing techniques to represent the code structure (functions, conditional branches, loops, etc.) as an abstract syntax tree (AST).

[0599] Step 3:

[0600] Based on the generated AST, the server traverses each path of the conditional branches and loops in the source code and automatically generates verification patterns that cover a wide range of scenarios and conditions.

[0601] Step 4:

[0602] The device clones or pulls the latest source code from the version control system (e.g. Git), which gives you the most up-to-date code base.

[0603] Step 5:

[0604] The terminal obtains the difference between the latest commit and the previous commit, which identifies what has been added or changed.

[0605] Step 6:

[0606] The device transfers the retrieved differences to the server, allowing the server to see the latest code changes.

[0607] Step 7:

[0608] The server analyzes the received differences, identifying any newly added or modified code sections.

[0609] Step 8:

[0610] Based on the analyzed differences, the server identifies existing verification patterns that may be affected and generates new test patterns.

[0611] Step 9:

[0612] Users configure the server with the automation tools they want to use, for example, by selecting tools such as Selenium or Puppeteer through a web interface.

[0613] Step 10:

[0614] The server runs the test scripts using configured automation tools, which may include filling out forms or pressing buttons on the web application.

[0615] Step 11:

[0616] The server activates an emotion engine to recognize the user's emotions in real time. The emotion engine monitors the user's facial expressions and tone of voice while operating the user interface.

[0617] Step 12:

[0618] The server analyzes the recognized emotion data and generates additional verification patterns based on that data. By providing special verification for operations that cause stress or confusion to the user, the user experience is improved.

[0619] Step 13:

[0620] The server records detailed logs of the test execution results, including the success or failure of each test case, error messages, and the user's emotional state.

[0621] Step 14:

[0622] The server notifies the user of the recorded results via email and a dashboard, where the user can view the test results in detail.

[0623] Through these steps, this system effectively and efficiently verifies source code, and by taking into account user feelings in particular, it supports the development of higher quality, user-friendly software.

[0624] Example 2

[0625] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0626] Conventional automated testing systems focus on recognizing and analyzing source code, but no systems have taken into account user emotions and psychological responses. As a result, improvements to the user experience and the generation of emotion-based verification patterns have not been fully realized. Furthermore, the generation of verification patterns based on differential analysis of source code changes and additions has sometimes been inefficient. A solution to these problems was needed.

[0627] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving and analyzing source code, means for generating a verification pattern from the analyzed source code, means for performing an automated test based on the generated verification pattern, means for recognizing a user's emotions in real time while operating a user interface and generating a verification pattern based on the emotions, means for recording and notifying the test results, and means for acquiring differences in the source code and generating a new verification pattern based on the differences. This enables high-quality automated testing that takes into account the user's emotions and psychological reactions, and also realizes efficient generation of verification patterns for modified source code.

[0628] "Source code" is the set of instructions or statements that make up a program or application, usually written as a text file.

[0629] "Analysis" is the operation of breaking down given information or data into detail and clarifying its structure and relationships.

[0630] A "verification pattern" refers to a series of test cases or scenarios that should be verified by automated testing based on source code conditions or scenarios.

[0631] "Automated testing" is the process of verifying the functionality and performance of software or applications using automated tools and scripts.

[0632] "Test Results" refers to the information and data obtained after an automated test is performed, including success / failure status and error messages.

[0633] "Notifying" is the action or process of making certain information or results known to others.

[0634] "User interface" refers to the interaction means and screen layout that allow a user to interact with a computer system or application.

[0635] "Emotion" refers to the psychological state or reaction that a user feels during operation, and includes states such as joy, sadness, and surprise.

[0636] "Natural language processing technology" is a technology that allows computers to understand human language and perform analysis and generation.

[0637] "Delta" refers to changes or differences between different versions of data or code.

[0638] The present invention is a system that efficiently performs everything from analyzing source code to generating verification patterns, conducting automated tests, and recording and notifying test results. It realizes higher quality testing by recognizing user emotions in real time and utilizing them in generating verification patterns. The main components and operating procedures for implementing this invention are described in detail below.

[0639] System Configuration

[0640] This system mainly includes the following hardware and software:

[0641] Server: The central component that analyzes source code, generates verification patterns, manages automated tests, records and notifies test results, etc.

[0642] Terminal: Retrieves source code from a version control system (e.g., Git), analyzes the differences, and sends them to the server.

[0643] Emotion engine: Recognizes the user's emotions in real time while operating the user interface and sends the data to the server.

[0644] The specific software technologies used include:

[0645] Natural language processing technology: Technology for analyzing source code (e.g., Python's ast module).

[0646] Automation tools: Tools for performing automated testing (e.g., Selenium, Puppeteer).

[0647] Receiving and analyzing the source code

[0648] Users upload source code to the server through a web interface. Once the source code is uploaded, the server receives it and analyzes it using natural language processing technology. Specifically, the source code structure (functions, conditional branches, loops, etc.) is represented as an abstract syntax tree (AST) and the analysis results are obtained.

[0649] Generating validation patterns

[0650] Based on the generated AST, the server follows each path of conditional branches and loops in the source code and automatically generates validation patterns. For example, if the condition "if (userAge > 18)" is included, the server will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less."

[0651] Differential analysis and test pattern generation

[0652] The terminal clones or pulls the latest source code from the version control system, obtains the difference from the previous commit, and transfers this difference to the server. The server analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[0653] Emotion engine integration

[0654] The server integrates an emotion engine into the system. The emotion engine recognizes the user's emotions in real time while operating the user interface and sends the data to the server. For example, if the user feels stress or anxiety while filling out a form, the emotion engine detects this and the server generates additional verification patterns based on the emotion data.

[0655] Running automated tests and recording results

[0656] Users configure automation tools using a web interface, for example by selecting a tool such as Selenium or Puppeteer, and saving the automated test configuration. The server then executes test scripts using the selected automation tool, including form entries and button presses in web applications. After the tests are executed, the server records detailed test results in a log and notifies the user via email or a dashboard.

[0657] Examples of concrete examples and prompts

[0658] For example, if the source code contains a conditional branch "if (userAge > 18)", the server will generate two validation patterns - "if userAge is 19 or greater" and "if userAge is 18 or less".

[0659] Example prompt sentence:

[0660] We have added a new user login feature. Please generate test patterns for this feature that show correct and incorrect login information.

[0661] As described above, this embodiment makes it possible to carry out an automatic test that takes into account the user's emotions, and realizes a system that can obtain high-quality test results.

[0662] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0663] Step 1:

[0664] The user uploads source code to the server through a web interface. The input is a source code file, and the output is the source code saved on the server. Specifically, the user selects the target source code file using a file selection dialog and presses the "Upload" button.

[0665] Step 2:

[0666] After the server receives the uploaded source code, it analyzes it using natural language processing technology. The input is the source code file uploaded by the user, and the output is a parsed abstract syntax tree (AST). Specifically, it uses the Python ast module to analyze the structure of the source code and extract components such as functions, conditional branches, and loops.

[0667] Step 3:

[0668] Based on the generated AST, the server follows each path of conditional branches and loops in the source code and automatically generates verification patterns. The input is the AST, and the output is a list of verification patterns. Specifically, test cases are generated by dividing the scenarios into conditions for conditional branches (e.g., if (userAge > 18)). For example, two patterns are generated: "if userAge is 19 or greater" and "if userAge is 18 or less."

[0669] Step 4:

[0670] The terminal clones or pulls the latest source code from the version control system. The input is the repository URL and authentication information, and the output is the latest source code stored locally. Next, the terminal obtains the difference from the previous commit and sends that difference information to the server. Specifically, it uses Git commands to check the difference and sends the difference file to the server.

[0671] Step 5:

[0672] The server analyzes the received differences and identifies newly added or modified code sections. The input is the difference file, and the output is a list of changed sections. It then generates new test patterns based on these changes. Specifically, if the change includes "adding a new user login function," it generates test patterns that cover various scenarios for the login function (correct information, incorrect information).

[0673] Step 6:

[0674] The server integrates an emotion engine into the system and recognizes the user's emotions in real time while operating the user interface. The input is the user's operation data, and the output is the recognized emotion data. Based on this emotion data, the server generates additional verification patterns. For example, if the user feels stressed while filling out a form, the server detects that emotion and creates additional verification patterns accordingly.

[0675] Step 7:

[0676] The user configures the automation tool using a web interface. The input is the selection information for the tool, such as Selenium or Puppeteer, and the output is the configuration information saved on the server. Specifically, the user selects the automation tool on the configuration screen and presses the "Save" button.

[0677] Step 8:

[0678] The server runs the test script using an automation tool configured by the user. The input is the verification pattern and the automation tool's configuration information, and the output is the test execution results. Specifically, Puppeteer is used to automatically operate the browser and perform form input and button presses in the web application.

[0679] Step 9:

[0680] The server records the test execution results as a detailed log and notifies the user via email or dashboard. The input is the test execution results, and the output is notification information. Specifically, the success or failure of test cases, any error messages that occur, etc. are recorded in detail and notified to the user.

[0681] (Application example 2)

[0682] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0683] Current automated testing systems do not take into account the user's emotions and psychological state, making it difficult to reduce the stress and anxiety of operators, especially in real-world environments such as logistics centers. This leads to increased frustration and errors in system operation, hindering efficient business operations. Furthermore, analysis of test results does not include emotional data, resulting in a lack of information useful for improving the user experience.

[0684] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing source code, means for generating a verification pattern from the analyzed source code, means for recognizing the worker's emotions, means for performing an automatic test based on the generated verification pattern, and means for recording and notifying the test results and emotional data. This makes it possible to recognize the worker's emotions in real time and generate more realistic and effective verification patterns based on that data. Furthermore, by performing post-processing analysis based on the emotional data, it is possible to improve the user experience and reduce operational stress.

[0685] "Source code" is a set of textual instructions that make up a program or application.

[0686] "Analysis" is the process of understanding specific data or information and clarifying its meaning and structure.

[0687] A "verification pattern" is a specific condition or scenario designed to test the behavior of source code or a system.

[0688] "Automated testing" is the process of executing test cases using automated tools and checking the results.

[0689] "Means for recognizing emotions" refers to technologies and devices that can identify a worker's psychological state and emotions in real time.

[0690] "Test results" refers to information such as the success or failure of the tests performed and error messages.

[0691] "Emotional data" is information about the psychological reactions and emotions of workers measured in real time.

[0692] "Means for recording and notifying" refers to technology for storing test results and emotional data and notifying them to users and other systems.

[0693] "Natural language processing technology" is a technology in the field of computer science that analyzes human language (natural language) and understands its meaning.

[0694] "Difference" refers to changes or differences between versions of source code, etc.

[0695] "Real-time" means that processing occurs as fast as real time.

[0696] This invention relates to a system that efficiently and effectively performs a series of processes: analyzing source code, generating verification patterns, conducting automated testing based on those patterns, and recognizing, recording, and notifying workers' emotional data in real time.

[0697] Receiving and analyzing the source code

[0698] The server has the function of receiving source code from users through a web interface. The source code is analyzed by the server using natural language processing technology and is expressed as an abstract syntax tree (AST) in the process.

[0699] Generating validation patterns

[0700] Based on the generated abstract syntax tree (AST), the server follows each path of the conditional branches and loops in the source code and automatically generates validation patterns. This takes into account each scenario and condition in the source code. For example, if the source code contains a conditional branch "if (userAge > 18)", the server will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less".

[0701] Differential analysis and test pattern generation

[0702] The terminal clones or pulls the latest source code from the version control system and obtains the difference from the previous commit. This difference is then transferred to the server, which analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[0703] Emotion engine integration

[0704] The server connects to the device used by the worker and integrates the emotion engine into the operation. The emotion engine recognizes and monitors the worker's emotional data in real time using devices such as smart glasses or head-mounted displays. This emotional data is used to generate verification patterns and conduct tests. For example, if a worker feels stressed or anxious while filling out a form, additional verification patterns are generated accordingly to verify how a specific operation affects the worker.

[0705] Automated testing

[0706] Users configure the server with automation tools, such as Selenium, through a web interface. The server then uses these automation tools to execute test scripts, which can include filling out forms and pressing buttons on web applications.

[0707] Recording and notifying test results

[0708] The server records test execution results and emotional data in a detailed log, including the success or failure of each test case, any error messages that occurred, and the emotional data of the tester. This record is then notified to the user via email or a dashboard.

[0709] Specific examples

[0710] For example, consider the case where a new product registration function is added to the web interface of a distribution center. The source code may contain the following new functions:

[0711] Input source code:

[0712] javascript

[0713] function addNewProduct(product) { if (user.isAuthenticated) { / Add product logic / } else { / Show error / }}

[0714] Example prompt sentence:

[0715] "I want to test whether the new product addition function works properly. I also want to make sure that the operation is not stressful for the workers."

[0716] The present invention makes it possible to improve the overall user experience, including the impact that system operation has on workers at a logistics center.

[0717] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0718] Step 1:

[0719] Receive source code from the user.

[0720] Users upload source code to the server through a web interface, and the server receives the source code and stores it in storage, providing the source code data to be analyzed as input.

[0721] Step 2:

[0722] It analyzes the source code and generates an abstract syntax tree (AST).

[0723] The server analyzes the stored source code using natural language processing technology (e.g., SpaCy). During this analysis, the structure of the source code is understood and represented as an abstract syntax tree (AST). The input is the source code, and the output is the analyzed AST.

[0724] Step 3:

[0725] Generates verification patterns based on an abstract syntax tree (AST).

[0726] The server detects conditional branches and loops in the source code based on the generated AST and generates validation patterns. For example, if there is a conditional branch "if (userAge > 18)", it will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less". The input is the AST, and the output is multiple validation patterns.

[0727] Step 4:

[0728] Recognize worker emotions in real time using an emotion engine.

[0729] Using smart glasses or a head-mounted display connected to a terminal, the worker's emotions are recognized in real time and data is collected. An emotion engine (e.g., OpenFace) analyzes the input from this device and obtains the worker's emotional state (e.g., stress, anxiety, satisfaction) as output.

[0730] Step 5:

[0731] Conduct automated testing and leverage sentiment data.

[0732] The server executes the test script based on the verification pattern generated using an automation tool (e.g., Selenium) selected by the user. During the test, the device acquires emotion data using an emotion engine and sends it to the server. The input is the verification pattern and emotion data, and the test result and emotion data are obtained as output.

[0733] Step 6:

[0734] Test results and emotional data will be recorded and communicated.

[0735] The server records test results and emotional data as detailed logs (e.g., Elasticsearch). The recorded data is used for analysis and notifications. The server notifies the user of the results via email or a dashboard, which outputs information about test success or failure and the emotional state of the worker, which the user can review.

[0736] Each step involves the following specific actions:

[0737] Step 1: Upload source code, receive and save data.

[0738] Step 2: Code analysis and AST generation using natural language processing techniques.

[0739] Step 3: Detect conditional branches and loops and generate verification patterns.

[0740] Step 4: Collecting emotion data using smart devices and analyzing it in real time.

[0741] Step 5: Configure and run the automated testing tool and integrate the sentiment data.

[0742] Step 6: Logging the results, saving them to a database, and implementing notifications.

[0743] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0744] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0745] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0746] [Third embodiment]

[0747] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0748] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0749] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0750] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0751] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0752] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0753] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0754] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0755] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0756] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0757] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0758] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0759] The present invention relates to a technique for automatically generating verification patterns and efficiently performing automatic testing in the process of verifying a system or network.

[0760] The system contains several key components, each performing a specific function. These components and their functions are described below.

[0761] Receiving and analyzing the source code

[0762] The server receives source code from the user. For example, the user uploads the source code through a web interface. The server analyzes the source code using natural language processing techniques. During the analysis, the structure of the source code (e.g., functions, conditional branches, loops, etc.) is represented as an abstract syntax tree (AST).

[0763] Generating validation patterns

[0764] The server automatically generates verification patterns based on each path of conditional branches and loops from the analyzed source code. These verification patterns cover a wide range of scenarios and conditions, preventing oversights in manual verification.

[0765] As a concrete example, if there is a conditional branch "if (userAge > 18)" in the source code, the server will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less."

[0766] Differential analysis and test pattern generation

[0767] The terminal clones or pulls the latest source code from a version control system (e.g., Git). The terminal then obtains the difference from the previous commit and transfers this difference to the server for analysis. The server analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[0768] As a specific example, if the difference includes "adding a new user login function," the server generates test patterns such as "when the user enters correct login information" and "when the user enters incorrect login information."

[0769] GUI-based verification and automated testing

[0770] Users configure automation tools (such as Selenium or Puppeteer) on the server, which then uses these automation tools to run test scripts, which can include filling out forms or pressing buttons on web applications.

[0771] As a concrete example, when a server automatically fills in a user registration form for a web application, it enters values ​​for each field (e.g., "user name," "email address," "password," etc.), clicks the submit button, and records the resulting message (success message or error message) that is displayed.

[0772] Recording and notifying test results

[0773] The server records the test execution results in a detailed log, including the success or failure of each test case and any error messages that occurred. This log is then notified to the user via email and a dashboard, where the user can view the results in detail.

[0774] As explained above, the system of the present invention can efficiently and effectively carry out a series of processes, from analyzing source code to generating verification patterns, analyzing differences, and conducting automated tests and notifying the results. This system eliminates the need for engineers to manually consider a huge number of verification patterns, significantly reducing the risk of missing verifications and improving the quality of system releases.

[0775] The processing flow will be explained below.

[0776] Step 1:

[0777] Users provide source code to the server, which can be uploaded through a web interface.

[0778] Step 2:

[0779] The server analyzes the received source code using natural language processing techniques to represent the code structure (functions, conditional branches, loops, etc.) as an abstract syntax tree (AST).

[0780] Step 3:

[0781] Based on the generated AST, the server traverses each path of the conditional branches and loops in the source code and automatically generates verification patterns, taking into account each scenario and condition in the source code.

[0782] Step 4:

[0783] The device clones or pulls the latest source code from the version control system (e.g. Git), which gives you the most up-to-date code base.

[0784] Step 5:

[0785] The terminal obtains the difference between the latest commit and the previous commit to identify what has been newly added or changed.

[0786] Step 6:

[0787] The device transfers the retrieved differences to the server, allowing the server to see the latest code changes.

[0788] Step 7:

[0789] The server analyzes the received differences, identifying any newly added or modified code sections.

[0790] Step 8:

[0791] Based on the analyzed differences, the server identifies existing verification patterns that may be affected and generates new test patterns.

[0792] Step 9:

[0793] Users configure the automation tools they want to use on their servers, for example by selecting tools such as Selenium or Puppeteer through a web interface.

[0794] Step 10:

[0795] The server runs the test scripts using configured automation tools, which may include filling out forms or pressing buttons on the web application.

[0796] Step 11:

[0797] The server records detailed test execution results in a log, including the success or failure of each test case and any error messages.

[0798] Step 12:

[0799] The server notifies the user of the recorded results via email and a dashboard, where the user can view the test results in detail.

[0800] Example 1

[0801] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0802] In today's complex software development, manual verification and testing not only requires a great deal of time and effort, but also carries a high risk of human error and missed verifications. Furthermore, because software is frequently changed due to program modifications and the implementation of additional features, a system is required to efficiently verify only the changed parts. However, it has been difficult to meet these requirements with conventional manual testing methods.

[0803] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0804] In this invention, the server includes means for receiving and analyzing software, means for generating a verification pattern from the analyzed software, means for performing an automated test based on the generated verification pattern, means for recording and notifying the test results, means for acquiring software differences from a version control system and generating a new verification pattern based on the differences, and means for uploading settings via a user interface and executing tests using an automated tool. This makes it possible to efficiently verify software changes, significantly reduce manual verification work, and reduce the risk of missing verification points.

[0805] "Software" is a set of instructions or programs designed to cause a computer to perform a specific function.

[0806] "Means of analysis" are methods or devices that analyze and interpret the contents of software in detail in order to understand its internal structure and operating logic.

[0807] A "verification pattern" is a test scenario based on specific situations or conditions designed to verify that software behaves as expected.

[0808] "Means for performing automated testing" are programs and tools that can be used to verify that software functions properly without human intervention.

[0809] "Means for recording and notifying test results" refers to a method or device for storing the results of an automated test and communicating that information to the user.

[0810] A "version control system" is a system for managing software change history and tracking and integrating different versions.

[0811] "Software differences" refer to the parts that have been changed or newly added compared to the previous version.

[0812] "User interface" is the collective term for the means and methods by which a user interacts with software or a system.

[0813] An "automation tool" is software or a program designed to perform routine tasks or processes automatically.

[0814] A "means of configuration upload" is a method by which a user enters or transmits specific parameters or configuration information into the system.

[0815] The present invention relates to a technology for automatically generating verification patterns and efficiently performing automated testing in the verification process of a system or network. This system includes important components, each of which performs a specific function.

[0816] Receiving and analyzing the source code

[0817] The server receives the software. Users upload the software through a web interface. The server analyzes the software using natural language processing techniques. During the analysis, the software structure (e.g., functions, conditionals, loops, etc.) is represented as an abstract syntax tree (AST).

[0818] As a concrete example, a user opens a browser, clicks the "Choose File" button in the interface of a web application, selects the file to upload, and clicks the "Submit" button.

[0819] Generating validation patterns

[0820] The server automatically generates verification patterns based on each path of conditional branches and loops from the analyzed software, allowing it to cover a wide range of scenarios and conditions.

[0821] For example, if the source code has a conditional branch if (userAge > 18), two validation patterns will be generated - "if userAge is greater than or equal to 19" and "if userAge is less than or equal to 18".

[0822] Differential analysis and test pattern generation

[0823] The terminal clones or pulls the latest software from a version control system (e.g., Git). The terminal then obtains the differences from the previous commit and transfers these differences to the server. The server analyzes the received differences, identifies newly added or modified code sections, and generates test patterns based on them.

[0824] As a specific example, if the difference includes "adding a new user login function," test patterns for "when the user enters correct login information" and "when the user enters incorrect login information" will be generated.

[0825] GUI-based verification and automated testing

[0826] Users configure automation tools (such as Selenium or Puppeteer) on the server, which then uses these automation tools to run test scripts, which can include filling out forms or pressing buttons on a web application.

[0827] As a concrete example, when a server automatically fills in a user registration form for a web application, it enters values ​​for each field (e.g., "user name," "email address," "password," etc.), clicks the submit button, and records the resulting message (success message or error message).

[0828] Recording and notifying test results

[0829] The server records the test execution results in a detailed log, including the success or failure of each test case and any error messages that occurred. This log is then notified to the user via email and a dashboard, where the user can view the results in detail.

[0830] For example, if the user has email notifications enabled, they will receive an email with the test result details attached.

[0831] Examples of prompt statements

[0832] Below are some example prompts for input to generative AI models for validation and automated testing systems:

[0833] In this system, users upload software, and the server analyzes the software and automatically generates verification and test patterns. It also retrieves differences from the version control system and generates test cases based on the added or modified parts. It runs tests using automated tools, records the results as detailed logs, and notifies the user. A specific example is the automatic filling of user registration forms and recording the submission results.

[0834] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0835] Step 1:

[0836] The user uploads the software through a web interface: the user opens a browser, clicks the "Choose File" button in the web application interface, selects the file to upload, and clicks the "Submit" button.

[0837] Input: User uploaded software file

[0838] Output: Software files sent to the server

[0839] Step 2:

[0840] The server receives the uploaded software and analyzes it using natural language processing techniques. Specifically, it converts the software into an abstract syntax tree (AST) using Python's ast module, and then uses this AST to analyze the code structure (e.g., function definitions, conditional branches, loops, etc.).

[0841] Input: Uploaded software file

[0842] Output: Software structure as an Abstract Syntax Tree (AST)

[0843] Step 3:

[0844] The server automatically generates verification patterns from the analyzed software. Specifically, it generates test cases that cover all possible execution paths based on conditional branching and loop structures. For example, if the condition is "if (userAge > 18)," it generates both scenarios: "when userAge is 19 or greater" and "when userAge is 18 or less."

[0845] Input: Abstract Syntax Tree (AST)

[0846] Output: Validation pattern

[0847] Step 4:

[0848] The terminal clones or pulls the latest software from a version control system (e.g., Git). Specifically, the terminal executes the git clone command or git pull command to obtain the latest code locally.

[0849] Input: Version control system repository information

[0850] Output: A local copy of the latest software code

[0851] Step 5:

[0852] The terminal obtains the difference from the previous commit and transfers this difference to the server. Specifically, it uses the git diff command to obtain the difference from the previous commit and transfers that data to the server.

[0853] Input: Locally cloned or pulled software code

[0854] Output: Differential data

[0855] Step 6:

[0856] The server analyzes the received differences to identify newly added or modified code parts. Specifically, it analyzes the difference data to identify new functions, modified conditional branches, etc.

[0857] Input: differential data

[0858] Output: Identified new or modified code sections

[0859] Step 7:

[0860] The server generates test patterns based on the identified code parts. Specifically, if the difference includes "adding a new user login function," test cases will be generated for "what happens when the user enters correct login information" and "what happens when the user enters incorrect login information."

[0861] Input: New or modified code fragment

[0862] Output: Test pattern

[0863] Step 8:

[0864] The user configures the automation tool (such as Selenium or Puppeteer) on the server. Specifically, the user uploads the automation tool's configuration file (e.g., config.json) from the web interface and enters the required settings (browser type, test script path, etc.).

[0865] Input: User configuration files and settings information

[0866] Output: Settings saved on the server

[0867] Step 9:

[0868] The server runs the test script using an automation tool, which includes filling out forms and pressing buttons in the web application. Specifically, the server uses the configured automation tool (e.g., Selenium WebDriver) to launch a browser, open a web application page, and run the test.

[0869] Input: Configured automation tools and test scripts

[0870] Output: Test execution results

[0871] Step 10:

[0872] The server records the test execution results as a detailed log. This log includes the success or failure of each test case, any error messages that occurred, etc. Specifically, the execution results of each test case (success, failure, error messages, etc.) are saved in a text file or database.

[0873] Input: Test execution results

[0874] Output: Detailed log data

[0875] Step 11:

[0876] The server notifies the user of the test results by sending the generated log file as an email attachment and / or displaying the results in a web interface dashboard.

[0877] Input: Detailed log data

[0878] Output: Notification to user (email, dashboard display, etc.)

[0879] (Application example 1)

[0880] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0881] Currently, there is a lack of efficient and highly accurate means for verifying the performance of robotic motion sequences and tasks in factory environments. This can lead to unexpected malfunctions, resulting in reduced operational efficiency and safety issues. Furthermore, manual verification is time-consuming and labor-intensive, and carries the risk of missing verification steps. Therefore, a more efficient and automated verification system is needed.

[0882] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0883] In this invention, the server includes means for receiving and analyzing source code, means for generating verification patterns from the analyzed source code, means for performing automated testing based on the generated verification patterns, means for recording and notifying test results, means for analyzing the operation sequences and task performance of a robot in a factory environment and generating verification patterns, and means for performing automated testing of the robot based on the generated verification patterns. This makes it possible to automatically verify the operations and tasks of factory robots, thereby improving operational efficiency and reliability.

[0884] "Source code" is a collection of textual instructions written to direct the operation of a program.

[0885] "Means for analyzing" refers to a method or device for analyzing input data or information and understanding its structure and content.

[0886] A "verification pattern" refers to a combination of various test scenarios and conditions to be performed, which serves as a basis for evaluating functionality and performance.

[0887] "Means for performing automated testing" refers to an automated method or device for performing testing without human intervention.

[0888] "Means for recording and notifying test results" refers to a method or device for storing the results of a test after it has been performed and notifying relevant parties.

[0889] "Factory environment" refers to the physical location where manufacturing or processing takes place and the systems and equipment used therein.

[0890] "Robot movement sequence" refers to the order and timing of movements performed by a robot.

[0891] "Task performance" is an indicator of how effectively a robot performs a specific task.

[0892] The "means for generating verification patterns" refers to a method or device that automatically creates appropriate verification methods or scenarios based on analyzed information.

[0893] "Means for performing automated testing of a robot" refers to a method or device for automatically verifying the operation or functionality of a robot.

[0894] "Natural language processing technology" refers to technology that enables computers to understand, interpret, and generate human language.

[0895] "Difference" is a concept that indicates changes between different versions of data or files.

[0896] A system for realizing the present invention includes the following configuration and processing procedures.

[0897] Receiving and analyzing the source code

[0898] The server receives source code from users. Users upload the source code through a web interface. The server analyzes the source code using natural language processing techniques. During the analysis, the structure of the source code (functions, conditional branches, loops, etc.) is represented as an abstract syntax tree (AST).

[0899] Generating validation patterns

[0900] The server automatically generates verification patterns based on each path of conditional branches and loops from the analyzed source code. These verification patterns cover a wide range of scenarios and conditions, preventing oversights in manual verification.

[0901] Differential analysis and test pattern generation

[0902] The terminal clones or pulls the latest source code from a version control system (e.g., Git). The terminal then obtains the difference from the previous commit and transfers this difference to the server for analysis. The server analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[0903] GUI-based verification and automated testing

[0904] Users configure automation tools (such as Selenium or Puppeteer) on the server, which then uses these automation tools to run test scripts, which can include filling out forms or pressing buttons on web applications.

[0905] Recording and notifying test results

[0906] The server records the test execution results in a detailed log, including the success or failure of each test case and any error messages that occurred. This log is then notified to the user via email and a dashboard, where the user can view the results in detail.

[0907] Additional processes when applied to factory robots

[0908] In a factory environment, a means is added to analyze the robot's operation sequence and task performance and generate verification patterns. The server analyzes the robot's operation log and sensor input data and derives the optimal verification pattern from the results. Then, automatic testing of the robot is performed based on the generated verification pattern.

[0909] Hardware and software used

[0910] The following hardware and software are used to implement this system:

[0911] Hardware: Servers, terminals, robots

[0912] Software: Git (version control system), Selenium (automation tool), Puppeteer (automation tool), natural language processing libraries (e.g., NLTK)

[0913] Examples of concrete examples and prompts

[0914] As a concrete example, when a server automatically fills in a user registration form for a web application, it enters values ​​for each field (e.g., "user name," "email address," "password," etc.), clicks the submit button, and records the resulting message (success message or error message) that is displayed.

[0915] An example of a prompt sentence to be input to the generative AI model is written as follows:

[0916] "Detect conditional branches and loops in the source code of factory robots and generate the necessary verification patterns based on them. Also, run automated tests based on the generated patterns and log the results."

[0917] With the above configuration, the present invention can automatically verify the operations and tasks of factory robots, thereby improving operational efficiency and reliability.

[0918] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0919] Step 1:

[0920] The server receives source code from the user. The user uploads the source code through a web interface. The server receives the uploaded source code and prepares the data for analysis. Specifically, the server saves the source code as a text file and prepares it for analysis. The input is the text data of the source code provided by the user, and the output is a source code file stored internally on the server.

[0921] Step 2:

[0922] The server analyzes the received source code. This analysis uses natural language processing techniques to represent the source code structure (functions, conditional branches, loops, etc.) as an abstract syntax tree (AST). Specifically, it runs the source code through an AST library to generate an AST object. The input is the source code file saved in step 1, and the output is the parsed abstract syntax tree object.

[0923] Step 3:

[0924] The server generates verification patterns from the parsed abstract syntax tree. It automatically generates verification patterns based on each path of conditional branches and loops. Specifically, it traverses each node of the AST and adds verification scenarios at points where conditional branches or loops are detected. The input is the abstract syntax tree object generated in step 2, and the output is a list of generated verification patterns.

[0925] Step 4:

[0926] The terminal clones or pulls the latest source code from a version control system (e.g., Git). Then, it gets the diff against the previous commit in the repository you are using. Specifically, the terminal runs Git commands to get the latest commit and extract the diff. The input is the remote Git repository, and the output is the diff data between the old and new commits.

[0927] Step 5:

[0928] The terminal transfers the acquired differential data to the server. The server analyzes the received differential data, identifies newly added or modified code parts, and generates a new verification pattern based on them. The input is the differential data acquired in step 4, and the output is the newly generated verification pattern.

[0929] Step 6:

[0930] The user configures automation tools (such as Selenium or Puppeteer) on the server. The server then uses these automation tools to execute test scripts. Specifically, the server starts the automation tools and automatically simulates the operation of web applications or robots based on the configuration. The input is the automation script configured by the user, and the output is the test execution results.

[0931] Step 7:

[0932] The server records the test execution results as a detailed log. This log includes the success or failure of each test case, any error messages that occurred, etc. The server notifies the user of this record. Specifically, the server generates log data and notifies the user via email or the dashboard. The input is the test execution results obtained in step 6, and the output is the generated log data and notification message.

[0933] Step 8:

[0934] In a factory environment, a server analyzes the robot's operation log and sensor input data, and derives the optimal verification pattern from the results. Based on the generated verification pattern, an automatic test of the robot is performed. The input is the robot's operation log and sensor input data, and the output is the results of the automatic test. A specific example includes a test to confirm whether the robot operates accurately based on an operation scenario.

[0935] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0936] This invention relates to an automatic testing system that combines an emotion engine in the verification process of systems and networks. This system efficiently and effectively performs a series of processes that incorporates an emotion engine, from analyzing source code to generating verification patterns, conducting automatic tests, and recording and notifying test results.

[0937] The system includes the following main components:

[0938] Receiving and analyzing the source code

[0939] The server receives source code from users. Users upload the source code through a web interface. The server analyzes the source code using natural language processing techniques. During the analysis process, the structure of the source code (functions, conditional branches, loops, etc.) is represented as an abstract syntax tree (AST).

[0940] Generating validation patterns

[0941] Based on the generated AST, the server traverses each path of the conditional branches and loops in the source code and automatically generates verification patterns, taking into account each scenario and condition in the source code.

[0942] As a concrete example, if there is a conditional branch "if (userAge > 18)" in the source code, the server will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less."

[0943] Differential analysis and test pattern generation

[0944] The terminal clones or pulls the latest source code from a version control system (e.g., Git). The terminal obtains the difference from the previous commit and transfers this difference to the server for analysis. The server analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[0945] As a specific example, if the difference includes "adding a new user login function," the server generates test patterns such as "when the user enters correct login information" and "when the user enters incorrect login information."

[0946] Emotion engine integration

[0947] The server incorporates an emotion engine into its operation. The emotion engine recognizes and monitors the user's emotions in real time while operating the user interface. This emotion data is used to generate verification patterns and conduct tests.

[0948] As a specific example, if a user experiences stress or anxiety when filling out a form, additional validation patterns are generated to verify the impact of specific operations on the user.

[0949] Automated testing

[0950] Users configure automation tools on the server, for example by selecting tools such as Selenium or Puppeteer through a web interface. The server then uses these automation tools to run test scripts, which can include filling out forms or pressing buttons on web applications.

[0951] Recording and notifying test results

[0952] The server records the test execution results in a detailed log, including the success or failure of each test case, any error messages that occurred, etc. This log is then sent to the user via email or a dashboard.

[0953] As explained above, the system of the present invention can achieve more comprehensive and high-quality verification by analyzing source code, generating verification patterns, analyzing differences, conducting automated tests, and notifying the results, as well as taking into account the user's psychological reactions using an emotion engine. This system eliminates the need for engineers to manually consider a huge number of verification patterns, significantly reducing the risk of missing verifications and improving the quality of system releases.

[0954] The processing flow will be explained below.

[0955] Step 1:

[0956] Users provide source code to the server, which can be uploaded through a web interface.

[0957] Step 2:

[0958] The server analyzes the received source code using natural language processing techniques to represent the code structure (functions, conditional branches, loops, etc.) as an abstract syntax tree (AST).

[0959] Step 3:

[0960] Based on the generated AST, the server traverses each path of the conditional branches and loops in the source code and automatically generates verification patterns that cover a wide range of scenarios and conditions.

[0961] Step 4:

[0962] The device clones or pulls the latest source code from the version control system (e.g. Git), which gives you the most up-to-date code base.

[0963] Step 5:

[0964] The terminal obtains the difference between the latest commit and the previous commit, which identifies what has been added or changed.

[0965] Step 6:

[0966] The device transfers the retrieved differences to the server, allowing the server to see the latest code changes.

[0967] Step 7:

[0968] The server analyzes the received differences, identifying any newly added or modified code sections.

[0969] Step 8:

[0970] Based on the analyzed differences, the server identifies existing verification patterns that may be affected and generates new test patterns.

[0971] Step 9:

[0972] Users configure the server with the automation tools they want to use, for example, by selecting tools such as Selenium or Puppeteer through a web interface.

[0973] Step 10:

[0974] The server runs the test scripts using configured automation tools, which may include filling out forms or pressing buttons on the web application.

[0975] Step 11:

[0976] The server activates an emotion engine to recognize the user's emotions in real time. The emotion engine monitors the user's facial expressions and tone of voice while operating the user interface.

[0977] Step 12:

[0978] The server analyzes the recognized emotion data and generates additional verification patterns based on that data. By providing special verification for operations that cause stress or confusion to the user, the user experience is improved.

[0979] Step 13:

[0980] The server records detailed logs of the test execution results, including the success or failure of each test case, error messages, and the user's emotional state.

[0981] Step 14:

[0982] The server notifies the user of the recorded results via email and a dashboard, where the user can view the test results in detail.

[0983] Through these steps, this system effectively and efficiently verifies source code, and by taking into account user feelings in particular, it supports the development of higher quality, user-friendly software.

[0984] Example 2

[0985] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0986] Conventional automated testing systems focus on recognizing and analyzing source code, but no systems have taken into account user emotions and psychological responses. As a result, improvements to the user experience and the generation of emotion-based verification patterns have not been fully realized. Furthermore, the generation of verification patterns based on differential analysis of source code changes and additions has sometimes been inefficient. A solution to these problems was needed.

[0987] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving and analyzing source code, means for generating a verification pattern from the analyzed source code, means for performing an automated test based on the generated verification pattern, means for recognizing a user's emotions in real time while operating a user interface and generating a verification pattern based on the emotions, means for recording and notifying the test results, and means for acquiring differences in the source code and generating a new verification pattern based on the differences. This enables high-quality automated testing that takes into account the user's emotions and psychological reactions, and also realizes efficient generation of verification patterns for modified source code.

[0988] "Source code" is the set of instructions or statements that make up a program or application, usually written as a text file.

[0989] "Analysis" is the operation of breaking down given information or data into detail and clarifying its structure and relationships.

[0990] A "verification pattern" refers to a series of test cases or scenarios that should be verified by automated testing based on source code conditions or scenarios.

[0991] "Automated testing" is the process of verifying the functionality and performance of software or applications using automated tools and scripts.

[0992] "Test Results" refers to the information and data obtained after an automated test is performed, including success / failure status and error messages.

[0993] "Notifying" is the action or process of making certain information or results known to others.

[0994] "User interface" refers to the interaction means and screen layout that allow a user to interact with a computer system or application.

[0995] "Emotion" refers to the psychological state or reaction that a user feels during operation, and includes states such as joy, sadness, and surprise.

[0996] "Natural language processing technology" is a technology that allows computers to understand human language and perform analysis and generation.

[0997] "Delta" refers to changes or differences between different versions of data or code.

[0998] The present invention is a system that efficiently performs everything from analyzing source code to generating verification patterns, conducting automated tests, and recording and notifying test results. It realizes higher quality testing by recognizing user emotions in real time and utilizing them in generating verification patterns. The main components and operating procedures for implementing this invention are described in detail below.

[0999] System Configuration

[1000] This system mainly includes the following hardware and software:

[1001] Server: The central component that analyzes source code, generates verification patterns, manages automated tests, records and notifies test results, etc.

[1002] Terminal: Retrieves source code from a version control system (e.g., Git), analyzes the differences, and sends them to the server.

[1003] Emotion engine: Recognizes the user's emotions in real time while operating the user interface and sends the data to the server.

[1004] The specific software technologies used include:

[1005] Natural language processing technology: Technology for analyzing source code (e.g., Python's ast module).

[1006] Automation tools: Tools for performing automated testing (e.g., Selenium, Puppeteer).

[1007] Receiving and analyzing the source code

[1008] Users upload source code to the server through a web interface. Once the source code is uploaded, the server receives it and analyzes it using natural language processing technology. Specifically, the source code structure (functions, conditional branches, loops, etc.) is represented as an abstract syntax tree (AST) and the analysis results are obtained.

[1009] Generating validation patterns

[1010] Based on the generated AST, the server follows each path of conditional branches and loops in the source code and automatically generates validation patterns. For example, if the condition "if (userAge > 18)" is included, the server will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less."

[1011] Differential analysis and test pattern generation

[1012] The terminal clones or pulls the latest source code from the version control system, obtains the difference from the previous commit, and transfers this difference to the server. The server analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[1013] Emotion engine integration

[1014] The server integrates an emotion engine into the system. The emotion engine recognizes the user's emotions in real time while operating the user interface and sends the data to the server. For example, if the user feels stress or anxiety while filling out a form, the emotion engine detects this and the server generates additional verification patterns based on the emotion data.

[1015] Running automated tests and recording results

[1016] Users configure automation tools using a web interface, for example by selecting a tool such as Selenium or Puppeteer, and saving the automated test configuration. The server then executes test scripts using the selected automation tool, including form entries and button presses in web applications. After the tests are executed, the server records detailed test results in a log and notifies the user via email or a dashboard.

[1017] Examples of concrete examples and prompts

[1018] For example, if the source code contains a conditional branch "if (userAge > 18)", the server will generate two validation patterns - "if userAge is 19 or greater" and "if userAge is 18 or less".

[1019] Example prompt sentence:

[1020] We have added a new user login feature. Please generate test patterns for this feature that show correct and incorrect login information.

[1021] As described above, this embodiment makes it possible to carry out an automatic test that takes into account the user's emotions, and realizes a system that can obtain high-quality test results.

[1022] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1023] Step 1:

[1024] The user uploads source code to the server through a web interface. The input is a source code file, and the output is the source code saved on the server. Specifically, the user selects the target source code file using a file selection dialog and presses the "Upload" button.

[1025] Step 2:

[1026] After the server receives the uploaded source code, it analyzes it using natural language processing technology. The input is the source code file uploaded by the user, and the output is a parsed abstract syntax tree (AST). Specifically, it uses the Python ast module to analyze the structure of the source code and extract components such as functions, conditional branches, and loops.

[1027] Step 3:

[1028] Based on the generated AST, the server follows each path of conditional branches and loops in the source code and automatically generates verification patterns. The input is the AST, and the output is a list of verification patterns. Specifically, test cases are generated by dividing the scenarios into conditions for conditional branches (e.g., if (userAge > 18)). For example, two patterns are generated: "if userAge is 19 or greater" and "if userAge is 18 or less."

[1029] Step 4:

[1030] The terminal clones or pulls the latest source code from the version control system. The input is the repository URL and authentication information, and the output is the latest source code stored locally. Next, the terminal obtains the difference from the previous commit and sends that difference information to the server. Specifically, it uses Git commands to check the difference and sends the difference file to the server.

[1031] Step 5:

[1032] The server analyzes the received differences and identifies newly added or modified code sections. The input is the difference file, and the output is a list of changed sections. It then generates new test patterns based on these changes. Specifically, if the change includes "adding a new user login function," it generates test patterns that cover various scenarios for the login function (correct information, incorrect information).

[1033] Step 6:

[1034] The server integrates an emotion engine into the system and recognizes the user's emotions in real time while operating the user interface. The input is the user's operation data, and the output is the recognized emotion data. Based on this emotion data, the server generates additional verification patterns. For example, if the user feels stressed while filling out a form, the server detects that emotion and creates additional verification patterns accordingly.

[1035] Step 7:

[1036] The user configures the automation tool using a web interface. The input is the selection information for the tool, such as Selenium or Puppeteer, and the output is the configuration information saved on the server. Specifically, the user selects the automation tool on the configuration screen and presses the "Save" button.

[1037] Step 8:

[1038] The server runs the test script using an automation tool configured by the user. The input is the verification pattern and the automation tool's configuration information, and the output is the test execution results. Specifically, Puppeteer is used to automatically operate the browser and perform form input and button presses in the web application.

[1039] Step 9:

[1040] The server records the test execution results as a detailed log and notifies the user via email or dashboard. The input is the test execution results, and the output is notification information. Specifically, the success or failure of test cases, any error messages that occur, etc. are recorded in detail and notified to the user.

[1041] (Application example 2)

[1042] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1043] Current automated testing systems do not take into account the user's emotions and psychological state, making it difficult to reduce the stress and anxiety of operators, especially in real-world environments such as logistics centers. This leads to increased frustration and errors in system operation, hindering efficient business operations. Furthermore, analysis of test results does not include emotional data, resulting in a lack of information useful for improving the user experience.

[1044] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing source code, means for generating a verification pattern from the analyzed source code, means for recognizing the worker's emotions, means for performing an automatic test based on the generated verification pattern, and means for recording and notifying the test results and emotional data. This makes it possible to recognize the worker's emotions in real time and generate more realistic and effective verification patterns based on that data. Furthermore, by performing post-processing analysis based on the emotional data, it is possible to improve the user experience and reduce operational stress.

[1045] "Source code" is a set of textual instructions that make up a program or application.

[1046] "Analysis" is the process of understanding specific data or information and clarifying its meaning and structure.

[1047] A "verification pattern" is a specific condition or scenario designed to test the behavior of source code or a system.

[1048] "Automated testing" is the process of executing test cases using automated tools and checking the results.

[1049] "Means for recognizing emotions" refers to technologies and devices that can identify a worker's psychological state and emotions in real time.

[1050] "Test results" refers to information such as the success or failure of the tests performed and error messages.

[1051] "Emotional data" is information about the psychological reactions and emotions of workers measured in real time.

[1052] "Means for recording and notifying" refers to technology for storing test results and emotional data and notifying them to users and other systems.

[1053] "Natural language processing technology" is a technology in the field of computer science that analyzes human language (natural language) and understands its meaning.

[1054] "Difference" refers to changes or differences between versions of source code, etc.

[1055] "Real-time" means that processing occurs as fast as real time.

[1056] This invention relates to a system that efficiently and effectively performs a series of processes: analyzing source code, generating verification patterns, conducting automated testing based on those patterns, and recognizing, recording, and notifying workers' emotional data in real time.

[1057] Receiving and analyzing the source code

[1058] The server has the function of receiving source code from users through a web interface. The source code is analyzed by the server using natural language processing technology and is expressed as an abstract syntax tree (AST) in the process.

[1059] Generating validation patterns

[1060] Based on the generated abstract syntax tree (AST), the server follows each path of the conditional branches and loops in the source code and automatically generates validation patterns. This takes into account each scenario and condition in the source code. For example, if the source code contains a conditional branch "if (userAge > 18)", the server will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less".

[1061] Differential analysis and test pattern generation

[1062] The terminal clones or pulls the latest source code from the version control system and obtains the difference from the previous commit. This difference is then transferred to the server, which analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[1063] Emotion engine integration

[1064] The server connects to the device used by the worker and integrates the emotion engine into the operation. The emotion engine recognizes and monitors the worker's emotional data in real time using devices such as smart glasses or head-mounted displays. This emotional data is used to generate verification patterns and conduct tests. For example, if a worker feels stressed or anxious while filling out a form, additional verification patterns are generated accordingly to verify how a specific operation affects the worker.

[1065] Automated testing

[1066] Users configure the server with automation tools, such as Selenium, through a web interface. The server then uses these automation tools to execute test scripts, which can include filling out forms and pressing buttons on web applications.

[1067] Recording and notifying test results

[1068] The server records test execution results and emotional data in a detailed log, including the success or failure of each test case, any error messages that occurred, and the emotional data of the tester. This record is then notified to the user via email or a dashboard.

[1069] Specific examples

[1070] For example, consider the case where a new product registration function is added to the web interface of a distribution center. The source code may contain the following new functions:

[1071] Input source code:

[1072] javascript

[1073] function addNewProduct(product) { if (user.isAuthenticated) { / Add product logic / } else { / Show error / }}

[1074] Example prompt sentence:

[1075] "I want to test whether the new product addition function works properly. I also want to make sure that the operation is not stressful for the workers."

[1076] The present invention makes it possible to improve the overall user experience, including the impact that system operation has on workers at a logistics center.

[1077] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1078] Step 1:

[1079] Receive source code from the user.

[1080] Users upload source code to the server through a web interface, and the server receives the source code and stores it in storage, providing the source code data to be analyzed as input.

[1081] Step 2:

[1082] It analyzes the source code and generates an abstract syntax tree (AST).

[1083] The server analyzes the stored source code using natural language processing technology (e.g., SpaCy). During this analysis, the structure of the source code is understood and represented as an abstract syntax tree (AST). The input is the source code, and the output is the analyzed AST.

[1084] Step 3:

[1085] Generates verification patterns based on an abstract syntax tree (AST).

[1086] The server detects conditional branches and loops in the source code based on the generated AST and generates validation patterns. For example, if there is a conditional branch "if (userAge > 18)", it will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less". The input is the AST, and the output is multiple validation patterns.

[1087] Step 4:

[1088] Recognize worker emotions in real time using an emotion engine.

[1089] Using smart glasses or a head-mounted display connected to a terminal, the worker's emotions are recognized in real time and data is collected. An emotion engine (e.g., OpenFace) analyzes the input from this device and obtains the worker's emotional state (e.g., stress, anxiety, satisfaction) as output.

[1090] Step 5:

[1091] Conduct automated testing and leverage sentiment data.

[1092] The server executes the test script based on the verification pattern generated using an automation tool (e.g., Selenium) selected by the user. During the test, the device acquires emotion data using an emotion engine and sends it to the server. The input is the verification pattern and emotion data, and the test result and emotion data are obtained as output.

[1093] Step 6:

[1094] Test results and emotional data will be recorded and communicated.

[1095] The server records test results and emotional data as detailed logs (e.g., Elasticsearch). The recorded data is used for analysis and notifications. The server notifies the user of the results via email or a dashboard, which outputs information about test success or failure and the emotional state of the worker, which the user can review.

[1096] Each step involves the following specific actions:

[1097] Step 1: Upload source code, receive and save data.

[1098] Step 2: Code analysis and AST generation using natural language processing techniques.

[1099] Step 3: Detect conditional branches and loops and generate verification patterns.

[1100] Step 4: Collecting emotion data using smart devices and analyzing it in real time.

[1101] Step 5: Configure and run the automated testing tool and integrate the sentiment data.

[1102] Step 6: Logging the results, saving them to a database, and implementing notifications.

[1103] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1104] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1105] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1106] [Fourth embodiment]

[1107] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1108] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1109] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1110] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1111] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1112] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1113] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1114] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1115] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1116] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1117] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1118] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1119] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1120] The present invention relates to a technique for automatically generating verification patterns and efficiently performing automatic testing in the process of verifying a system or network.

[1121] The system contains several key components, each performing a specific function. These components and their functions are described below.

[1122] Receiving and analyzing the source code

[1123] The server receives source code from the user. For example, the user uploads the source code through a web interface. The server analyzes the source code using natural language processing techniques. During the analysis, the structure of the source code (e.g., functions, conditional branches, loops, etc.) is represented as an abstract syntax tree (AST).

[1124] Generating validation patterns

[1125] The server automatically generates verification patterns based on each path of conditional branches and loops from the analyzed source code. These verification patterns cover a wide range of scenarios and conditions, preventing oversights in manual verification.

[1126] As a concrete example, if there is a conditional branch "if (userAge > 18)" in the source code, the server will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less."

[1127] Differential analysis and test pattern generation

[1128] The terminal clones or pulls the latest source code from a version control system (e.g., Git). The terminal then obtains the difference from the previous commit and transfers this difference to the server for analysis. The server analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[1129] As a specific example, if the difference includes "adding a new user login function," the server generates test patterns such as "when the user enters correct login information" and "when the user enters incorrect login information."

[1130] GUI-based verification and automated testing

[1131] Users configure automation tools (such as Selenium or Puppeteer) on the server, which then uses these automation tools to run test scripts, which can include filling out forms or pressing buttons on web applications.

[1132] As a concrete example, when a server automatically fills in a user registration form for a web application, it enters values ​​for each field (e.g., "user name," "email address," "password," etc.), clicks the submit button, and records the resulting message (success message or error message) that is displayed.

[1133] Recording and notifying test results

[1134] The server records the test execution results in a detailed log, including the success or failure of each test case and any error messages that occurred. This log is then notified to the user via email and a dashboard, where the user can view the results in detail.

[1135] As explained above, the system of the present invention can efficiently and effectively carry out a series of processes, from analyzing source code to generating verification patterns, analyzing differences, and conducting automated tests and notifying the results. This system eliminates the need for engineers to manually consider a huge number of verification patterns, significantly reducing the risk of missing verifications and improving the quality of system releases.

[1136] The processing flow will be explained below.

[1137] Step 1:

[1138] Users provide source code to the server, which can be uploaded through a web interface.

[1139] Step 2:

[1140] The server analyzes the received source code using natural language processing techniques to represent the code structure (functions, conditional branches, loops, etc.) as an abstract syntax tree (AST).

[1141] Step 3:

[1142] Based on the generated AST, the server traverses each path of the conditional branches and loops in the source code and automatically generates verification patterns, taking into account each scenario and condition in the source code.

[1143] Step 4:

[1144] The device clones or pulls the latest source code from the version control system (e.g. Git), which gives you the most up-to-date code base.

[1145] Step 5:

[1146] The terminal obtains the difference between the latest commit and the previous commit to identify what has been newly added or changed.

[1147] Step 6:

[1148] The device transfers the retrieved differences to the server, allowing the server to see the latest code changes.

[1149] Step 7:

[1150] The server analyzes the received differences, identifying any newly added or modified code sections.

[1151] Step 8:

[1152] Based on the analyzed differences, the server identifies existing verification patterns that may be affected and generates new test patterns.

[1153] Step 9:

[1154] Users configure the automation tools they want to use on their servers, for example by selecting tools such as Selenium or Puppeteer through a web interface.

[1155] Step 10:

[1156] The server runs the test scripts using configured automation tools, which may include filling out forms or pressing buttons on the web application.

[1157] Step 11:

[1158] The server records detailed test execution results in a log, including the success or failure of each test case and any error messages.

[1159] Step 12:

[1160] The server notifies the user of the recorded results via email and a dashboard, where the user can view the test results in detail.

[1161] Example 1

[1162] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1163] In today's complex software development, manual verification and testing not only requires a great deal of time and effort, but also carries a high risk of human error and missed verifications. Furthermore, because software is frequently changed due to program modifications and the implementation of additional features, a system is required to efficiently verify only the changed parts. However, it has been difficult to meet these requirements with conventional manual testing methods.

[1164] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1165] In this invention, the server includes means for receiving and analyzing software, means for generating a verification pattern from the analyzed software, means for performing an automated test based on the generated verification pattern, means for recording and notifying the test results, means for acquiring software differences from a version control system and generating a new verification pattern based on the differences, and means for uploading settings via a user interface and executing tests using an automated tool. This makes it possible to efficiently verify software changes, significantly reduce manual verification work, and reduce the risk of missing verification points.

[1166] "Software" is a set of instructions or programs designed to cause a computer to perform a specific function.

[1167] "Means of analysis" are methods or devices that analyze and interpret the contents of software in detail in order to understand its internal structure and operating logic.

[1168] A "verification pattern" is a test scenario based on specific situations or conditions designed to verify that software behaves as expected.

[1169] "Means for performing automated testing" are programs and tools that can be used to verify that software functions properly without human intervention.

[1170] "Means for recording and notifying test results" refers to a method or device for storing the results of an automated test and communicating that information to the user.

[1171] A "version control system" is a system for managing software change history and tracking and integrating different versions.

[1172] "Software differences" refer to the parts that have been changed or newly added compared to the previous version.

[1173] "User interface" is the collective term for the means and methods by which a user interacts with software or a system.

[1174] An "automation tool" is software or a program designed to perform routine tasks or processes automatically.

[1175] A "means of configuration upload" is a method by which a user enters or transmits specific parameters or configuration information into the system.

[1176] The present invention relates to a technology for automatically generating verification patterns and efficiently performing automated testing in the verification process of a system or network. This system includes important components, each of which performs a specific function.

[1177] Receiving and analyzing the source code

[1178] The server receives the software. Users upload the software through a web interface. The server analyzes the software using natural language processing techniques. During the analysis, the software structure (e.g., functions, conditionals, loops, etc.) is represented as an abstract syntax tree (AST).

[1179] As a concrete example, a user opens a browser, clicks the "Choose File" button in the interface of a web application, selects the file to upload, and clicks the "Submit" button.

[1180] Generating validation patterns

[1181] The server automatically generates verification patterns based on each path of conditional branches and loops from the analyzed software, allowing it to cover a wide range of scenarios and conditions.

[1182] For example, if the source code has a conditional branch if (userAge > 18), two validation patterns will be generated - "if userAge is greater than or equal to 19" and "if userAge is less than or equal to 18".

[1183] Differential analysis and test pattern generation

[1184] The terminal clones or pulls the latest software from a version control system (e.g., Git). The terminal then obtains the differences from the previous commit and transfers these differences to the server. The server analyzes the received differences, identifies newly added or modified code sections, and generates test patterns based on them.

[1185] As a specific example, if the difference includes "adding a new user login function," test patterns for "when the user enters correct login information" and "when the user enters incorrect login information" will be generated.

[1186] GUI-based verification and automated testing

[1187] Users configure automation tools (such as Selenium or Puppeteer) on the server, which then uses these automation tools to run test scripts, which can include filling out forms or pressing buttons on a web application.

[1188] As a concrete example, when a server automatically fills in a user registration form for a web application, it enters values ​​for each field (e.g., "user name," "email address," "password," etc.), clicks the submit button, and records the resulting message (success message or error message).

[1189] Recording and notifying test results

[1190] The server records the test execution results in a detailed log, including the success or failure of each test case and any error messages that occurred. This log is then notified to the user via email and a dashboard, where the user can view the results in detail.

[1191] For example, if the user has email notifications enabled, they will receive an email with the test result details attached.

[1192] Examples of prompt statements

[1193] Below are some example prompts for input to generative AI models for validation and automated testing systems:

[1194] In this system, users upload software, and the server analyzes the software and automatically generates verification and test patterns. It also retrieves differences from the version control system and generates test cases based on the added or modified parts. It runs tests using automated tools, records the results as detailed logs, and notifies the user. A specific example is the automatic filling of user registration forms and recording the submission results.

[1195] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1196] Step 1:

[1197] The user uploads the software through a web interface: the user opens a browser, clicks the "Choose File" button in the web application interface, selects the file to upload, and clicks the "Submit" button.

[1198] Input: User uploaded software file

[1199] Output: Software files sent to the server

[1200] Step 2:

[1201] The server receives the uploaded software and analyzes it using natural language processing techniques. Specifically, it converts the software into an abstract syntax tree (AST) using Python's ast module, and then uses this AST to analyze the code structure (e.g., function definitions, conditional branches, loops, etc.).

[1202] Input: Uploaded software file

[1203] Output: Software structure as an Abstract Syntax Tree (AST)

[1204] Step 3:

[1205] The server automatically generates verification patterns from the analyzed software. Specifically, it generates test cases that cover all possible execution paths based on conditional branching and loop structures. For example, if the condition is "if (userAge > 18)," it generates both scenarios: "when userAge is 19 or greater" and "when userAge is 18 or less."

[1206] Input: Abstract Syntax Tree (AST)

[1207] Output: Validation pattern

[1208] Step 4:

[1209] The terminal clones or pulls the latest software from a version control system (e.g., Git). Specifically, the terminal executes the git clone command or git pull command to obtain the latest code locally.

[1210] Input: Version control system repository information

[1211] Output: A local copy of the latest software code

[1212] Step 5:

[1213] The terminal obtains the difference from the previous commit and transfers this difference to the server. Specifically, it uses the git diff command to obtain the difference from the previous commit and transfers that data to the server.

[1214] Input: Locally cloned or pulled software code

[1215] Output: Differential data

[1216] Step 6:

[1217] The server analyzes the received differences to identify newly added or modified code parts. Specifically, it analyzes the difference data to identify new functions, modified conditional branches, etc.

[1218] Input: differential data

[1219] Output: Identified new or modified code sections

[1220] Step 7:

[1221] The server generates test patterns based on the identified code parts. Specifically, if the difference includes "adding a new user login function," test cases will be generated for "what happens when the user enters correct login information" and "what happens when the user enters incorrect login information."

[1222] Input: New or modified code fragment

[1223] Output: Test pattern

[1224] Step 8:

[1225] The user configures the automation tool (such as Selenium or Puppeteer) on the server. Specifically, the user uploads the automation tool's configuration file (e.g., config.json) from the web interface and enters the required settings (browser type, test script path, etc.).

[1226] Input: User configuration files and settings information

[1227] Output: Settings saved on the server

[1228] Step 9:

[1229] The server runs the test script using an automation tool, which includes filling out forms and pressing buttons in the web application. Specifically, the server uses the configured automation tool (e.g., Selenium WebDriver) to launch a browser, open a web application page, and run the test.

[1230] Input: Configured automation tools and test scripts

[1231] Output: Test execution results

[1232] Step 10:

[1233] The server records the test execution results as a detailed log. This log includes the success or failure of each test case, any error messages that occurred, etc. Specifically, the execution results of each test case (success, failure, error messages, etc.) are saved in a text file or database.

[1234] Input: Test execution results

[1235] Output: Detailed log data

[1236] Step 11:

[1237] The server notifies the user of the test results by sending the generated log file as an email attachment and / or displaying the results in a web interface dashboard.

[1238] Input: Detailed log data

[1239] Output: Notification to user (email, dashboard display, etc.)

[1240] (Application example 1)

[1241] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1242] Currently, there is a lack of efficient and highly accurate means for verifying the performance of robotic motion sequences and tasks in factory environments. This can lead to unexpected malfunctions, resulting in reduced operational efficiency and safety issues. Furthermore, manual verification is time-consuming and labor-intensive, and carries the risk of missing verification steps. Therefore, a more efficient and automated verification system is needed.

[1243] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1244] In this invention, the server includes means for receiving and analyzing source code, means for generating verification patterns from the analyzed source code, means for performing automated testing based on the generated verification patterns, means for recording and notifying test results, means for analyzing the operation sequences and task performance of a robot in a factory environment and generating verification patterns, and means for performing automated testing of the robot based on the generated verification patterns. This makes it possible to automatically verify the operations and tasks of factory robots, thereby improving operational efficiency and reliability.

[1245] "Source code" is a collection of textual instructions written to direct the operation of a program.

[1246] "Means for analyzing" refers to a method or device for analyzing input data or information and understanding its structure and content.

[1247] A "verification pattern" refers to a combination of various test scenarios and conditions to be performed, which serves as a basis for evaluating functionality and performance.

[1248] "Means for performing automated testing" refers to an automated method or device for performing testing without human intervention.

[1249] "Means for recording and notifying test results" refers to a method or device for storing the results of a test after it has been performed and notifying relevant parties.

[1250] "Factory environment" refers to the physical location where manufacturing or processing takes place and the systems and equipment used therein.

[1251] "Robot movement sequence" refers to the order and timing of movements performed by a robot.

[1252] "Task performance" is an indicator of how effectively a robot performs a specific task.

[1253] The "means for generating verification patterns" refers to a method or device that automatically creates appropriate verification methods or scenarios based on analyzed information.

[1254] "Means for performing automated testing of a robot" refers to a method or device for automatically verifying the operation or functionality of a robot.

[1255] "Natural language processing technology" refers to technology that enables computers to understand, interpret, and generate human language.

[1256] "Difference" is a concept that indicates changes between different versions of data or files.

[1257] A system for realizing the present invention includes the following configuration and processing procedures.

[1258] Receiving and analyzing the source code

[1259] The server receives source code from users. Users upload the source code through a web interface. The server analyzes the source code using natural language processing techniques. During the analysis, the structure of the source code (functions, conditional branches, loops, etc.) is represented as an abstract syntax tree (AST).

[1260] Generating validation patterns

[1261] The server automatically generates verification patterns based on each path of conditional branches and loops from the analyzed source code. These verification patterns cover a wide range of scenarios and conditions, preventing oversights in manual verification.

[1262] Differential analysis and test pattern generation

[1263] The terminal clones or pulls the latest source code from a version control system (e.g., Git). The terminal then obtains the difference from the previous commit and transfers this difference to the server for analysis. The server analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[1264] GUI-based verification and automated testing

[1265] Users configure automation tools (such as Selenium or Puppeteer) on the server, which then uses these automation tools to run test scripts, which can include filling out forms or pressing buttons on web applications.

[1266] Recording and notifying test results

[1267] The server records the test execution results in a detailed log, including the success or failure of each test case and any error messages that occurred. This log is then notified to the user via email and a dashboard, where the user can view the results in detail.

[1268] Additional processes when applied to factory robots

[1269] In a factory environment, a means is added to analyze the robot's operation sequence and task performance and generate verification patterns. The server analyzes the robot's operation log and sensor input data and derives the optimal verification pattern from the results. Then, automatic testing of the robot is performed based on the generated verification pattern.

[1270] Hardware and software used

[1271] The following hardware and software are used to implement this system:

[1272] Hardware: Servers, terminals, robots

[1273] Software: Git (version control system), Selenium (automation tool), Puppeteer (automation tool), natural language processing libraries (e.g., NLTK)

[1274] Examples of concrete examples and prompts

[1275] As a concrete example, when a server automatically fills in a user registration form for a web application, it enters values ​​for each field (e.g., "user name," "email address," "password," etc.), clicks the submit button, and records the resulting message (success message or error message) that is displayed.

[1276] An example of a prompt sentence to be input to the generative AI model is written as follows:

[1277] "Detect conditional branches and loops in the source code of factory robots and generate the necessary verification patterns based on them. Also, run automated tests based on the generated patterns and log the results."

[1278] With the above configuration, the present invention can automatically verify the operations and tasks of factory robots, thereby improving operational efficiency and reliability.

[1279] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1280] Step 1:

[1281] The server receives source code from the user. The user uploads the source code through a web interface. The server receives the uploaded source code and prepares the data for analysis. Specifically, the server saves the source code as a text file and prepares it for analysis. The input is the text data of the source code provided by the user, and the output is a source code file stored internally on the server.

[1282] Step 2:

[1283] The server analyzes the received source code. This analysis uses natural language processing techniques to represent the source code structure (functions, conditional branches, loops, etc.) as an abstract syntax tree (AST). Specifically, it runs the source code through an AST library to generate an AST object. The input is the source code file saved in step 1, and the output is the parsed abstract syntax tree object.

[1284] Step 3:

[1285] The server generates verification patterns from the parsed abstract syntax tree. It automatically generates verification patterns based on each path of conditional branches and loops. Specifically, it traverses each node of the AST and adds verification scenarios at points where conditional branches or loops are detected. The input is the abstract syntax tree object generated in step 2, and the output is a list of generated verification patterns.

[1286] Step 4:

[1287] The terminal clones or pulls the latest source code from a version control system (e.g., Git). Then, it gets the diff against the previous commit in the repository you are using. Specifically, the terminal runs Git commands to get the latest commit and extract the diff. The input is the remote Git repository, and the output is the diff data between the old and new commits.

[1288] Step 5:

[1289] The terminal transfers the acquired differential data to the server. The server analyzes the received differential data, identifies newly added or modified code parts, and generates a new verification pattern based on them. The input is the differential data acquired in step 4, and the output is the newly generated verification pattern.

[1290] Step 6:

[1291] The user configures automation tools (such as Selenium or Puppeteer) on the server. The server then uses these automation tools to execute test scripts. Specifically, the server starts the automation tools and automatically simulates the operation of web applications or robots based on the configuration. The input is the automation script configured by the user, and the output is the test execution results.

[1292] Step 7:

[1293] The server records the test execution results as a detailed log. This log includes the success or failure of each test case, any error messages that occurred, etc. The server notifies the user of this record. Specifically, the server generates log data and notifies the user via email or the dashboard. The input is the test execution results obtained in step 6, and the output is the generated log data and notification message.

[1294] Step 8:

[1295] In a factory environment, a server analyzes the robot's operation log and sensor input data, and derives the optimal verification pattern from the results. Based on the generated verification pattern, an automatic test of the robot is performed. The input is the robot's operation log and sensor input data, and the output is the results of the automatic test. A specific example includes a test to confirm whether the robot operates accurately based on an operation scenario.

[1296] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1297] This invention relates to an automatic testing system that combines an emotion engine in the verification process of systems and networks. This system efficiently and effectively performs a series of processes that incorporates an emotion engine, from analyzing source code to generating verification patterns, conducting automatic tests, and recording and notifying test results.

[1298] The system includes the following main components:

[1299] Receiving and analyzing the source code

[1300] The server receives source code from users. Users upload the source code through a web interface. The server analyzes the source code using natural language processing techniques. During the analysis process, the structure of the source code (functions, conditional branches, loops, etc.) is represented as an abstract syntax tree (AST).

[1301] Generating validation patterns

[1302] Based on the generated AST, the server traverses each path of the conditional branches and loops in the source code and automatically generates verification patterns, taking into account each scenario and condition in the source code.

[1303] As a concrete example, if there is a conditional branch "if (userAge > 18)" in the source code, the server will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less."

[1304] Differential analysis and test pattern generation

[1305] The terminal clones or pulls the latest source code from a version control system (e.g., Git). The terminal obtains the difference from the previous commit and transfers this difference to the server for analysis. The server analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[1306] As a specific example, if the difference includes "adding a new user login function," the server generates test patterns such as "when the user enters correct login information" and "when the user enters incorrect login information."

[1307] Emotion engine integration

[1308] The server incorporates an emotion engine into its operation. The emotion engine recognizes and monitors the user's emotions in real time while operating the user interface. This emotion data is used to generate verification patterns and conduct tests.

[1309] As a specific example, if a user experiences stress or anxiety when filling out a form, additional validation patterns are generated to verify the impact of specific operations on the user.

[1310] Automated testing

[1311] Users configure automation tools on the server, for example by selecting tools such as Selenium or Puppeteer through a web interface. The server then uses these automation tools to run test scripts, which can include filling out forms or pressing buttons on web applications.

[1312] Recording and notifying test results

[1313] The server records the test execution results in a detailed log, including the success or failure of each test case, any error messages that occurred, etc. This log is then sent to the user via email or a dashboard.

[1314] As explained above, the system of the present invention can achieve more comprehensive and high-quality verification by analyzing source code, generating verification patterns, analyzing differences, conducting automated tests, and notifying the results, as well as taking into account the user's psychological reactions using an emotion engine. This system eliminates the need for engineers to manually consider a huge number of verification patterns, significantly reducing the risk of missing verifications and improving the quality of system releases.

[1315] The processing flow will be explained below.

[1316] Step 1:

[1317] Users provide source code to the server, which can be uploaded through a web interface.

[1318] Step 2:

[1319] The server analyzes the received source code using natural language processing techniques to represent the code structure (functions, conditional branches, loops, etc.) as an abstract syntax tree (AST).

[1320] Step 3:

[1321] Based on the generated AST, the server traverses each path of the conditional branches and loops in the source code and automatically generates verification patterns that cover a wide range of scenarios and conditions.

[1322] Step 4:

[1323] The device clones or pulls the latest source code from the version control system (e.g. Git), which gives you the most up-to-date code base.

[1324] Step 5:

[1325] The terminal obtains the difference between the latest commit and the previous commit, which identifies what has been added or changed.

[1326] Step 6:

[1327] The device transfers the retrieved differences to the server, allowing the server to see the latest code changes.

[1328] Step 7:

[1329] The server analyzes the received differences, identifying any newly added or modified code sections.

[1330] Step 8:

[1331] Based on the analyzed differences, the server identifies existing verification patterns that may be affected and generates new test patterns.

[1332] Step 9:

[1333] Users configure the server with the automation tools they want to use, for example, by selecting tools such as Selenium or Puppeteer through a web interface.

[1334] Step 10:

[1335] The server runs the test scripts using configured automation tools, which may include filling out forms or pressing buttons on the web application.

[1336] Step 11:

[1337] The server activates an emotion engine to recognize the user's emotions in real time. The emotion engine monitors the user's facial expressions and tone of voice while operating the user interface.

[1338] Step 12:

[1339] The server analyzes the recognized emotion data and generates additional verification patterns based on that data. By providing special verification for operations that cause stress or confusion to the user, the user experience is improved.

[1340] Step 13:

[1341] The server records detailed logs of the test execution results, including the success or failure of each test case, error messages, and the user's emotional state.

[1342] Step 14:

[1343] The server notifies the user of the recorded results via email and a dashboard, where the user can view the test results in detail.

[1344] Through these steps, this system effectively and efficiently verifies source code, and by taking into account user feelings in particular, it supports the development of higher quality, user-friendly software.

[1345] Example 2

[1346] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1347] Conventional automated testing systems focus on recognizing and analyzing source code, but no systems have taken into account user emotions and psychological responses. As a result, improvements to the user experience and the generation of emotion-based verification patterns have not been fully realized. Furthermore, the generation of verification patterns based on differential analysis of source code changes and additions has sometimes been inefficient. A solution to these problems was needed.

[1348] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving and analyzing source code, means for generating a verification pattern from the analyzed source code, means for performing an automated test based on the generated verification pattern, means for recognizing a user's emotions in real time while operating a user interface and generating a verification pattern based on the emotions, means for recording and notifying the test results, and means for acquiring differences in the source code and generating a new verification pattern based on the differences. This enables high-quality automated testing that takes into account the user's emotions and psychological reactions, and also realizes efficient generation of verification patterns for modified source code.

[1349] "Source code" is the set of instructions or statements that make up a program or application, usually written as a text file.

[1350] "Analysis" is the operation of breaking down given information or data into detail and clarifying its structure and relationships.

[1351] A "verification pattern" refers to a series of test cases or scenarios that should be verified by automated testing based on source code conditions or scenarios.

[1352] "Automated testing" is the process of verifying the functionality and performance of software or applications using automated tools and scripts.

[1353] "Test Results" refers to the information and data obtained after an automated test is performed, including success / failure status and error messages.

[1354] "Notifying" is the action or process of making certain information or results known to others.

[1355] "User interface" refers to the interaction means and screen layout that allow a user to interact with a computer system or application.

[1356] "Emotion" refers to the psychological state or reaction that a user feels during operation, and includes states such as joy, sadness, and surprise.

[1357] "Natural language processing technology" is a technology that allows computers to understand human language and perform analysis and generation.

[1358] "Delta" refers to changes or differences between different versions of data or code.

[1359] The present invention is a system that efficiently performs everything from analyzing source code to generating verification patterns, conducting automated tests, and recording and notifying test results. It realizes higher quality testing by recognizing user emotions in real time and utilizing them in generating verification patterns. The main components and operating procedures for implementing this invention are described in detail below.

[1360] System Configuration

[1361] This system mainly includes the following hardware and software:

[1362] Server: The central component that analyzes source code, generates verification patterns, manages automated tests, records and notifies test results, etc.

[1363] Terminal: Retrieves source code from a version control system (e.g., Git), analyzes the differences, and sends them to the server.

[1364] Emotion engine: Recognizes the user's emotions in real time while operating the user interface and sends the data to the server.

[1365] The specific software technologies used include:

[1366] Natural language processing technology: Technology for analyzing source code (e.g., Python's ast module).

[1367] Automation tools: Tools for performing automated testing (e.g., Selenium, Puppeteer).

[1368] Receiving and analyzing the source code

[1369] Users upload source code to the server through a web interface. Once the source code is uploaded, the server receives it and analyzes it using natural language processing technology. Specifically, the source code structure (functions, conditional branches, loops, etc.) is represented as an abstract syntax tree (AST) and the analysis results are obtained.

[1370] Generating validation patterns

[1371] Based on the generated AST, the server follows each path of conditional branches and loops in the source code and automatically generates validation patterns. For example, if the condition "if (userAge > 18)" is included, the server will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less."

[1372] Differential analysis and test pattern generation

[1373] The terminal clones or pulls the latest source code from the version control system, obtains the difference from the previous commit, and transfers this difference to the server. The server analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[1374] Emotion engine integration

[1375] The server integrates an emotion engine into the system. The emotion engine recognizes the user's emotions in real time while operating the user interface and sends the data to the server. For example, if the user feels stress or anxiety while filling out a form, the emotion engine detects this and the server generates additional verification patterns based on the emotion data.

[1376] Running automated tests and recording results

[1377] Users configure automation tools using a web interface, for example by selecting a tool such as Selenium or Puppeteer, and saving the automated test configuration. The server then executes test scripts using the selected automation tool, including form entries and button presses in web applications. After the tests are executed, the server records detailed test results in a log and notifies the user via email or a dashboard.

[1378] Examples of concrete examples and prompts

[1379] For example, if the source code contains a conditional branch "if (userAge > 18)", the server will generate two validation patterns - "if userAge is 19 or greater" and "if userAge is 18 or less".

[1380] Example prompt sentence:

[1381] We have added a new user login feature. Please generate test patterns for this feature that show correct and incorrect login information.

[1382] As described above, this embodiment makes it possible to carry out an automatic test that takes into account the user's emotions, and realizes a system that can obtain high-quality test results.

[1383] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1384] Step 1:

[1385] The user uploads source code to the server through a web interface. The input is a source code file, and the output is the source code saved on the server. Specifically, the user selects the target source code file using a file selection dialog and presses the "Upload" button.

[1386] Step 2:

[1387] After the server receives the uploaded source code, it analyzes it using natural language processing technology. The input is the source code file uploaded by the user, and the output is a parsed abstract syntax tree (AST). Specifically, it uses the Python ast module to analyze the structure of the source code and extract components such as functions, conditional branches, and loops.

[1388] Step 3:

[1389] Based on the generated AST, the server follows each path of conditional branches and loops in the source code and automatically generates verification patterns. The input is the AST, and the output is a list of verification patterns. Specifically, test cases are generated by dividing the scenarios into conditions for conditional branches (e.g., if (userAge > 18)). For example, two patterns are generated: "if userAge is 19 or greater" and "if userAge is 18 or less."

[1390] Step 4:

[1391] The terminal clones or pulls the latest source code from the version control system. The input is the repository URL and authentication information, and the output is the latest source code stored locally. Next, the terminal obtains the difference from the previous commit and sends that difference information to the server. Specifically, it uses Git commands to check the difference and sends the difference file to the server.

[1392] Step 5:

[1393] The server analyzes the received differences and identifies newly added or modified code sections. The input is the difference file, and the output is a list of changed sections. It then generates new test patterns based on these changes. Specifically, if the change includes "adding a new user login function," it generates test patterns that cover various scenarios for the login function (correct information, incorrect information).

[1394] Step 6:

[1395] The server integrates an emotion engine into the system and recognizes the user's emotions in real time while operating the user interface. The input is the user's operation data, and the output is the recognized emotion data. Based on this emotion data, the server generates additional verification patterns. For example, if the user feels stressed while filling out a form, the server detects that emotion and creates additional verification patterns accordingly.

[1396] Step 7:

[1397] The user configures the automation tool using a web interface. The input is the selection information for the tool, such as Selenium or Puppeteer, and the output is the configuration information saved on the server. Specifically, the user selects the automation tool on the configuration screen and presses the "Save" button.

[1398] Step 8:

[1399] The server runs the test script using an automation tool configured by the user. The input is the verification pattern and the automation tool's configuration information, and the output is the test execution results. Specifically, Puppeteer is used to automatically operate the browser and perform form input and button presses in the web application.

[1400] Step 9:

[1401] The server records the test execution results as a detailed log and notifies the user via email or dashboard. The input is the test execution results, and the output is notification information. Specifically, the success or failure of test cases, any error messages that occur, etc. are recorded in detail and notified to the user.

[1402] (Application example 2)

[1403] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1404] Current automated testing systems do not take into account the user's emotions and psychological state, making it difficult to reduce the stress and anxiety of operators, especially in real-world environments such as logistics centers. This leads to increased frustration and errors in system operation, hindering efficient business operations. Furthermore, analysis of test results does not include emotional data, resulting in a lack of information useful for improving the user experience.

[1405] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing source code, means for generating a verification pattern from the analyzed source code, means for recognizing the worker's emotions, means for performing an automatic test based on the generated verification pattern, and means for recording and notifying the test results and emotional data. This makes it possible to recognize the worker's emotions in real time and generate more realistic and effective verification patterns based on that data. Furthermore, by performing post-processing analysis based on the emotional data, it is possible to improve the user experience and reduce operational stress.

[1406] "Source code" is a set of textual instructions that make up a program or application.

[1407] "Analysis" is the process of understanding specific data or information and clarifying its meaning and structure.

[1408] A "verification pattern" is a specific condition or scenario designed to test the behavior of source code or a system.

[1409] "Automated testing" is the process of executing test cases using automated tools and checking the results.

[1410] "Means for recognizing emotions" refers to technologies and devices that can identify a worker's psychological state and emotions in real time.

[1411] "Test results" refers to information such as the success or failure of the tests performed and error messages.

[1412] "Emotional data" is information about the psychological reactions and emotions of workers measured in real time.

[1413] "Means for recording and notifying" refers to technology for storing test results and emotional data and notifying them to users and other systems.

[1414] "Natural language processing technology" is a technology in the field of computer science that analyzes human language (natural language) and understands its meaning.

[1415] "Difference" refers to changes or differences between versions of source code, etc.

[1416] "Real-time" means that processing occurs as fast as real time.

[1417] This invention relates to a system that efficiently and effectively performs a series of processes: analyzing source code, generating verification patterns, conducting automated testing based on those patterns, and recognizing, recording, and notifying workers' emotional data in real time.

[1418] Receiving and analyzing the source code

[1419] The server has the function of receiving source code from users through a web interface. The source code is analyzed by the server using natural language processing technology and is expressed as an abstract syntax tree (AST) in the process.

[1420] Generating validation patterns

[1421] Based on the generated abstract syntax tree (AST), the server follows each path of the conditional branches and loops in the source code and automatically generates validation patterns. This takes into account each scenario and condition in the source code. For example, if the source code contains a conditional branch "if (userAge > 18)", the server will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less".

[1422] Differential analysis and test pattern generation

[1423] The terminal clones or pulls the latest source code from the version control system and obtains the difference from the previous commit. This difference is then transferred to the server, which analyzes the received difference, identifies newly added or modified code parts, and generates test patterns based on them.

[1424] Emotion engine integration

[1425] The server connects to the device used by the worker and integrates the emotion engine into the operation. The emotion engine recognizes and monitors the worker's emotional data in real time using devices such as smart glasses or head-mounted displays. This emotional data is used to generate verification patterns and conduct tests. For example, if a worker feels stressed or anxious while filling out a form, additional verification patterns are generated accordingly to verify how a specific operation affects the worker.

[1426] Automated testing

[1427] Users configure the server with automation tools, such as Selenium, through a web interface. The server then uses these automation tools to execute test scripts, which can include filling out forms and pressing buttons on web applications.

[1428] Recording and notifying test results

[1429] The server records test execution results and emotional data in a detailed log, including the success or failure of each test case, any error messages that occurred, and the emotional data of the tester. This record is then notified to the user via email or a dashboard.

[1430] Specific examples

[1431] For example, consider the case where a new product registration function is added to the web interface of a distribution center. The source code may contain the following new functions:

[1432] Input source code:

[1433] javascript

[1434] function addNewProduct(product) { if (user.isAuthenticated) { / Add product logic / } else { / Show error / }}

[1435] Example prompt sentence:

[1436] "I want to test whether the new product addition function works properly. I also want to make sure that the operation is not stressful for the workers."

[1437] The present invention makes it possible to improve the overall user experience, including the impact that system operation has on workers at a logistics center.

[1438] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1439] Step 1:

[1440] Receive source code from the user.

[1441] Users upload source code to the server through a web interface, and the server receives the source code and stores it in storage, providing the source code data to be analyzed as input.

[1442] Step 2:

[1443] It analyzes the source code and generates an abstract syntax tree (AST).

[1444] The server analyzes the stored source code using natural language processing technology (e.g., SpaCy). During this analysis, the structure of the source code is understood and represented as an abstract syntax tree (AST). The input is the source code, and the output is the analyzed AST.

[1445] Step 3:

[1446] Generates verification patterns based on an abstract syntax tree (AST).

[1447] The server detects conditional branches and loops in the source code based on the generated AST and generates validation patterns. For example, if there is a conditional branch "if (userAge > 18)", it will generate two validation patterns: "if userAge is 19 or greater" and "if userAge is 18 or less". The input is the AST, and the output is multiple validation patterns.

[1448] Step 4:

[1449] Recognize worker emotions in real time using an emotion engine.

[1450] Using smart glasses or a head-mounted display connected to a terminal, the worker's emotions are recognized in real time and data is collected. An emotion engine (e.g., OpenFace) analyzes the input from this device and obtains the worker's emotional state (e.g., stress, anxiety, satisfaction) as output.

[1451] Step 5:

[1452] Conduct automated testing and leverage sentiment data.

[1453] The server executes the test script based on the verification pattern generated using an automation tool (e.g., Selenium) selected by the user. During the test, the device acquires emotion data using an emotion engine and sends it to the server. The input is the verification pattern and emotion data, and the test result and emotion data are obtained as output.

[1454] Step 6:

[1455] Test results and emotional data will be recorded and communicated.

[1456] The server records test results and emotional data as detailed logs (e.g., Elasticsearch). The recorded data is used for analysis and notifications. The server notifies the user of the results via email or a dashboard, which outputs information about test success or failure and the emotional state of the worker, which the user can review.

[1457] Each step involves the following specific actions:

[1458] Step 1: Upload source code, receive and save data.

[1459] Step 2: Code analysis and AST generation using natural language processing techniques.

[1460] Step 3: Detect conditional branches and loops and generate verification patterns.

[1461] Step 4: Collecting emotion data using smart devices and analyzing it in real time.

[1462] Step 5: Configure and run the automated testing tool and integrate the sentiment data.

[1463] Step 6: Logging the results, saving them to a database, and implementing notifications.

[1464] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1465] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1466] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1467] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1468] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1469] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1470] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1471] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1472] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1473] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1474] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1475] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1476] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1477] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1478] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1479] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1480] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1481] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1482] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1483] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1484] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1485] The following is further disclosed regarding the above embodiment.

[1486] (Claim 1)

[1487] means for receiving and analyzing the source code;

[1488] means for generating a verification pattern from the analyzed source code;

[1489] A means for performing an automated test based on the generated verification pattern;

[1490] A system that includes a means for recording and communicating test results.

[1491] (Claim 2)

[1492] 2. The system according to claim 1, wherein the analysis of the source code includes using a natural language processing technique.

[1493] (Claim 3)

[1494] 2. The system according to claim 1, further comprising means for obtaining a difference in the source code and generating a new verification pattern based on the difference.

[1495] (Claim 4)

[1496] 4. The system according to claim 3, wherein said difference acquisition includes means for using a version control system.

[1497] (Claim 5)

[1498] 10. The system of claim 1, further comprising means for implementing automated operation of the user interface.

[1499] (Claim 6)

[1500] 6. The system of claim 5, wherein the automated operation includes using an automated tool.

[1501] (Claim 7)

[1502] 2. The system according to claim 1, further comprising: means for recording the test results as a log; and means for notifying a user of the log.

[1503] (Claim 8)

[1504] 2. The system according to claim 1, wherein the verification pattern generating means includes means for generating a plurality of verification patterns based on a conditional branch or a loop structure.

[1505] "Example 1"

[1506] (Claim 1)

[1507] a means for receiving and analyzing the software;

[1508] means for generating a verification pattern from the analyzed software;

[1509] means for performing an automatic test based on the generated verification pattern;

[1510] a means for recording and communicating test results;

[1511] A means for acquiring software differences from a version control system and generating a new verification pattern based on the differences;

[1512] a means for uploading configurations through a user interface and running tests using automation tools;

[1513] A system including:

[1514] (Claim 2)

[1515] 10. The system of claim 1, wherein the analysis of the software includes using natural language processing techniques.

[1516] (Claim 3)

[1517] 2. The system according to claim 1, further comprising means for analyzing the differences in the software and generating test patterns based on the modified portions.

[1518] "Application Example 1"

[1519] (Claim 1)

[1520] means for receiving and analyzing the source code;

[1521] means for generating a verification pattern from the analyzed source code;

[1522] A means for performing an automated test based on the generated verification pattern;

[1523] a means of recording and communicating test results;

[1524] a means for analyzing the motion sequences and task performance of the robot in a factory environment and generating verification patterns;

[1525] means for performing an automatic test of the robot based on the generated verification pattern;

[1526] A system including:

[1527] (Claim 2)

[1528] 2. The system according to claim 1, wherein the analysis of the source code includes using a natural language processing technique.

[1529] (Claim 3)

[1530] 2. The system according to claim 1, further comprising means for obtaining a difference in the source code and generating a new verification pattern based on the difference.

[1531] "Example 2: Combining Emotion Engines"

[1532] (Claim 1)

[1533] means for receiving and analyzing the source code;

[1534] means for generating a verification pattern from the analyzed source code;

[1535] A means for performing an automated test based on the generated verification pattern;

[1536] a means of recording and communicating test results;

[1537] A system including means for recognizing a user's emotions in real time while operating a user interface and generating a verification pattern based thereon.

[1538] (Claim 2)

[1539] 2. The system according to claim 1, wherein the analysis of the source code includes using a natural language processing technique.

[1540] (Claim 3)

[1541] 2. The system according to claim 1, further comprising means for obtaining a difference in the source code and generating a new verification pattern based on the difference.

[1542] "Application example 2 when combining emotion engines"

[1543] (Claim 1)

[1544] means for receiving and analyzing the source code;

[1545] means for generating a verification pattern from the analyzed source code;

[1546] a means for recognizing the emotion of the worker;

[1547] A means for performing an automated test based on the generated verification pattern;

[1548] A system including means for recording and reporting test results and emotional data.

[1549] (Claim 2)

[1550] 2. The system of claim 1, wherein the analysis of the source code includes using natural language processing techniques.

[1551] (Claim 3)

[1552] 2. The system according to claim 1, further comprising: means for acquiring differences in source code and generating a new verification pattern based on the differences; and means for taking into account emotion data of a worker. [Explanation of symbols]

[1553] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving and analyzing the source code; means for generating a verification pattern from the analyzed source code; A means for performing an automated test based on the generated verification pattern; A system that includes a means for recording and communicating test results.

2. 2. The system according to claim 1, wherein the analysis of the source code includes using a natural language processing technique.

3. 2. The system according to claim 1, further comprising means for obtaining differences in the source code and generating a new verification pattern based on the differences.

4. 4. The system according to claim 3, further comprising means for using a version control system to acquire the difference.

5. 10. The system of claim 1, further comprising means for implementing automated operation of the user interface.

6. 6. The system of claim 5, wherein said automated operation includes using an automated tool.

7. 2. The system according to claim 1, further comprising: means for recording the test results as a log; and means for notifying a user of the log.

8. 2. The system according to claim 1, wherein the verification pattern generating means includes means for generating a plurality of verification patterns based on a conditional branch or loop structure.

Images