Oscilloscope control system and method based on natural language

Abstract

The invention discloses an oscilloscope control system and method based on a natural language, and the system achieves the fundamental change of an oscilloscope control mode through the construction of an oscilloscope control architecture of a front-end interface-oscilloscope control middleware-a large language model-oscilloscope control middleware-an oscilloscope-the large language model-the front-end interface. Lossless conversion from a natural language to oscilloscope control feedback is realized for the first time, while professional control precision is maintained, learning cost of a user is greatly reduced, and key technical support is provided for intelligent laboratory construction.

Description

Technical Field

[0001] The present invention belongs to the technical field of test and measurement instrument control, and more particularly, relates to an oscilloscope control system and method based on natural language. Background Art

[0002] As a core instrument in the field of electronic measurement, oscilloscopes play an irreplaceable role in circuit debugging, signal analysis, and fault diagnosis. However, with the upgrading of measurement requirements, the functional complexity of modern oscilloscopes has increased exponentially, resulting in severe challenges for traditional control methods:

[0003] (1) There are two major limitations of the purely manual operation method using physical buttons / knobs: the oscilloscope operation interface is too deep and the parameter configuration logic is cumbersome. Manual control of the oscilloscope makes it impossible for the user to adjust the oscilloscope to the appropriate measurement state in a short time, thus missing the measurement value; in dangerous scenarios such as high voltage and radiation, operators need to frequently approach the equipment to adjust parameters, which not only reduces efficiency but also endangers safety.

[0004] (2) Build a "front-end interface - oscilloscope control middleware - oscilloscope" architecture, and forward the SCPI commands entered by the user in the front-end interface to the oscilloscope through the oscilloscope control middleware, thereby controlling the oscilloscope to complete the specified operation. However, this method still has significant defects: users need to master the oscilloscope's dedicated SCPI command set and need to frequently refer to hundreds of pages of programming manuals; it only supports precise SCPI command control and lacks semantic fault tolerance. Even slight format deviations (such as missing spaces, unconverted parameter units) will cause task execution to fail;

[0005] (3) Construct a "front-end interface - preset semantic conversion middleware - oscilloscope control middleware - oscilloscope" architecture. By introducing a preset semantic conversion layer, regular expressions are used to convert the natural language commands input into SCPI commands before forwarding them to the oscilloscope, thereby achieving natural language control of the oscilloscope. However, this method still has technical bottlenecks: the regular expression rule library is difficult to exhaust, resulting in insufficient coverage; it is unable to parse ambiguous intentions, such as the lack of quantitative parameters for "appropriately increasing time resolution"; the device compatibility is poor, and the regular expression conversion rules need to be rewritten when the oscilloscope model is changed;

[0006] (4) Construct a "large language model-model-end middleware-oscilloscope control middleware-oscilloscope" architecture. The large language model generates preset control instructions based on the received natural language instructions and outputs text information. The model-end middleware monitors the text output of the large language model in real time. When a preset control instruction is detected, it extracts it and stores it in a text file. The oscilloscope control middleware continuously monitors the instruction file generated by the model-end middleware. When the instruction file is updated, it reads the new instruction and forwards it to the oscilloscope, thereby realizing the control of the oscilloscope. However, this method still has obvious shortcomings: the lack of a dedicated oscilloscope prompt project results in the large language model lacking oscilloscope domain knowledge and low SCPI instruction generation accuracy; the lack of dynamic context perception capabilities makes it unable to adapt to the control scenario of continuous instructions; the lack of a closed-loop feedback mechanism makes it impossible to feedback the oscilloscope control results.

[0007] The existing solutions have systematic defects in five dimensions: insufficient depth of natural language understanding, weak device generalization capability, lack of precise control, faults in continuous command control, and lack of feedback on operation results. It is urgent to propose innovative oscilloscope control methods. Summary of the Invention

[0008] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide an oscilloscope control system and method based on natural language. The oscilloscope control method realizes the conversion of natural language into oscilloscope control instructions (Standard Commands for Programmable Instruments, SCPI) through a large language model, and then controls the oscilloscope to execute SCPI instructions and generate an interpretable execution report. The method is suitable for oscilloscope control scenarios.

[0009] To achieve the above-mentioned object of the invention, the present invention provides an oscilloscope control system based on natural language, characterized by comprising:

[0010] Front-end interface: runs on a remote terminal and is connected to the oscilloscope control middleware via a socket. The front-end interface provides a natural language interactive interface. It receives natural language commands input by the user and sends them to the oscilloscope control middleware. It also receives execution reports input by the oscilloscope control middleware and prints them on the interactive interface.

[0011] Oscilloscope control middleware: runs on the oscilloscope's industrial computer, connects to the front-end interface via a socket, connects to the large language model via the Internet, and interconnects with the oscilloscope within the local area network. It is the hub connecting the front-end interface, large language model, and oscilloscope.

[0012] The oscilloscope control middleware consists of two operating phases: the control phase and the feedback phase. The control phase is used to accept natural language commands sent by the front-end interface, build / update prompt words and send them to the large language model, extract the SCPI command sequence from the message returned by the large language model, and send the SCPI command to the oscilloscope. The feedback phase is used to accept the raw byte stream sent by the oscilloscope, update the prompt words and send them to the large language model, then extract the execution report from the message returned by the large language model and send it to the front-end interface.

[0013] Large language model: This runs on an Internet server and is connected to the oscilloscope control middleware via the Internet. The large language model generates inference results based on prompt words and feeds them back to the oscilloscope control middleware. The inference results generated in the control phase are SCPI command sequences, and the inference results generated in the feedback phase are execution reports.

[0014] Oscilloscope: As a lower computer, it is interconnected with the oscilloscope control middleware in the local area network, used to execute the SCPI instructions sent by the oscilloscope control middleware and return the original byte stream to it.

[0015] In addition, the present invention also provides an oscilloscope control method using an oscilloscope control system based on natural language, which is characterized by comprising the following steps:

[0016] (1) Build the front-end interface;

[0017] (2) Build oscilloscope control middleware;

[0018] (3) Utilize the front-end interface and oscilloscope control middleware to achieve lossless conversion from user intent to execution report, thereby realizing the control of the oscilloscope.

[0019] The object of the invention of the present invention is achieved like this:

[0020] The natural language-based oscilloscope control system and method of the present invention achieves a fundamental change in the oscilloscope control method by constructing an oscilloscope control architecture of "front-end interface-oscilloscope control middleware-large language model-oscilloscope control middleware-oscilloscope-large language model-front-end interface". It realizes the lossless conversion from natural language to oscilloscope control feedback for the first time, while maintaining professional-level control accuracy, which will greatly reduce the user's learning cost and provide key technical support for the construction of smart laboratories.

[0021] At the same time, the oscilloscope control system and method based on natural language of the present invention also have the following beneficial effects:

[0022] (1) Compared with the traditional manual operation of the oscilloscope, the present invention realizes the automatic generation of oscilloscope control commands through the intelligent analysis and conversion of natural language instructions by a large language model, which greatly simplifies the operation complexity of the oscilloscope, helps users improve test efficiency and capture more measurement values; the present invention supports remote control of the oscilloscope to avoid damage to measurement personnel in extreme test scenarios;

[0023] (2) Compared with directly sending SCPI commands to the oscilloscope remotely, the present invention can significantly reduce the user's dependence on professional programming skills and oscilloscope operation experience. The user does not need to read the tedious oscilloscope programming manual and can use natural language to complete the precise control of the oscilloscope;

[0024] (3) Compared with using regular expressions to achieve the conversion from natural language to SCPI instructions, the present invention solves the pain point of incomplete coverage of regular expression rules. Developers do not need to exhaust regular expression rules for natural language that may appear in various control scenarios. The large language model can be used to achieve lossless conversion from natural language to SCPI instructions. The present invention also solves the pain point of weak generalization ability of equipment. When the test system needs to change the oscilloscope model, there is no need for developers to update the regular expression rule library. They only need to upload the programming manual of different oscilloscopes to automatically complete the support for specific oscilloscopes.

[0025] (4) Compared with directly using a large language model to generate SCPI instructions, the present invention solves the pain points of low SCPI instruction generation accuracy and continuous instruction control faults. By constructing a special prompt word for the oscilloscope and dynamically updating the prompt word in multiple rounds of control, when the user continuously sends natural language instructions, the present invention can generate accurate SCPI instructions based on the executed SCPI instructions and the fuzzy intention of the current user input; the present invention also solves the pain point of the lack of feedback on operation results. The user can obtain an execution report every time he sends a natural language instruction, thereby improving the user experience and broadening the application boundaries of the oscilloscope in the test system. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a diagram illustrating an architecture of a specific embodiment of an oscilloscope control system based on natural language according to the present invention;

[0027] Figure 2 It is a timing diagram of the oscilloscope control method based on natural language of the present invention;

[0028] Figure 3 It is the display interface of the front-end interface in the embodiment of the present invention;

[0029] Figure 4 This is an excerpt from the Uni-T MSO7000X oscilloscope programming manual used in the embodiments of the present invention;

[0030] Figure 5 This is an excerpt from a dictionary list in an embodiment of the present invention;

[0031] Figure 6 This is an example of the return value of the large language model in the control phase of an embodiment of the present invention;

[0032] Figure 7 This is an example of the return value of the large language model in the feedback phase of an embodiment of the present invention;

[0033] Figure 8 This is the front-end interface and oscilloscope interface after multiple rounds of interaction in the embodiment of the present invention. DETAILED DESCRIPTION

[0034] The following describes the specific embodiments of the present invention in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when detailed descriptions of known functions and designs may dilute the main content of the present invention, such descriptions will be omitted here.

[0035] Example

[0036] For the convenience of description, the relevant professional terms appearing in the specific implementation methods are first explained:

[0037] Socket: A socket is a software abstract interface used for network communication or inter-process communication. It provides data sending and receiving capabilities for applications through a combination of IP address, port number, and transport layer protocol.

[0038] SCPI (Standard Commands for Programmable Instruments): Standard Commands for Programmable Instruments;

[0039] tkinter: a standard graphical user interface development toolkit for the Python programming language;

[0040] queue - a queue management toolkit for the Python programming language.

[0041] threading: a concurrency control, thread management, and resource scheduling toolkit for the Python programming language.

[0042] bs4 - a toolkit for extracting, converting, and cleaning HTML / XML documents in the Python programming language.

[0043] json: a toolkit for optimizing serialization / deserialization and format conversion for the Python programming language.

[0044] HTML (HyperText Markup Language): Hypertext Markup Language;

[0045] role: role type;

[0046] system: system role;

[0047] user: user role;

[0048] assistant: assistant role;

[0049] content: the specific content of each character's prompt;

[0050] OpenAI - a toolkit for the Python programming language that simplifies and enhances the integration and tuning of large language model services.

[0051] API (Application Programming Interface): Application programming interface;

[0052] pyvisa: a standardized instrument control toolkit for the Python programming language.

[0053] VISA (Virtual Instrument Software Architecture): Virtual instrument software architecture;

[0054] Figure 1 This is a specific implementation architecture diagram of the natural language-based oscilloscope control system of the present invention.

[0055] In this embodiment, if Figure 1 As shown in the figure, the oscilloscope control architecture of "front-end interface - oscilloscope control middleware - large language model - oscilloscope control middleware - oscilloscope - large language model - front-end interface" is constructed, specifically:

[0056] Front-end interface: runs on a remote terminal and is connected to the oscilloscope control middleware via a socket. The front-end interface provides a natural language interactive interface. It receives natural language commands input by the user and sends them to the oscilloscope control middleware. It also receives execution reports input by the oscilloscope control middleware and prints them on the interactive interface.

[0057] Oscilloscope control middleware: runs on the oscilloscope's industrial computer, connects to the front-end interface via a socket, connects to the large language model via the Internet, and interconnects with the oscilloscope within the local area network. It is the hub connecting the front-end interface, large language model, and oscilloscope.

[0058] The oscilloscope control middleware consists of two operating phases: the control phase and the feedback phase. The control phase is used to accept natural language commands sent by the front-end interface, build / update prompt words and send them to the large language model, extract the SCPI command sequence from the message returned by the large language model, and send the SCPI command to the oscilloscope. The feedback phase is used to accept the raw byte stream sent by the oscilloscope, update the prompt words and send them to the large language model, then extract the execution report from the message returned by the large language model and send it to the front-end interface.

[0059] Large language model: This runs on an Internet server and is connected to the oscilloscope control middleware via the Internet. The large language model generates inference results based on prompt words and feeds them back to the oscilloscope control middleware. The inference results generated in the control phase are SCPI command sequences, and the inference results generated in the feedback phase are execution reports.

[0060] Oscilloscope: As a lower computer, it is interconnected with the oscilloscope control middleware in the local area network, used to execute the SCPI instructions sent by the oscilloscope control middleware and return the original byte stream to it.

[0061] The present invention also provides an oscilloscope control method using an oscilloscope control system based on natural language, which controls the timing as follows: Figure 2 As shown in the figure, assume that the control commands "add a signal source with a frequency of 18kHz and an amplitude of 50mV", "the vertical signal occupies too small a portion of the screen", "it is still a little small", and "add amplitude measurement and frequency measurement" are sent to the oscilloscope control system in sequence to verify the control and feedback effects of the system. The specific implementation steps are as follows:

[0062] (1) Build the front-end interface;

[0063] (1.1) Use the tkinter component library to build the display interface: The display interface exists in the form of a window, which includes a socket server address configuration panel consisting of a text edit box and a "connect" button, a chat display area consisting of a multi-line text box with a scroll bar, and a natural language input area consisting of a message edit box. In the chat display area, the sending text style and the receiving text style are configured to distinguish between local and remote message interfaces. Figure 3 As shown;

[0064] (1.2) Use the socket component library to implement the connection management function: When you click the "Connect" button, the front-end interface will act as a socket client and initiate a connection request to the IP "127.0.0.1" and port "9999" configured in the socket server address configuration panel;

[0065] (1.3) Implement the asynchronous message sending and receiving function using the queue and threading component libraries: After the front-end interface establishes a connection with the Socket server, create a message sending queue and a message receiving queue, and start the message sending thread and the message receiving thread; the message sending thread loops to read the messages in the sending queue. When a message in the sending queue is read, the front-end interface sends the message to the Socket server through the Socket connection; the messages sent by the Socket server to the front-end interface will be added to the message receiving queue; the message receiving thread loops to read the messages in the receiving queue. When a message in the receiving queue is read, the message is displayed in the chat display area using the received text style.

[0066] (1.4) Bind the Enter key: When the user enters natural language in the natural language input area and presses the Enter key, the natural language will be added to the message sending queue and displayed in the chat display area using the sent text style at the same time.

[0067] (2) Build the oscilloscope control middleware.

[0068] (2.1) Implement the connection management function using the socket component library: When the oscilloscope control middleware starts, it binds the address "127.0.0.1:9999" as the Socket server, starts listening, and loops to accept connections.

[0069] (2.2) Implement the dynamic conversion function from the oscilloscope programming manual to the oscilloscope control knowledge graph using the bs4 and json component libraries.

[0070] (2.2.1) The oscilloscope control middleware dynamically loads the oscilloscope programming manual HTML document from the industrial control computer. Figure 4 It is an excerpt sample of the programming manual for the ULi MSO7000X oscilloscope.

[0071] (2.2.2) Set the SCPI instruction "*CLS" as the query target, and then retrieve the start tag of the node where "*CLS" is located in the oscilloscope programming manual HTML document as " <p id＝'cid_0_0'> ", the end tag is " ".

[0072] (2.2.3) Filter out all SCPI instruction nodes through the prefix "<p id='cid_" of the start tag.

[0073] (2.2.4) Dynamically locate the chapter segmentation tag as " <hr> ", the start tag of the function description node as " ", the end tag is " ", and the start tag of the parameter constraint node as " ", the end tag is " " by comparing "*CLS" with the adjacent SCPI instruction "*ESE value" node and its nested child nodes.

[0074] (2.2.5) Starting from each SCPI instruction node, with the chapter segmentation label " <hr> " as the end point, using the start label "<pid='cid_" and end label " " of the SCPI instruction node, the start label " " and the closing tag " " of the function description node, and the start label " " and the closing tag " " of the parameter constraint node as the judgment basis, capture the content of the SCPI instruction node, function description node, and parameter constraint node in sequence, obtain a dictionary containing three key-value pairs of instruction, function, and parameter, save the dictionary to a list of dictionaries, Figure 5 Sample excerpt of the list of dictionaries;

[0075] (2.2.6) Traverse all SCPI instruction nodes, repeat step (2.2.5), and finally serialize the list of dictionaries into a string to obtain the oscilloscope control knowledge graph;

[0076] (2.3) Implement the prompt management function: The prompt is a text instruction input to the large language model, used to guide the model to generate the expected output, usually composed of three types of roles: the system role prompt for setting global behavior, the user role prompt for driving specific responses, and the assistant role prompt for maintaining context coherence; in the control stage, first dynamically construct the role prompts for each control stage, then combine the role prompts into a control prompt to guide the large language model to generate an SCPI instruction sequence. To reduce the length of the prompts sent to the large language model, only send the oscilloscope control knowledge graph to the large language model when interacting with the large language model for the first time. At the same time, to enable the large language model to have context understanding ability, send the historical natural language instructions and execution results to the large language model every time; in the feedback stage, first dynamically construct the role prompts for each feedback stage, then combine the role prompts into a feedback prompt to guide the large language model to generate an execution report;

[0077] (2.3.1) Construct the system role prompt for the control stage: "{\"role\":\"system\",\"content\":\"Please generate an SCPI instruction sequence by referring to the oscilloscope control knowledge graph. The oscilloscope control knowledge graph is as follows: {oscilloscope control knowledge graph}. Please select an appropriate SCPI instruction according to the function description and replace the variable parameters in the SCPI instruction according to the parameter requirements combined with the user's natural language instructions}\". The \"oscilloscope control knowledge graph\" is dynamically replaced with the string of the oscilloscope control knowledge graph generated in step (2.2.6);

[0078] (2.3.2) After the oscilloscope control middleware receives the natural language instruction sent by the front-end interface, it constructs the user role prompt word for the control phase: "{"role":"user","content":"{natural language instruction}"}". In each round of interaction, the "natural language instruction" is replaced with the natural language instruction sent by the front-end interface. For example, in the first round of interaction, the user role prompt word for the control phase is "{"role":"user","content":"Add a signal source with a frequency of 18 kHz and an amplitude of 50 mV"}".

[0079] (2.3.3) After the oscilloscope control middleware extracts the execution report from the return message of the large language model in the feedback phase, it constructs the assistant role prompt: "{"role":"assistant","content":"{execution report}"}", where "execution report" is dynamically replaced with the oscilloscope's execution report in this round of interaction. For example, the assistant role prompt in the control phase of the first round of interaction is "{"role":"assistant","content":"-AWG1 is enabled. \n-Base frequency is set to 18000. \n-Base amplitude is set to 0.05."}";

[0080] (2.3.4) Group the role prompt words into a set and construct the control prompt words; among them, the control prompt words for the first round of interaction are: "[the system role prompt words constructed in step (2.3.1) and the user role prompt words constructed in step (2.3.2)]"; the control prompt words for non-first round control are: "[the control prompt words constructed in step (2.3.4) in the previous round of interaction, the assistant role prompt words constructed in step (2.3.3) in the previous round of interaction, and the user role prompt words constructed in step (2.3.2) in the current round of interaction]";

[0081] (2.3.5) Build the system role prompt in the feedback phase: "{"role":"system","content":"You are an oscilloscope assistant. Please generate an execution report based on the SCPI commands and corresponding execution results returned by the oscilloscope."}";

[0082] (2.3.6) After the oscilloscope executes the corresponding SCPI command and the oscilloscope control middleware generates the execution result, the user role prompt word is constructed in the feedback phase: "{"role":"user","content":"{execution result}"}", where "execution result" is dynamically replaced with the execution result of the oscilloscope in this round of interaction. For example, the user role prompt word in the feedback phase of the first round of interaction is "{"role":"user","content":":AWG1:ENABle ON executed successfully,:AWG1:BASe:FREQuency 18000 executed successfully,:AWG1:BASe:AMPLitude 0.05 executed successfully"}".

[0083] (2.3.7) The role prompt words constructed from step (2.3.5) to step (2.3.6) are combined into a set to construct the feedback prompt word: "[the system role prompt word constructed in step (2.3.5) in this round of interaction, the user role prompt word constructed in step (2.3.6) in this round of interaction]";

[0084] (2.4) Use the OpenAI component library to implement large language model interaction functions;

[0085] (2.4.1) In the control phase, the control prompt words constructed in step (2.3.4) are sent to the large language model by calling the standard API interface opened by the large language model used, and the SCPI command sequence is extracted from the return value of the large language model through attribute chain access, such as the return value of the large language model in the control phase of the first round of interaction. Figure 6 As shown in the figure, the SCPI command sequence extracted by property chain access is "[":AWG1:ENABle ON",":AWG1:BASe:FREQuency 18000",":AWG1:BASe:AMPLitude 0.05"]";

[0086] (2.4.2) In the feedback phase, the feedback prompt words constructed in step (2.3.7) are sent to the large language model by calling the standard API interface opened by the large language model used, and the execution report is extracted from the return value of the large language model through attribute chain access, such as the return value of the large language model in the feedback phase of the first round of interaction. Figure 7 As shown in the figure, the execution report extracted by property chain access is "'-AWG1 is enabled.\n-The base frequency is set to 18000.\n-The base amplitude is set to 0.05.'";

[0087] (2.5) Use the pyvisa component library to implement the oscilloscope interaction function;

[0088] (2.5.1) Split the SCPI command sequence: Split the SCPI command sequence extracted from the large language model return result into multiple independent SCPI commands. For example, split "[":AWG1:ENABle ON",":AWG1:BASe:FREQuency18000",":AWG1:BASe:AMPLitude 0.05"]" into ":AWG1:ENABle ON", ":AWG1:BASe:FREQuency 18000", and ":AWG1:BASe:AMPLitude 0.05".

[0089] (2.5.2) Connect to the oscilloscope: Create a VISA Explorer, obtain a list of available devices, select the oscilloscope and create an oscilloscope session channel, and send "*IDN?" to verify device activity;

[0090] (2.5.3) Send SCPI commands to the oscilloscope in sequence and generate execution results based on the oscilloscope's return values;

[0091] In the first round of interaction, the SCPI commands ":AWG1:ENABle ON," ":AWG1:BASe:FREQuency 18000," and ":AWG1:BASe:AMPLitude 0.05" are all setup commands. The oscilloscope returns values, and the corresponding execution results are ":AWG1:ENABle ON executed successfully," ":AWG1:BASe:FREQuency 18000 executed successfully," and ":AWG1:BASe:AMPLitude 0.05 executed successfully."

[0092] (2.5.4) Release the session channel: Close the oscilloscope session channel created in step (2.5.2);

[0093] (2.6) Implementing the feedback function: After the oscilloscope control middleware extracts the execution report from the message returned by the large language model, the oscilloscope control middleware sends the execution report to the front-end interface through the established Socket connection;

[0094] (3) Using the front-end interface constructed in step (1) and the oscilloscope control middleware constructed in step (2), lossless conversion from user intent to execution report is achieved;

[0095] (3.1) Make sure the IP address of the industrial computer is "127.0.0.1", the available port is "9999", and the available file path is "D:\files\". Save the oscilloscope programming manual to the "D:\files\" path.

[0096] (3.2) Start the oscilloscope control middleware and open a listening loop to accept connections at "127.0.0.1:9999" through the connection management function implemented in step (2.1);

[0097] (3.3) Start the front-end interface, enter "127.0.0.1:9999" in the socket server address configuration panel, and click the "Connect" button to establish a connection between the oscilloscope control middleware and the front-end interface;

[0098] (3.4) The oscilloscope control middleware converts the oscilloscope programming manual stored on the industrial computer into an oscilloscope control knowledge graph string through the dynamic conversion function of the oscilloscope programming manual to the oscilloscope control knowledge graph implemented in step (2.2);

[0099] (3.5) The user enters "Add a signal source, frequency 18kHz, amplitude 50mV", "The vertical signal occupies too small a portion of the screen", "It's still a bit small", "Add amplitude measurement and frequency measurement" in the natural language input area of the front-end interface. After pressing the Enter key for each round of interaction, the front-end interface sends the natural language command to the oscilloscope control middleware through the Socket connection established in step (3.3);

[0100] (3.6) The oscilloscope control middleware dynamically constructs control prompt words through the prompt word management function implemented in step (2.3);

[0101] (3.7) The oscilloscope control middleware sends the control prompt words to the large language model through the large language model interaction function implemented in step (2.4), and extracts the SCPI command sequence from the large language model return message;

[0102] (3.8) The oscilloscope control middleware implements the oscilloscope interaction function through step (2.5), splits the SCPI command sequence into multiple SCPI commands, establishes an oscilloscope connection, sends SCPI commands to the oscilloscope one by one and generates the oscilloscope execution results corresponding to each SCPI command, and finally closes the oscilloscope connection;

[0103] (3.9) The oscilloscope control middleware dynamically constructs feedback prompt words through the "prompt word management" function implemented in step (2.3);

[0104] (3.10) The oscilloscope control middleware sends the feedback prompt word to the large language model through the large language model interaction function implemented in step (2.4), and extracts the oscilloscope execution report from the large language model return message;

[0105] (3.11), the oscilloscope control middleware implements the feedback function through step (2.6), and sends the execution report to the front-end interface through the Socket connection established in step (3.3);

[0106] (3.12) The front-end interface prints the execution report in the chat display area;

[0107] (3.13) Repeat steps (3.5) to (3.12) to achieve multiple rounds of control and feedback of the oscilloscope. Figure 8 This is the front-end interface and oscilloscope interface after multiple rounds of interaction.

[0108] Although the above describes the illustrative specific embodiments of the present invention to facilitate understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations using the concepts of the present invention are protected.

Claims

1. An oscilloscope control system based on natural language, characterized in that: include: Front-end interface: runs on the remote terminal and is connected to the oscilloscope control middleware via Socket; The front-end interface provides a natural language interactive interface; Receive natural language commands input by the user and send natural language commands to the oscilloscope control middleware; receive execution reports input by the oscilloscope control middleware and print the execution reports on the interactive interface; Oscilloscope control middleware: runs on the oscilloscope's industrial computer, connects to the front-end interface via a socket, connects to the large language model via the Internet, and interconnects with the oscilloscope within the local area network. It is the hub connecting the front-end interface, large language model, and oscilloscope. The oscilloscope control middleware consists of two operating phases: the control phase and the feedback phase. The control phase is used to accept natural language commands sent by the front-end interface, build / update prompt words and send them to the large language model, extract the SCPI command sequence from the message returned by the large language model, and send the SCPI command to the oscilloscope. The feedback phase is used to accept the raw byte stream sent by the oscilloscope, update the prompt words and send them to the large language model, then extract the execution report from the message returned by the large language model and send it to the front-end interface. Large language model: This runs on an Internet server and is connected to the oscilloscope control middleware via the Internet. The large language model generates inference results based on prompt words and feeds them back to the oscilloscope control middleware. The inference results generated in the control phase are SCPI command sequences, and the inference results generated in the feedback phase are execution reports. Oscilloscope: As a lower computer, it is interconnected with the oscilloscope control middleware in the local area network, used to execute the SCPI instructions sent by the oscilloscope control middleware and return the original byte stream to it.

2. A natural language based oscilloscope control method, characterized in that: The following steps are involved: (1) Build the front-end interface; (1.1) Use the tkinter component library to build the display interface: The display interface exists in the form of a window, which includes a socket server address configuration panel consisting of a text edit box and a "connect" button, a chat display area consisting of a multi-line text box with a scroll bar, and a natural language input area consisting of a message edit box. The send text style and receive text style are configured in the chat display area to distinguish between locally sent and remotely received messages; (1.2) Use the socket component library to implement the connection management function: When you click the "Connect" button, the front-end interface will act as a Socket client and initiate a connection request to the IP and port configured in the Socket server address configuration panel; (1.3) Use the queue and threading component library to implement asynchronous message sending and receiving functions: After the front-end interface establishes a connection with the Socket server, it creates a message sending queue and a message receiving queue, and starts the message sending thread and the message receiving thread; the message sending thread cyclically reads the messages in the sending queue. When the message in the sending queue is read, the front-end interface sends the message to the Socket server through the Socket connection; the message sent by the Socket server to the front-end interface will be added to the message receiving queue; the message receiving thread cyclically reads the messages in the receiving queue. When the message in the receiving queue is read, the message is displayed in the chat display area using the received text style; (1.4) Bind the Enter key: When the user enters a natural language in the natural language input area and presses the Enter key, the natural language will be added to the message sending queue and displayed in the chat display area using the sent text style; (2) Build oscilloscope control middleware; (2.1) Use the socket component library to implement the connection management function: When the oscilloscope control middleware starts, it reads the IP and port address set by the user, binds the IP and port address as the socket server, starts listening, and accepts connections in a loop; (2.2) Use bs4 and json component libraries to realize the dynamic conversion function from oscilloscope programming manual to oscilloscope control knowledge graph; (2.2.1) The oscilloscope control middleware dynamically loads the oscilloscope programming manual HTML document from the industrial computer; (2.2.2) Set a common SCPI command as the query target, and then search for the start and end tags of the node where the SCPI command is located in the HTML document of the oscilloscope programming manual; (2.2.3) Filter out all SCPI command nodes in the oscilloscope programming manual HTML document by the prefix of the start tag; (2.2.4) Dynamically locate chapter segmentation tags, the start and end tags of function description nodes, and the start and end tags of parameter constraint nodes by comparing adjacent SCPI command nodes and their nested subnodes; (2.2.5) Taking each SCPI instruction node as the starting point and the chapter separation tag as the end point, and using the start tag and end tag of the SCPI instruction node, the start tag and end tag of the function description node, and the start tag and end tag of the parameter constraint node as the judgment basis, sequentially capture the contents of the SCPI instruction node, the function description node, and the parameter constraint node, obtain a dictionary containing three key-value pairs of instruction, function, and parameter, and save the dictionary to a dictionary list; (2.2.6) Traverse all SCPI command nodes and repeat step (2.2.5). Finally, serialize the dictionary list into a string to obtain the oscilloscope control knowledge graph. (2.3) Realize prompt word management function; (2.3.1) Construct the system role prompt words in the control phase: "{"role":"system","content":"Please refer to the oscilloscope control knowledge graph to generate the SCPI command sequence. The oscilloscope control knowledge graph is as follows: {Oscilloscope control knowledge graph}, please refer to the function description to select the appropriate SCPI command, and replace the variable parameters in the SCPI command according to the parameter requirements and the user's natural language instructions"}", where "Oscilloscope control knowledge graph" is dynamically replaced with the oscilloscope control knowledge graph string generated in step (2.2.6); (2.3.2) After the oscilloscope control middleware receives the natural language command sent by the front-end interface, it constructs the user role prompt words for the control phase: "{"role":"user","content":"{natural language command}"}", where "natural language command" is dynamically replaced with the natural language command sent by the front-end interface in this round of interaction; (2.3.3) After the oscilloscope control middleware extracts the execution report from the return message of the large language model in the feedback phase, it constructs the assistant role prompt: "{"role":"assistant","content":"{execution report}"}", where "execution report" is dynamically replaced with the execution report of the oscilloscope in this round of interaction; (2.3.4) Group the role prompt words into a set and construct the control prompt words; among them, the control prompt words for the first round of interaction are: "[the system role prompt words constructed in step (2.3.1) and the user role prompt words constructed in step (2.3.2)]"; the control prompt words for non-first round control are: "[the control prompt words constructed in step (2.3.4) in the previous round of interaction, the assistant role prompt words constructed in step (2.3.3) in the previous round of interaction, and the user role prompt words constructed in step (2.3.2) in the current round of interaction]"; (2.3.5) Build the system role prompt in the feedback phase: "{"role":"system","content":"You are an oscilloscope assistant. Please generate an execution report based on the SCPI commands and corresponding execution results returned by the oscilloscope"}"; (2.3.6) After the oscilloscope executes the corresponding SCPI command and the oscilloscope control middleware generates the execution result, a user role prompt is constructed for the feedback phase: "{"role":"user","content":"{execution result}"}", where "execution result" is dynamically replaced with the execution result of the oscilloscope in this round of interaction; (2.3.7) The role prompt words constructed from step (2.3.5) to step (2.3.6) are combined into a set to construct the feedback prompt word: "[the system role prompt word constructed in step (2.3.5) in this round of interaction, the user role prompt word constructed in step (2.3.6) in this round of interaction]"; (2.4) Use the OpenAI component library to implement large language model interaction functions; (2.4.1) Control phase: By calling the standard API interface opened by the large language model used, the control prompt words constructed in step (2.3.4) are sent to the large language model, and the SCPI command sequence is extracted from the return value of the large language model through attribute chain access; (2.4.2) Feedback phase: By calling the standard API interface opened by the large language model used, the feedback prompt word constructed in step (2.3.7) is sent to the large language model, and the execution report is extracted from the return value of the large language model through attribute chain access; (2.5) Use the pyvisa component library to implement the oscilloscope interaction function; (2.5.1) Split the SCPI command sequence: Split the SCPI command sequence extracted from the result returned by the large language model into multiple independent SCPI commands; (2.5.2) Connect to the oscilloscope: Create a VISA Explorer, obtain a list of available devices, select the oscilloscope and create an oscilloscope session channel, and send "*IDN?" to verify device activity; (2.5.3) Send SCPI commands to the oscilloscope in sequence and generate execution results based on the oscilloscope's return values; If the SCPI command contains "?", it is a query command. If the oscilloscope returns "NaN", the execution result is "{SCPI command} query failed", otherwise the execution result is "{SCPI command} query successful: {query value}", where "SCPI command" and "query value" are dynamically replaced with the specific query command and the corresponding query result. If the SCPI command does not contain "?", it is a setting command. If the oscilloscope returns a value, the execution result is "{SCPI command} execution successful", otherwise the execution result is "{SCPI command} execution failed", where "SCPI command" is dynamically replaced with the specific setting command. (2.5.4) Release the session channel: Close the oscilloscope session channel created in step (2.5.2); (2.6) Implementing the feedback function: After the oscilloscope control middleware extracts the execution report from the message returned by the large language model, the oscilloscope control middleware sends the execution report to the front-end interface through the established Socket connection; (3) Using the front-end interface constructed in step (1) and the oscilloscope control middleware constructed in step (2), lossless conversion from user intent to execution report is achieved; (3.1) Determine the IP, available port and available file path of the industrial computer, and save the oscilloscope programming manual to the available file path; (3.2) Start the oscilloscope control middleware and use the connection management function implemented in step (2.1) to open a listening loop to accept connections at the IP and port determined in step (3.1); (3.3) Start the front-end interface, enter the IP and port determined in step (3.1) in the Socket server address configuration panel, and click the "Connect" button to establish a connection between the oscilloscope control middleware and the front-end interface; (3.4) The oscilloscope control middleware converts the oscilloscope programming manual stored on the industrial computer into an oscilloscope control knowledge graph string through the dynamic conversion function of the oscilloscope programming manual to the oscilloscope control knowledge graph implemented in step (2.2); (3.5) The user enters a natural language command in the natural language input area of the front-end interface and presses the Enter key to send it. The front-end interface sends the natural language command to the oscilloscope control middleware through the Socket connection established in step (3.3); (3.6) The oscilloscope control middleware dynamically constructs control prompt words through the prompt word management function implemented in step (2.3); (3.7) The oscilloscope control middleware sends the control prompt words to the large language model through the large language model interaction function implemented in step (2.4), and extracts the SCPI command sequence from the large language model return message; (3.8) The oscilloscope control middleware implements the oscilloscope interaction function through step (2.5), splits the SCPI command sequence into multiple SCPI commands, establishes an oscilloscope connection, sends SCPI commands to the oscilloscope one by one and generates the oscilloscope execution results corresponding to each SCPI command, and finally closes the oscilloscope connection; (3.9) The oscilloscope control middleware dynamically constructs feedback prompt words through the "prompt word management" function implemented in step (2.3); (3.10) The oscilloscope control middleware sends the feedback prompt word to the large language model through the large language model interaction function implemented in step (2.4), and extracts the oscilloscope execution report from the large language model return message; (3.11), the oscilloscope control middleware implements the feedback function through step (2.6), and sends the execution report to the front-end interface through the Socket connection established in step (3.3); (3.12) The front-end interface prints the execution report in the chat display area; (3.13) Repeat steps (3.5) to (3.12) to achieve multiple rounds of control and feedback of the oscilloscope.

Images