Code assisting method and system based on cursor context perception and multi-mode AI linkage
By monitoring the cursor position in the code editing interface to build an AST syntax tree and a cross-file reference graph, generating a context feature vector, and selecting local or cloud AI service assistance, it solves the problems of sensitive data leakage, code completion delays, and resource usage in the development environment, and implements efficient and secure code assistance functions.
Patent Information
- Application Number
- CN202510657748.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-21
- Publication Date
- 2025-09-05
AI Technical Summary
Existing technologies have risks of sensitive data leakage, high code completion delays, performance bottlenecks caused by excessive resource usage, and poor user experience in development environments, especially severe input lag when developing large projects.
By monitoring the cursor position in the code editing interface, dynamically building an AST syntax tree and a cross-file reference graph, generating a context feature vector, and selecting local or cloud AI service assistance based on complexity, seamless switching between local processing of sensitive code and cloud collaboration in complex scenarios can be achieved, and cloud models can be updated in combination with federated learning.
It implements efficient, smooth, and secure code assistance functions in the code editor, balances privacy and performance, and provides an excellent user experience.
Smart Images
Figure CN120596087A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a code assistance method and system based on cursor context perception and multimodal AI linkage, a computing device, and a computer-readable storage medium. Background Art
[0002] Modern software development environments face multiple key challenges when handling complex projects. For example, using AI-related features such as AI code completion in a development environment requires uploading local code to a third-party server, which poses the risk of sensitive data leakage. If a large local model is used, the completion delay is high due to computing power limitations. There are also resource usage and performance bottlenecks. When running multiple functions such as syntax checking and type inference, excessive memory resources are used, resulting in input lag during large-scale project development and a poor user experience. Summary of the Invention
[0003] In view of this, the embodiments of the present application provide a code assistance method and system, a computing device and a computer-readable storage medium based on cursor context perception and multimodal AI linkage to solve the technical defects existing in the prior art.
[0004] According to a first aspect of an embodiment of the present application, a code assistance method based on cursor context perception and multimodal AI linkage is provided, comprising:
[0005] Monitor the cursor position in the code editing interface, record the cursor trajectory, and dynamically build the first AST syntax tree based on the current cursor position;
[0006] Scan the project structure to build a cross-file reference graph;
[0007] Constructing a context feature vector according to the cursor track, the first AST syntax tree, and the cross-file reference graph;
[0008] Selecting to use a local inference service or requesting assistance from a cloud service based on the complexity of the context feature vector;
[0009] The completion suggestions returned by the local inference service are displayed in the code editing interface.
[0010] According to a second aspect of an embodiment of the present application, a coding assistance system based on cursor context perception and multimodal AI linkage is provided, including:
[0011] A monitoring module is used to monitor the cursor position and record the cursor trajectory in the code editing interface, and dynamically construct the first AST syntax tree according to the current cursor position;
[0012] Scanning module, used to scan the project structure and build a cross-file reference graph;
[0013] A generation module, configured to construct a context feature vector based on the cursor track, the first AST syntax tree, and the cross-file reference graph;
[0014] A routing module, configured to select, based on the complexity of the context feature vector, a local inference service or a cloud service for assistance;
[0015] A display module, configured to display completion suggestions returned by the local inference model in a code editing interface;
[0016] As well as local inference services and cloud services.
[0017] According to a third aspect of an embodiment of the present application, a computing device is provided, comprising a memory, a processor, and computer instructions stored on the memory and executable on the processor. When the processor executes the instructions, the steps of the code assistance method based on cursor context perception and multimodal AI linkage are implemented.
[0018] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores computer instructions, which, when executed by a processor, provide steps of a code assistance method based on cursor context perception and multimodal AI linkage.
[0019] In the above embodiment of the present application, the user's cursor trajectory is tracked in real time in the code editor, the n-layer AST syntax tree at the current cursor position is dynamically obtained, and the context feature vector is generated in combination with the cursor trajectory and project dependencies and other features; in this process, the cursor trajectory is dynamically integrated with the code semantics, and the weights of the behavioral features and semantic features in the context features are dynamically adjusted according to the current actions of the developer. Furthermore, a dynamic routing method is adopted to select the optimal path according to the complexity of the context feature vector. When the complexity is low, the local reasoning service including the base layer is selected to ensure timely response of code assistance; when the complexity is high, a cloud service including an enhanced layer large model is requested for assistance. For cloud services, a large number of features encrypted and uploaded by clients are aggregated through a federated learning service, and the cloud-based enhanced layer large model is continuously updated through user feedback. The embodiment of the present application realizes the seamless switching between local processing of sensitive code and cloud-based collaboration of complex scenarios through the above method, balances privacy and performance, and provides a better user experience in the code editor environment. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 is a structural block diagram of a computing device provided in an embodiment of the present application;
[0021] Figure 2This is a flowchart of a code assistance method based on cursor context perception and multimodal AI linkage provided by an embodiment of the present application;
[0022] Figure 3 This is an architectural diagram of a code assistance system based on cursor context perception and multimodal AI linkage provided by an embodiment of the present application. DETAILED DESCRIPTION
[0023] The following description sets forth many specific details to facilitate a thorough understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of the present application. Therefore, the present application is not limited to the specific implementations disclosed below.
[0024] The terms used in one or more embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of the present application. The singular forms "a", "the" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more associated listed items.
[0025] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of the present application, first may also be referred to as second, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "in response to determining."
[0026] In this application, a code assistance method and system based on cursor context perception and multimodal AI linkage, a computing device and a computer-readable storage medium are provided, which are described in detail one by one in the following embodiments.
[0027] Figure 1 1 shows a block diagram of a computing device 100 according to an embodiment of the present application. Components of the computing device 100 include, but are not limited to, a memory 110 and a processor 120. The processor 120 is connected to the memory 110 via a bus 130, and a database 150 is used to store data.
[0028] The computing device 100 also includes an access device 140 that enables the computing device 100 to communicate via one or more networks 160. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 140 may include one or more of any type of network interface (e.g., a network interface card (NIC)), whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.
[0029] In one embodiment of the present application, the above components of the computing device 100 and Figure 1 Other components not shown in the figure may also be connected to each other, for example, via a bus. Figure 1 The computing device structure block diagram shown is for illustrative purposes only and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.
[0030] The computing device 100 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 100 may also be a mobile or stationary server.
[0031] In existing development environments, such as code editor tools, cloud-based AI code completion tools (such as Copilot) provide powerful real-time suggestion capabilities, but they require uploading the code to a third-party server. This method of data transmission may lead to the risk of leakage of sensitive corporate information; if you choose to run a lightweight model locally, due to computing resource limitations, the response time for code completion becomes very long, exceeding 500 milliseconds, affecting development efficiency. In localized solutions, if syntax checking, type inference, and AI completion are enabled at the same time, the development environment will also have significant resource usage, often resulting in memory usage exceeding 2GB. Therefore, input jams will occur when processing large projects, seriously affecting the developer's work experience. In addition, these code editors cannot predict and mark possible errors in advance during the coding stage, nor can they provide semantic refactoring suggestions during the coding process. This lack of functionality makes it difficult for developers to receive holistic support during debugging and refactoring.
[0032] Therefore, in order to solve the above problems, in the embodiment of the present application, a code assistance method and system based on cursor context perception and multimodal AI linkage is proposed. Next, a code assistance method based on cursor context perception and multimodal AI linkage provided by the embodiment of the present application is described, see Figure 2 , the method includes steps 202 to 210.
[0033] Step 202: Monitor the cursor position and record the cursor trajectory in the code editing interface, and dynamically construct a first AST syntax tree according to the current cursor position.
[0034] In an embodiment of the present application, the user's cursor behavior is continuously monitored in the code editing interface to obtain the user's cursor trajectory, focusing on recording the user's cursor behavior model during the programming process. The cursor trajectory includes the user's cursor movement path and editing frequency, dwell position and duration, as well as editing operations performed in specific code areas. The real-time recorded user cursor trajectory is then saved in a behavior log and a behavior trajectory database is constructed, for example:
[0035] {
[0036] timestamp:"2025-01-05T14:23:15", / / The specific time when the operation occurred
[0037] file:"src / components / UserList.tsx", / / The file path where the operation occurs
[0038] position:{line:45,column:12}, / / cursor position
[0039] action:"delete→type→pause(300ms)" / / Describes the specific operation sequence of the cursor and the cursor stay time
[0040] }
[0041] Furthermore, when it is monitored that the cursor stay time is greater than a first threshold, the AST syntax tree of the current cursor position is captured: including dynamically parsing the AST syntax tree of the current file, and then identifying the syntax structure of the cursor position to obtain a first AST syntax tree.
[0042] Specifically, in a feasible implementation, when it is detected that the cursor stays for more than 300ms, the cursor position is obtained, the entire file where the cursor is located is parsed to generate an AST syntax tree for the entire file, the AST syntax tree is further traversed, the corresponding AST syntax node is found according to the cursor position (such as real-time parsing of the n-layer syntax tree at the cursor position), and its node type, hierarchical relationship, semantics and other information are determined to obtain the first AST syntax tree, for example:
[0043] Identify function parameters:
[0044] Find the line where the cursor is located and check whether it is in a function declaration or call node; determine whether the cursor is in the parameter list.
[0045] Or, to identify the JSX element level:
[0046] Find the node where the cursor is located, calculate the nesting level, and calculate the depth by traversing the parent node ancestors.
[0047] Step 204: Scan the project structure to build a cross-file reference graph.
[0048] In an embodiment of the present application, all files under the project structure of the current project are further scanned to construct a cross-file reference graph.
[0049] Specifically, recursively scan the project root directory and collect all source code files (such as .js, .py, .java).
[0050] Furthermore, the module structure is identified based on language characteristics, for example:
[0051] JavaScript: exports or import path resolution based on package.json;
[0052] Python: Determine the package hierarchy through __init__.py and sys.path;
[0053] Java: Resolve module dependencies based on pom.xml or Gradle configuration.
[0054] Furthermore, the code structure of each file is parsed to extract cross-file reference relationships, including:
[0055] Syntax parsing: Use the parser to generate an AST syntax tree;
[0056] Symbol table construction: traverse AST to record all definitions (Definitions) and references (References):
[0057] Definition: function declaration, class definition, variable assignment, etc.;
[0058] Reference: function call, variable use, class inheritance, etc.
[0059] Cross-file association:
[0060] Detect import statements (such as import, require) and symbol references in the code:
[0061] For example, for the JavaScript package.json file, identify the third-party libraries that the project depends on and their entry files, and map import_from'lodash' to the actual file path for subsequent graph association.
[0062] For example, parse the path alias in the tsconfig.json file and convert the path alias (such as @ / *) into the actual file path; then, when traversing the AST, replace the alias path in the code with the physical path.
[0063] It also includes parsing .gitignore rules to skip directories that do not need to be analyzed.
[0064] Furthermore, the dependencies and symbolic references between the above files are structured and represented through a cross-file reference graph. The cross-file reference graph is a directed graph, where nodes represent files or code entities (such as functions, classes, and variables), and edges represent file dependencies and reference relationships (such as imports, calls, and inheritance).
[0065] Step 206: Construct a context feature vector based on the cursor track, the first AST syntax tree, and the cross-file reference graph.
[0066] In an embodiment of the present application, the raw data from the cursor trajectory, the first AST syntax tree, and the cross-file reference graph are vectorized and integrated into a context feature vector for model processing. The cursor trajectory data is in the form of a time series, including the path, position, and time-related information of the cursor movement in the editor.
[0067] Specifically, in a feasible implementation, the numerical vectors of the above-mentioned cursor trajectory, AST syntax tree and cross-file reference graph are spliced into a 128-dimensional context feature vector through steps such as splicing, normalization and dimensionality reduction.
[0068] Preferably, when generating the context feature vector, behavior-semantics joint modeling is performed, and the depth of context feature analysis is adjusted according to the current activity of the developer.
[0069] In one feasible implementation, the fusion weight of the cursor trajectory (behavioral feature) and the first AST syntax tree (semantic feature) is adjusted based on the developer's current activity, where the developer's current activity is obtained based on the user's cursor behavior log statistics.
[0070] For example, when a user frequently edits, you should pay more attention to the user's behavioral characteristics:
[0071] For highly active users (fast cursor movement, frequent editing), the behavioral feature weight is increased (0.8*behavioral feature + 0.2*semantic feature) to capture real-time operational intent.
[0072] When users stay in a certain code area or grammatical structure for a long time, they may pay more attention to semantic features:
[0073] When user behavior is low in activity (cursor stagnant, code viewing for a long time), the weight of semantic features is increased (0.3*behavioral features + 0.7*semantic features) to deeply analyze the code structure.
[0074] In another possible implementation, the fusion weight of the cursor trajectory and the first AST syntax tree is adjusted based on the context of the cursor's location in the code. For example, if the cursor is inside a macro (such as defineProps<|>), the semantic feature weight is increased to further analyze the code structure.
[0075] Preferably, the user's personal coding preferences are statistically analyzed based on the user's behavior trajectory database to generate a user profile. The cursor trajectory, AST syntax tree, cross-file reference map, and numerical vector of the user profile are concatenated into a context feature vector.
[0076] Step 208: Select to use a local reasoning model or cloud service for assistance based on the complexity corresponding to the context feature vector.
[0077] In an embodiment of the present application, in this step, it is selected to use a local inference model to perform inference based on the context feature vector according to the complexity corresponding to the context feature vector, or to use a cloud model to perform inference based on the context feature vector.
[0078] Furthermore, the calculation method of the complexity corresponding to the context feature vector includes the following steps:
[0079] 1. Complexity extraction of basic features
[0080] (1) Calculate the structural complexity of the first AST syntax tree
[0081] Analyze the n-layer code structure (such as functions, classes, and logic blocks) near the cursor position to capture the hierarchical relationship of the code; assign different weights to nodes that affect complexity, and recursively accumulate the scores of child nodes.
[0082] (2) Calculating cross-file dependency complexity
[0083] Count all external modules introduced by import or require in the project, and quantify the concentration of dependency distribution through entropy. The higher the entropy value, the more dispersed the dependencies are, and the lower the module coupling is. For example:
[0084] Calculate the dependency entropy value of the current file:
[0085] entropy = -sum(p*log(p)for p in import_frequency.values()), where p is the frequency of occurrence of each dependent item.
[0086] Preferably, the type system complexity can also be calculated. For example, for TypeScript code, generic nesting detection is performed, such as Array <Map<string,number> >More complex than string, the former nests two layers of generics; union types, such as string|number|boolean, are more complex than single types.
[0087] 2. Behavioral feature complexity extraction
[0088] Calculate the abnormality of the cursor trajectory, that is, detect whether the developer has encountered obstacles by using the cursor movement pattern. In one possible implementation, a dynamic time warping algorithm is used to compare the current cursor trajectory (movement speed, pause position) with the historical normal pattern and calculate the trajectory difference degree dtw_distance. Abnormal trajectories (such as frequent rollbacks) may indicate difficulty in understanding the code or complex logic.
[0089] Preferably, the probability of editing conflicts is calculated. In a feasible implementation, an LSTM model can be used to analyze the editing operations (addition, deletion, and modification frequency) in the past 30 seconds to predict the probability of code modification conflicts caused by multi-person collaboration or version asynchrony in the next n seconds.
[0090] 3. Comprehensive calculation and decision-making
[0091] In this step, the weighted fusion formula (FCS) is used to integrate the multi-dimensional features to generate an overall complexity score. In a feasible implementation, the complexity scores of the above different features are weighted and integrated to comprehensively evaluate the overall complexity of the current coding environment. For example:
[0092] Syntax tree structure (35%): the dominant factor, directly reflecting the complexity of the code structure.
[0093] Dependency entropy (25%): The degree of modularity affects maintenance costs.
[0094] Type system (20%): Type complexity affects the development experience.
[0095] Behavioral characteristics (15%): Cursor anomalies and conflict risks reflect real-time status.
[0096] Furthermore, the above complexity score is compared with the preset model threshold. When the complexity score is greater than the model threshold, the cloud service is requested for assistance; when the complexity score is less than the model threshold, the model routing sends the encrypted context feature vector to the local inference service.
[0097] To illustrate the overall complexity score above with a specific implementation, when a developer writes a complex generic Hook in a React component:
[0098] First AST syntax tree structure complexity calculation: Deep generic useState detected <Array<Record<string,number> >>, structure score 1.36.
[0099] Dependency entropy: 12 dependent modules are introduced, and the entropy value is 1.34, reflecting high module coupling.
[0100] Type complexity: Nested generics score 0.32.
[0101] Behavioral characteristics: Frequent cursor corrections result in a DTW anomaly of 0.91 and a conflict risk of 0.65.
[0102] Comprehensive calculation: FCS = 0.35*1.36+0.25*1.34+0.20*0.32+0.15*(0.91*0.65)=0.964, which is greater than the model threshold of 0.9, triggering cloud-based collaborative reasoning.
[0103] In one feasible implementation, the large inference model in the local inference service is a large base-layer model accelerated by the NPU, such as the first n layers of the CodeLlama model. When the complexity score is less than the model threshold, the local inference service directly infers completion suggestions based on the context feature vector and returns them to the client's code editing interface. The training and inference processes of the local inference service are both performed in the local TEE security zone.
[0104] When cloud-based collaborative reasoning is triggered, the model router sends an assistance request to the cloud service. In response to the assistance request, the cloud service sends the enhanced layer large model parameters to the local reasoning model. The local reasoning model receives the enhanced layer large model parameters, further performs local optimization based on the enhanced layer parameters, and then reasons on the context feature vector to provide completion suggestions.
[0105] In one possible implementation, the cloud-based enhancement layer model, such as the last m layers of the CodeLlama model, is continuously updated by the federated learning server. Specifically, the federated learning server's parameter aggregation function continuously updates the cloud-based enhancement layer model parameters based on the encrypted features uploaded by a large number of clients.
[0106] Preferably, the above model thresholds are automatically adjusted based on changes in device performance (e.g., GPU load). For example, when GPU load is high, the model threshold is lowered (e.g., by 20% at 80% utilization) to prioritize cloud resources. When network latency is high, the threshold is appropriately increased (e.g., by 10% when latency is <50ms) to encourage local processing.
[0107] Step 210: Display the completion suggestions returned by the local inference service in the code editing interface.
[0108] In this step, the completion suggestions returned by the local inference service are displayed in the code editing interface, for example, by triggering a floating UI and dynamically loading a comparison view of the completion suggestions and refactoring solutions.
[0109] Furthermore, in a feasible implementation, when the user adopts the completion suggestion, the user's selection result is fed back to the federated learning server.
[0110] Specifically, the local inference model updates local parameters;
[0111] Furthermore, the client sends the encrypted context feature vector to the federated learning server, for example, through a double-encrypted channel, using AES-256 and a quantum-safe algorithm for feature transmission.
[0112] Preferably, differential privacy noise is added before encryption, and Laplace noise is added to the context feature vector to prevent reverse deduction of the original code:
[0113] noisy_vector=original_vector+np.random.laplace(scale=0.1)
[0114] Furthermore, the federated learning server continuously updates the cloud-based enhancement layer model based on feedback from numerous clients through a parameter aggregation service. The federated learning server averages the key contextual features uploaded by each client to generate a more balanced cloud-based enhancement layer model. Therefore, the core content uploaded by the client to the federated learning server is the encrypted and desensitized contextual features, not the original code data. Privacy-preserving technologies (encryption, desensitization, and differential privacy) ensure user code privacy, while federated aggregation improves the code understanding and completion capabilities of the cloud-based enhancement layer model.
[0115] In the above embodiment of the present application, in order to realize efficient, smooth and safe code assistance function in the code editor, the user's cursor trajectory is first tracked in real time in the code editor, the n-layer AST syntax tree at the current cursor position is dynamically obtained, and the context feature vector is generated by combining the cursor trajectory and project dependencies and other features; in this process, the cursor trajectory is dynamically integrated with the code semantics, and the weights of the behavioral features and semantic features in the context features are dynamically adjusted according to the current actions of the developer. Furthermore, a dynamic routing method is adopted to select the optimal path according to the complexity of the context feature vector. When the complexity is low, the local reasoning service including the base layer is selected to ensure the timely response of the code assistance; when the complexity is high, a cloud service including the enhanced layer large model is requested for assistance. For cloud services, a large number of features encrypted and uploaded by the client are aggregated through the federated learning service, and the cloud enhanced layer large model is continuously updated through user feedback. The embodiment of the present application realizes the seamless switching between local processing of sensitive code and cloud collaboration of complex scenarios through the above method, balances privacy and performance, and provides a better user experience in the code editor environment.
[0116] Corresponding to the above method embodiment, the present application also provides an embodiment of a code assistance system based on cursor context perception and multimodal AI linkage, such as Figure 3 As shown, the system includes:
[0117] A monitoring module is used to monitor the cursor position and record the cursor trajectory in the code editing interface, and dynamically construct the first AST syntax tree according to the current cursor position;
[0118] Scanning module, used to scan the project structure and build a cross-file reference graph;
[0119] A generation module, configured to construct a context feature vector based on the cursor track, the first AST syntax tree, and the cross-file reference graph;
[0120] A routing module, configured to select, based on the complexity of the context feature vector, a local inference service or a cloud service for assistance;
[0121] A display module, configured to display completion suggestions returned by the local inference model in a code editing interface;
[0122] As well as local inference services and cloud services.
[0123] The above is a schematic scheme of a code assistance system based on cursor context perception and multimodal AI linkage in this embodiment. It should be noted that the technical solution of this system and the technical solution of the code assistance method based on cursor context perception and multimodal AI linkage mentioned above belong to the same concept. For details not described in detail in the technical solution of this system, please refer to the description of the technical solution of the code assistance method based on cursor context perception and multimodal AI linkage mentioned above.
[0124] In one embodiment of the present application, a computing device is also provided, including a memory, a processor, and computer instructions stored in the memory and executable on the processor. When the processor executes the instructions, the steps of the code assistance method based on cursor context perception and multimodal AI linkage are implemented.
[0125] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned code assistance method based on cursor context perception and multimodal AI linkage belong to the same concept. For details not described in detail in the technical scheme of the computing device, please refer to the description of the technical scheme of the above-mentioned code assistance method based on cursor context perception and multimodal AI linkage.
[0126] An embodiment of the present application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of a code assistance method based on cursor context perception and multimodal AI linkage as described above.
[0127] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned code assistance method based on cursor context perception and multimodal AI linkage belong to the same concept. For details not described in detail in the technical scheme of the storage medium, please refer to the description of the technical scheme of the above-mentioned code assistance method based on cursor context perception and multimodal AI linkage.
[0128] The foregoing description describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0129] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include: any entity or system capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.
[0130] It should be noted that for the aforementioned method embodiments, for ease of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.
[0131] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.
[0132] The preferred embodiments of the present application disclosed above are intended only to help illustrate the present application. The optional embodiments do not describe all details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made based on the content of this application. This application selects and describes these embodiments in detail in order to better explain the principles and practical applications of this application, so that those skilled in the art can better understand and utilize this application. This application is limited only by the claims and their full scope and equivalents.
Claims
1. A code assistance method based on cursor context perception and multimodal AI linkage, characterized in that: include: Monitor the cursor position in the code editing interface, record the cursor trajectory, and dynamically build the first AST syntax tree based on the current cursor position; Scan the project structure to build a cross-file reference graph; Constructing a context feature vector according to the cursor track, the first AST syntax tree, and the cross-file reference graph; Selecting to use a local inference service or requesting assistance from a cloud service based on the complexity of the context feature vector; The completion suggestions returned by the local inference service are displayed in the code editing interface.
2. The method according to claim 1, wherein Dynamically constructing the first AST syntax tree based on the current cursor position includes: When it is detected that the cursor stay time is greater than the first threshold, the AST syntax tree of the current file is parsed, and then the AST syntax tree is traversed to identify the n-layer syntax structure at the cursor position to obtain the first AST syntax tree, where n>0.
3. The method according to claim 1, wherein Scanning the project structure to build a cross-file reference graph includes: Import statements and symbol references in the code are detected to generate a cross-file reference graph, which is a directed graph in which nodes represent files or code entities and edges represent file dependencies and reference relationships.
4. The method according to claim 1, wherein Constructing a context feature vector according to the cursor track, the first AST syntax tree, and the cross-file reference graph includes: When generating the context feature vector, the fusion weight of the cursor track and the first AST syntax tree is adjusted according to the current activity of the developer and / or the fusion weight of the cursor track and the first AST syntax tree is adjusted according to the context position of the cursor in the code.
5. The method according to claim 1, wherein Constructing a context feature vector according to the cursor track, the first AST syntax tree, and the cross-file reference graph further includes: Generate a user profile based on the user's behavior trajectory record; splice the cursor trajectory, AST syntax tree, cross-file reference map and the numerical vector of the user profile into the context feature vector.
6. The method according to claim 1, wherein The calculation methods of the complexity corresponding to the context feature vector include: Calculate the structural complexity of the first AST syntax tree, the cross-file dependency complexity, and the abnormality of the cursor trajectory; assign weights to the complexity scores of the above different features and then perform weighted integration to obtain the complexity score corresponding to the context feature vector.
7. The method according to claim 6, wherein: Selecting to use a local inference service or requesting cloud services for assistance based on the complexity of the context feature vector includes: The complexity score is compared with a model threshold. When the complexity score is greater than the model threshold, the cloud service is requested for assistance. When the complexity score is less than the model threshold, the context feature vector is sent to the local inference service for direct processing.
8. The method according to claim 7, wherein: Selecting to use a local inference service or requesting cloud services for assistance based on the complexity of the context feature vector includes: The local inference service includes a base layer large model accelerated by the NPU; The cloud service includes an enhanced layer large model. In response to the assistance request, the cloud service sends the enhanced layer large model parameters to the local inference model. The local inference model receives the enhanced layer large model parameters and performs local optimization before inferring the context feature vector.
9. The method according to claim 8, wherein The completion suggestions returned by the local inference service are displayed in the code editing interface, including: When the user adopts the completion suggestion, the client sends the key context feature vector after differential privacy processing to the federated learning server. The federated learning server continuously updates the enhancement layer large model in the cloud based on the feedback uploaded by the client through the parameter aggregation service.
10. A code assistance system based on cursor context perception and multimodal AI linkage, characterized in that: The system includes: A monitoring module is used to monitor the cursor position and record the cursor trajectory in the code editing interface, and dynamically construct the first AST syntax tree according to the current cursor position; Scanning module, used to scan the project structure and build a cross-file reference graph; A generation module, configured to construct a context feature vector based on the cursor track, the first AST syntax tree, and the cross-file reference graph; A routing module, configured to select, based on the complexity of the context feature vector, a local inference service or a cloud service for assistance; A display module, configured to display completion suggestions returned by the local inference model in a code editing interface; As well as local inference services and cloud services.
11. A computing device comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein: When the processor executes the instructions, the steps of the method according to any one of claims 1 to 9 are implemented.
12. A computer-readable storage medium storing computer instructions, characterized in that: When the instruction is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.