Efficiency improvement evaluation method and device of code assistant, equipment and medium

Abstract

The invention discloses an efficiency improvement evaluation method and device for a code assistant, equipment and a medium. The efficiency improvement evaluation method comprises the steps that a problem sequence input by a developer is received through the code assistant; counting performance index data corresponding to different service functions in the code assistant according to a feedback result of the code assistant for the problem sequence; and determining an efficiency improvement evaluation result corresponding to the code assistant according to the performance index data corresponding to different service functions in the code assistant. According to the technical scheme, the overall efficiency improvement contribution of the code assistant for the development activity can be accurately evaluated, and enterprises are guided to deploy research and development cost.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, equipment and medium for evaluating the efficiency improvement of a code assistant. Background Art

[0002] Code assistant is a tool that uses artificial intelligence technology to help developers write code more efficiently and accurately. It greatly improves the efficiency and quality of developers' coding by understanding the context of the code, providing code completion, code optimization suggestions, error detection and other functions. Evaluating the specific contribution of code assistants in actual development activities and determining their overall impact on improving R&D efficiency can help companies identify the key links that should be focused on in the promotion of code assistants and improve the overall effectiveness of the R&D team.

[0003] In the prior art, when evaluating the contribution of code assistants to improving efficiency in development activities, usually only a single performance indicator of the code assistant is counted, such as the effective code generation rate, the code entry rate, etc. There is a lack of technical means to evaluate the contribution of code assistants to improving the overall efficiency of development activities. It is impossible to intuitively judge the impact of code assistants on improving the efficiency of development activities, and it is impossible to guide enterprises to allocate R&D costs. Summary of the invention

[0004] The present invention provides a method, device, equipment and medium for evaluating the efficiency improvement of a code assistant, which can accurately evaluate the contribution of the code assistant to the overall efficiency improvement of development activities and guide enterprises to allocate R&D costs.

[0005] According to one aspect of the present invention, a method for evaluating the efficiency improvement of a code assistant is provided, the method comprising:

[0006] Receive a sequence of questions input by the developer through the code assistant;

[0007] According to the feedback results of the code assistant for the problem sequence, statistics are collected on performance indicator data corresponding to different business functions in the code assistant;

[0008] According to the performance indicator data corresponding to different business functions in the code assistant, the efficiency improvement evaluation result corresponding to the code assistant is determined.

[0009] Optionally, before the question sequence that receives developer input via the code assistant, also include:

[0010] Using the internal project files corresponding to the code assistant as a corpus for the Retrieval-augmented Generation (RAG) technology;

[0011] Before collecting statistics on performance indicator data corresponding to different business functions in the code assistant according to the feedback results of the code assistant for the question sequence, the method further includes:

[0012] The RAG technology is used to generate feedback results of the code assistant for the question sequence based on the corpus.

[0013] Optionally, statistics are collected on performance indicator data corresponding to different business functions in the code assistant, including:

[0014] Statistics are collected on the usage rate of different business functions in the code assistant, the proportion of the functions in the daily work of developers, and the conversion rate.

[0015] Optionally, determining the efficiency improvement evaluation result corresponding to the code assistant according to the performance indicator data corresponding to different business functions in the code assistant includes:

[0016] Multiply the personnel utilization rate, the proportion of daily work of developers, and the conversion rate corresponding to each business function in the code assistant to obtain the multiplication result of the indicators corresponding to each business function;

[0017] The multiplication results of the indicators corresponding to all business functions in the code assistant are accumulated to obtain the efficiency improvement evaluation result corresponding to the code assistant.

[0018] Optionally, the method further includes:

[0019] The code assistant uses a network security protocol (Hypertext Transfer Protocol Secure, HTTPS) to transmit data.

[0020] Optionally, statistics are collected on performance indicator data corresponding to different business functions in the code assistant, and efficiency improvement evaluation results corresponding to the code assistant are determined according to the performance indicator data corresponding to different business functions in the code assistant, including:

[0021] According to the preset evaluation cycle, regularly collect statistics on performance indicator data corresponding to different business functions in the code assistant;

[0022] According to the performance indicator data of different business functions in the code assistant during the evaluation period, the efficiency improvement evaluation result corresponding to the code assistant during the evaluation period is determined.

[0023] According to another aspect of the present invention, a device for evaluating efficiency improvement of a code assistant is provided, the device comprising:

[0024] A data receiving module, used for receiving a question sequence input by a developer through a code assistant;

[0025] An indicator statistics module, used for counting performance indicator data corresponding to different business functions in the code assistant according to the feedback results of the code assistant for the problem sequence;

[0026] The evaluation module is used to determine the efficiency improvement evaluation result corresponding to the code assistant according to the performance indicator data corresponding to different business functions in the code assistant.

[0027] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0028] at least one processor; and

[0029] a memory communicatively connected to the at least one processor; wherein,

[0030] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the efficiency improvement evaluation method of the code assistant described in any embodiment of the present invention.

[0031] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the code assistant efficiency improvement evaluation method described in any embodiment of the present invention when executed.

[0032] According to another aspect of the present invention, a computer program product is provided, including a computer program, which, when executed by a processor, implements the method for evaluating the efficiency improvement of a code assistant according to any embodiment of the present invention.

[0033] The technical solution provided by the embodiment of the present invention receives a question sequence input by a developer through a code assistant, and based on the feedback results of the code assistant for the question sequence, counts the performance indicator data corresponding to different business functions in the code assistant, and determines the efficiency improvement evaluation results corresponding to the code assistant based on the performance indicator data corresponding to different business functions in the code assistant. It provides a method for evaluating the contribution of the code assistant to the overall efficiency improvement of development activities, can accurately evaluate the contribution of the code assistant to the overall efficiency improvement of development activities, and guide enterprises to allocate R&D costs.

[0034] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 is a flow chart of a method for evaluating efficiency improvement of a code assistant provided according to an embodiment of the present invention;

[0037] Figure 2 is a flowchart of another method for evaluating efficiency improvement of a code assistant provided according to an embodiment of the present invention;

[0038] Figure 3 is a structural schematic diagram of a device for evaluating efficiency improvement of a code assistant provided according to an embodiment of the present invention;

[0039] Figure 4 It is a structural schematic diagram of an electronic device for implementing the method for evaluating the efficiency improvement of a code assistant according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0041] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0042] Figure 1This is a flowchart of a method for evaluating the efficiency improvement of a code assistant provided by an embodiment of the present invention. This embodiment can be applied to evaluating the contribution of a code assistant to improving the overall efficiency of development activities. The method can be executed by a device for evaluating the efficiency improvement of a code assistant. The device for evaluating the efficiency improvement of a code assistant can be implemented in the form of hardware and / or software. The device for evaluating the efficiency improvement of a code assistant can be configured in an electronic device. Figure 1 As shown, the method includes:

[0043] Step 110: Receive the question sequence input by the developer through the code assistant.

[0044] In this embodiment, the background service corresponding to the code assistant can be one or more large language models, and the code assistant can implement core business functions such as code interpretation, code completion, code annotation, unit testing, knowledge questions and answers, and code error correction based on the large language model. The front-end service corresponding to the code assistant can be a plug-in for accessing the integrated development environment (IntelliJ IDEA), and the plug-in can be linked to the background service through pages and keyboard shortcuts to provide the carrier corresponding to the above core business functions.

[0045] In this step, the developer can enter a sequence of questions into the code assistant, such as "help me generate a random number generation code...", and the code assistant can provide the developer with feedback results corresponding to the sequence of questions through the business function. The business function can provide data service capabilities such as API, the number of times used, and detailed data on the effect, to facilitate subsequent statistics.

[0046] Step 120: According to the feedback result of the code assistant for the question sequence, statistics are collected on performance indicator data corresponding to different business functions in the code assistant.

[0047] In this step, specifically, performance indicator data of different business functions in the code assistant, such as the number of times used, detailed effect data, etc., can be counted according to the feedback results of the code assistant, and this embodiment does not limit this.

[0048] Step 130: Determine the efficiency improvement evaluation result corresponding to the code assistant according to the performance indicator data corresponding to different business functions in the code assistant.

[0049] In this embodiment, the overall efficiency improvement contribution of the code assistant to development activities (ie, the efficiency improvement evaluation result) can be determined by combining the performance indicator data of various business functions used by developers in the code assistant.

[0050] The technical solution provided by the embodiment of the present invention receives a question sequence input by a developer through a code assistant, and based on the feedback results of the code assistant for the question sequence, counts the performance indicator data corresponding to different business functions in the code assistant, and determines the efficiency improvement evaluation results corresponding to the code assistant based on the performance indicator data corresponding to different business functions in the code assistant. It provides a way to evaluate the contribution of the code assistant to the overall efficiency improvement of development activities, so that the development team can take targeted measures to optimize the application of the code assistant and comprehensively improve the efficiency of R&D work; it can enable enterprises to more clearly understand and compare the advantages and disadvantages of each code assistant, and provide a scientific basis for selecting a code assistant solution suitable for the R&D team; it can help enterprises to conduct pre-production analysis when promoting the establishment of internal code assistant projects. By evaluating the expected efficiency improvement and return on investment, enterprises can make more informed decisions to ensure effective investment of resources and maximize output; it can support enterprises to continuously monitor and analyze the use effect of code assistants, thereby continuously optimizing the R&D process; by evaluating the use of code assistants by different teams, enterprises can promote cross-team collaboration and knowledge sharing, thereby accelerating the improvement of R&D efficiency of the entire organization.

[0051] Figure 2 A flowchart of another method for evaluating the efficiency improvement of a code assistant provided by an embodiment of the present invention is shown in FIG. Figure 2 As shown, the method includes:

[0052] Step 210: Use the internal project file corresponding to the code assistant as the corpus of the RAG technology.

[0053] In this embodiment, RAG technology combines language models and information retrieval technology. When the model corresponding to the code assistant needs to generate text or answer questions, RAG technology can be used to first retrieve relevant information from a large document collection, and then use this information to guide text generation, thereby improving the accuracy of the prediction results.

[0054] Existing code assistants are not sufficient to understand the developer's local code repository, so the question-and-answer results and completion effects provided are subject to certain limitations. In order to improve the effectiveness of the code assistant evaluation results, this embodiment can use each internal project file and code corresponding to the code assistant as the corpus of the RAG technology.

[0055] The advantage of this setting is that it allows the code assistant to better understand the internal logic of the project based on the local corpus and provide more accurate question-answering and code generation capabilities.

[0056] Step 220: Receive the question sequence input by the developer through the code assistant.

[0057] Step 230: Generate feedback results of the code assistant for the question sequence based on the corpus using the RAG technology.

[0058] Step 240: According to the feedback results of the code assistant for the problem sequence, statistics are collected on the personnel usage rate, the proportion of the daily work of developers, and the conversion rate corresponding to different business functions in the code assistant.

[0059] Step 250: Multiply the personnel utilization rate, the proportion of the daily work of the developers, and the conversion rate corresponding to each business function in the code assistant to obtain the multiplication result of the indicators corresponding to each business function.

[0060] In this embodiment, if a certain business function in the code assistant effectively improves the R&D efficiency of developers after being used, it can be considered that the business function has achieved capability conversion and has a certain conversion rate.

[0061] Step 260: Add up the multiplication results of the indicators corresponding to all the business functions in the code assistant to obtain the efficiency improvement evaluation result corresponding to the code assistant.

[0062] In a specific embodiment, it is assumed that the staff usage rate corresponding to the business function i in the code assistant is P i , which accounts for S% of the daily work of developers i , the conversion rate is R i , then the efficiency improvement evaluation result E corresponding to the code assistant can be determined by the following formula:

[0063]

[0064] The advantage of this setting is that by combining the multi-dimensional performance indicator data of the business functions in the code assistant, it is possible to accurately evaluate the contribution of the code assistant to the overall efficiency improvement of development activities and guide enterprises to allocate R&D costs.

[0065] In one implementation of this embodiment, the code assistant uses the network security protocol HTTPS to transmit data, wherein the HTTPS protocol can provide secure data transmission by encrypting the communication between the client and the server.

[0066] The advantage of this setting is that by using the HTTPS protocol to encrypt the transmitted content, even if the data is intercepted during transmission, the attacker cannot directly read the data content; HTTPS uses digital certificates to verify the identity of the server to ensure that the client is connected to an authenticated server, not a fake server, thereby avoiding man-in-the-middle attacks; the HTTPS protocol uses a message integrity check mechanism to ensure that the data has not been tampered with during transmission. If the data packet is tampered with during transmission, the receiver can detect the change, thereby ensuring the integrity of the data transmission results; due to the encryption characteristics of HTTPS communication, it is difficult for attackers to take over the communication session between the user and the server by means of session hijacking.

[0067] In a specific embodiment, performance indicator data corresponding to different business functions in the code assistant are counted, and based on the performance indicator data corresponding to different business functions in the code assistant, an efficiency improvement evaluation result corresponding to the code assistant is determined, including: according to a preset evaluation period, regularly counting the performance indicator data corresponding to different business functions in the code assistant; and determining the efficiency improvement evaluation result corresponding to the code assistant within the evaluation period based on the performance indicator data of different business functions in the code assistant within the evaluation period.

[0068] Optionally, the efficiency improvement contribution of the code assistant can be evaluated according to the actual application of the code assistant within the enterprise and according to a preset evaluation cycle. For example, the efficiency improvement contribution of the code assistant can be evaluated according to the statistical cycle of day, week, month, quarter, half year, year, etc.

[0069] The technical solution provided by the embodiment of the present invention uses the internal project file corresponding to the code assistant as the corpus of the RAG technology, receives the question sequence input by the developer through the code assistant, generates the feedback results of the code assistant for the question sequence according to the corpus through the RAG technology, counts the personnel utilization rate, the proportion of the daily work of the developer and the conversion rate corresponding to different business functions in the code assistant, multiplies the personnel utilization rate, the proportion of the daily work of the developer and the conversion rate corresponding to each business function in the code assistant, and accumulates the multiplication results of the indicators corresponding to all business functions to obtain the efficiency improvement evaluation result. This technical means can accurately evaluate the overall efficiency improvement contribution of the code assistant to development activities and guide enterprises to allocate R&D costs.

[0070] Based on the above embodiment, this embodiment takes the following three scenarios as examples to evaluate the contribution of the code assistant to improving the overall efficiency of development activities:

[0071] Scenario 1: All employees use the technical Q&A and code completion functions of the code assistant, that is, the number of technical Q&A and code completion users is 100%, and the coverage rate is 100%. In this scenario, the efficiency improvement evaluation results of the code assistant can be shown in Table 1:

[0072] Table 1

[0073]

[0074] Scenario 2: All employees use all functions of the code assistant, that is, the coverage rate of each function is 100%. In this scenario, the efficiency improvement evaluation results of the code assistant can be shown in Table 2:

[0075] Table 2

[0076]

[0077] Scenario 3: All employees use all functions of the code assistant, and the acceptance rate of the code completion function increases to 20%. In this scenario, the efficiency improvement evaluation results of the code assistant can be shown in Table 3:

[0078] Table 3

[0079]

[0080] Figure 3 A schematic diagram of a device for evaluating the efficiency improvement of a code assistant provided by an embodiment of the present invention is provided. The device is applied to an electronic device, such as Figure 3 As shown, the device includes: a data receiving module 310, an indicator statistics module 320 and an evaluation module 330.

[0081] The data receiving module 310 is used to receive a question sequence input by a developer through a code assistant;

[0082] The indicator statistics module 320 is used to count the performance indicator data corresponding to different business functions in the code assistant according to the feedback results of the code assistant for the problem sequence;

[0083] The evaluation module 330 is used to determine the efficiency improvement evaluation result corresponding to the code assistant according to the performance indicator data corresponding to different business functions in the code assistant.

[0084] The technical solution provided by the embodiment of the present invention receives a question sequence input by a developer through a code assistant, and based on the feedback results of the code assistant for the question sequence, counts the performance indicator data corresponding to different business functions in the code assistant, and determines the efficiency improvement evaluation results corresponding to the code assistant based on the performance indicator data corresponding to different business functions in the code assistant. It provides a way to evaluate the contribution of the code assistant to the overall efficiency improvement of development activities, so that the development team can take targeted measures to optimize the application of the code assistant and comprehensively improve the efficiency of R&D work; it can enable enterprises to more clearly understand and compare the advantages and disadvantages of each code assistant, and provide a scientific basis for selecting a code assistant solution suitable for the R&D team; it can help enterprises to conduct pre-production analysis when promoting the establishment of internal code assistant projects. By evaluating the expected efficiency improvement and return on investment, enterprises can make more informed decisions to ensure effective investment of resources and maximize output; it can support enterprises to continuously monitor and analyze the use effect of code assistants, thereby continuously optimizing the R&D process; by evaluating the use of code assistants by different teams, enterprises can promote cross-team collaboration and knowledge sharing, thereby accelerating the improvement of R&D efficiency of the entire organization.

[0085] Based on the above embodiment, the code assistant uses the network security protocol HTTPS to transmit data.

[0086] The device also includes:

[0087] A corpus determination module, used to use the internal project file corresponding to the code assistant as a corpus for the retrieval enhancement generation RAG technology;

[0088] A feedback result generating module, used for generating feedback results of the code assistant for the question sequence according to the corpus by using the RAG technology;

[0089] The timing evaluation module is used to regularly count the performance indicator data corresponding to different business functions in the code assistant according to a preset evaluation period; and determine the efficiency improvement evaluation result corresponding to the code assistant in the evaluation period according to the performance indicator data of different business functions in the code assistant in the evaluation period.

[0090] The indicator statistics module 320 includes:

[0091] The statistical unit is used to count the personnel usage rate corresponding to different business functions in the code assistant, the proportion of daily work of developers, and the conversion rate.

[0092] The evaluation module 330 includes:

[0093] A multiplication unit, used to multiply the personnel utilization rate, the proportion of the daily work of the developers and the conversion rate corresponding to each business function in the code assistant to obtain the multiplication result of the indicators corresponding to each business function;

[0094] The accumulation unit is used to accumulate the multiplication results of the indicators corresponding to all business functions in the code assistant to obtain the efficiency improvement evaluation result corresponding to the code assistant.

[0095] The above device can execute the methods provided by all the above embodiments of the present invention, and has the corresponding functional modules and beneficial effects of executing the above methods. For technical details not fully described in the embodiments of the present invention, please refer to the methods provided by all the above embodiments of the present invention.

[0096] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0097] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0098] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0099] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the efficiency improvement evaluation method of the code assistant.

[0100] In some embodiments, the efficiency improvement evaluation method of the code assistant may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the efficiency improvement evaluation method of the code assistant described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the efficiency improvement evaluation method of the code assistant in any other appropriate manner (e.g., by means of firmware).

[0101] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0102] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0103] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0104] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0105] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0106] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0107] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0108] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for evaluating the efficiency improvement of a code assistant, characterized in that: The method comprises: Receive a sequence of questions input by the developer through the code assistant; According to the feedback results of the code assistant for the problem sequence, statistics are collected on performance indicator data corresponding to different business functions in the code assistant; According to the performance indicator data corresponding to different business functions in the code assistant, the efficiency improvement evaluation result corresponding to the code assistant is determined.

2. The method according to claim 1, characterized in that Before the sequence of questions that receive developer input via the code assistant, it also includes: Using the internal project files corresponding to the code assistant as a corpus for the retrieval enhancement generation RAG technology; Before collecting statistics of performance indicator data corresponding to different business functions in the code assistant according to the feedback results of the code assistant for the question sequence, the method further includes: The RAG technology is used to generate feedback results of the code assistant for the question sequence based on the corpus.

3. The method according to claim 1, characterized in that Statistics of performance indicator data corresponding to different business functions in the code assistant include: Statistics are collected on the usage rate of different business functions in the code assistant, the proportion of the functions in the daily work of developers, and the conversion rate.

4. The method according to claim 3, characterized in that Determining the efficiency improvement evaluation result corresponding to the code assistant according to the performance indicator data corresponding to different business functions in the code assistant includes: Multiply the personnel utilization rate, the proportion of daily work of developers, and the conversion rate corresponding to each business function in the code assistant to obtain the multiplication result of the indicators corresponding to each business function; The multiplication results of the indicators corresponding to all business functions in the code assistant are accumulated to obtain the efficiency improvement evaluation result corresponding to the code assistant.

5. The method according to claim 1, characterized in that The method further comprises: The code assistant uses the network security protocol HTTPS to transmit data.

6. The method according to claim 1, characterized in that Collecting statistics on performance indicator data corresponding to different business functions in the code assistant, and determining the efficiency improvement evaluation result corresponding to the code assistant according to the performance indicator data corresponding to different business functions in the code assistant, including: According to the preset evaluation cycle, regularly collect statistics on performance indicator data corresponding to different business functions in the code assistant; According to the performance indicator data of different business functions in the code assistant during the evaluation period, the efficiency improvement evaluation result corresponding to the code assistant during the evaluation period is determined.

7. A device for evaluating the efficiency improvement of a code assistant, characterized in that: The device comprises: A data receiving module, used for receiving a question sequence input by a developer through a code assistant; An indicator statistics module, used for counting performance indicator data corresponding to different business functions in the code assistant according to the feedback results of the code assistant for the problem sequence; The evaluation module is used to determine the efficiency improvement evaluation result corresponding to the code assistant according to the performance indicator data corresponding to different business functions in the code assistant.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the efficiency improvement evaluation method of the code assistant described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the efficiency improvement evaluation method of a code assistant according to any one of claims 1 to 6 when executed.

10. A computer program product, comprising a computer program, which, when executed by a processor, implements the method for evaluating the efficiency improvement of a code assistant according to any one of claims 1 to 6.

Images