A code generation detection method, apparatus, medium, and device

Abstract

The embodiment of the specification discloses a code generation detection method, which acquires a verification tool library containing at least an implicit call detection tool to support detection. The code samples in the code dataset are identified, the code fragments calling resources are determined, and the corresponding detection tools are determined from the verification tool library. The detection large model is fine-tuned by taking the code samples as training samples, the code fragments and the corresponding detection tools as labels, so that when facing code generation detection, the detection large model can identify the calling tool fragments in the code and the corresponding detection tools to perform code detection through the detection tools. The implicit dependent resource exception that cannot be covered by the rule matching method can be effectively detected, and the test case does not need to be manually written, the detection efficiency is improved, and the resource overhead is reduced.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a method, apparatus, storage medium and device for generating and detecting codes. Background Technology

[0002] With the development of Artificial Intelligence (AI) technology, Large Language Models (LLMs) have been widely applied in various fields. In the field of code generation, LLMs can provide developers with powerful automated programming support, significantly improving software development efficiency.

[0003] However, when generating code, LLMs often produce code that is syntactically correct and semantically seemingly reasonable, yet fails to execute as expected or meet specified requirements. One scenario where code execution fails due to calls to external resources is known as a "resource illusion." For example, the called file, library, or interface may not exist, the path may be incorrect, or permissions may be insufficient; these are commonly referred to as "resource illusions."

[0004] In existing technologies, static code analysis tools can be used to parse the code's syntax structure and match it with preset resource metadata to detect whether the code calls resources abnormally. However, for implicitly dependent resource calls, such as dynamic file paths concatenated by strings, it is impossible to verify whether the resource is abnormal when the code runs. Detection through test cases not only requires manually writing test cases, but also suffers from being time-consuming, costly, and having complex environment configurations, making it difficult to apply in large-scale code generation scenarios.

[0005] Therefore, how to efficiently and accurately detect whether the code generated by a large model calls resources abnormally has become an urgent problem to be solved. Thus, this manual provides a method for detecting generated code. Summary of the Invention

[0006] This specification provides a method, apparatus, storage medium, and electronic device for generating code detection, in order to partially solve the problems existing in the prior art.

[0007] The embodiments in this specification adopt the following technical solutions: This specification provides a method for generating code detection, the method comprising: A verification tool library is determined, which includes at least an implicit call detection tool. The implicit call detection tool constructs a resource access request in response to a resource access statement to perform resource call verification. Obtain a code dataset, identify each code sample in the code dataset, determine the code fragments that call resources in each code sample, and determine the detection tools corresponding to the code fragments from the verification tool library; Using the code sample as the training sample, and the code fragments identified in the code sample and the corresponding detection tools as annotations, a large detection model is trained; In response to a code detection request, the code to be detected is input into a trained detection model to determine the code segments that call resources in the code to be detected and the corresponding detection tools. Based on the verification results of the code segments by the corresponding detection tools, the detection result of the code to be detected is determined.

[0008] This specification provides a code generation detection device, the device comprising: A tool building module is used to determine a verification tool library, which includes at least an implicit call detection tool. The implicit call detection tool responds to a resource access statement and constructs a resource access request to perform resource call verification. The sample construction module is used to acquire a code dataset, identify each code sample in the code dataset, determine the code fragments that call resources in each code sample, and determine the detection tools corresponding to the code fragments from the verification tool library; The training module is used to train a large detection model using the code sample as the training sample, and the code fragments identified in the code sample and the corresponding detection tools as annotations. The detection module is used to respond to code detection requests by inputting the code to be detected into the trained detection model, determining the code segments that call resources in the code to be detected and the corresponding detection tools, and determining the detection result of the code to be detected based on the verification results of the corresponding detection tools on the code segments.

[0009] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described code generation detection method.

[0010] This specification provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-described code generation detection method.

[0011] The above-described at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects: This specification discloses a method for detecting generated code. The method acquires a verification tool library containing at least implicit dependency detection tools to support detection. It identifies code samples in a code dataset, determines the code fragments that call resources, and identifies the corresponding detection tools from the verification tool library. Using code samples as training samples and code fragments and their corresponding detection tools as annotations, a large-scale detection model is fine-tuned. Thus, when facing generated code detection, the large-scale detection model can identify the tool-calling fragments and their corresponding detection tools in the code, enabling code detection through these tools. This method effectively detects implicit dependency resource anomalies that rule-matching methods cannot cover, and eliminates the need for manually writing test cases, improving detection efficiency and reducing resource overhead. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings: Figure 1 This specification provides a flowchart of a code generation detection process as an embodiment. Figure 2 This is a schematic diagram of the code detection process provided in the embodiments of this specification; Figure 3 This is a schematic diagram of a code generation detection device provided in an embodiment of this specification; Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this specification. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0014] Large Language Models (LLMs), also known simply as large models, are models that perform logical reasoning on input natural language text and output user-understandable natural language descriptions. In the field of code generation, in addition to checking whether the logic of the code itself is correct, it is also necessary to check whether the code actually runs correctly. One aspect of this is detecting code execution failures caused by calling external resources.

[0015] Existing methods for detecting resource illusions during code generation typically employ static rule-based code analysis tools for initial screening. These tools parse the code syntax structure, extract explicitly declared resource paths, and match them with predefined resource metadata to determine whether the called files, libraries, or interfaces exist or meet version compatibility requirements.

[0016] However, the limitation of static rules is that they can only utilize resource identifiers explicitly written in the code. For implicit resource paths that are dynamically generated through string concatenation, environment variable reading, or configuration file references, they cannot restore the actual resource pointer at runtime, and therefore cannot verify whether the resource exists or is accessible in the execution environment.

[0017] Another method relies on test cases for detection, which involves executing code in an isolated environment (such as a Docker container or sandbox) and judging resource validity by capturing runtime resource access exceptions. While this method can cover implicit dependency scenarios and has high detection accuracy, it suffers from low execution efficiency, complex environment configuration, and typically requires manually writing targeted test cases or simulating data. Furthermore, the completeness of test cases heavily depends on human experience, making it difficult to guarantee comprehensive coverage of various resource exception scenarios. Additionally, the resource overhead of this test is significant, making it unsuitable for batch processing scenarios involving large-scale code generation.

[0018] In summary, current solutions to the code resource illusion problem face a dual dilemma: static detection cannot cover implicit dependencies, while dynamic testing is costly and difficult to scale. How to achieve efficient identification of implicit resource dependencies while ensuring detection accuracy has become a pressing technical problem. This document provides a method for detecting resource calls in generated code based on the reasoning capabilities of a large model.

[0019] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0020] Figure 1 A code generation detection flowchart provided in this specification includes the following steps: S100: Determine a verification tool library, which includes at least an implicit call detection tool. The implicit call detection tool responds to a resource access statement by constructing a resource access request to perform resource call verification.

[0021] In the embodiments described in this specification, the following are employed: Figure 1 The device used for code detection in large model generation, as illustrated, can be any electronic device, such as a computer, server, or server cluster consisting of multiple servers. For ease of description, the following explanation uses a server as an example.

[0022] In the embodiments described in this specification, the detection tool is a pre-defined functional module used to verify resource calls in code. A common detection tool is a functional module that automatically performs resource validity verification on resource access statements in code snippets within a simulation testing environment.

[0023] Specifically, the server needs to acquire a verification toolkit, which must contain at least an implicit call detection tool. This toolkit can be built upon existing detection tools used in simulation environment testing. Since test cases can be used to verify the authenticity and validity of resources by executing resource access requests, and detection tools are the concrete carriers for implementing this testing, to avoid the limitations of static rules, the server needs to acquire detection tools capable of executing tests. The actual verification capabilities of these tools can then compensate for the detection blind spots that static rules cannot cover.

[0024] Furthermore, to address implicit calls that are difficult to cover using static rules, in this embodiment of the specification, the server may specifically designate an implicit call detection tool as a detection tool in the verification tool library. This implicit call detection tool responds to resource access statements by constructing resource access requests to perform resource call verification.

[0025] Furthermore, in the embodiments of this specification, since there are different types of implicit calls, different implicit call detection tools can be determined for different types of servers.

[0026] Specifically, to address the issue of dynamically generated resource paths in code through string concatenation, environment variable reading, or configuration file referencing, the resource location and detection tool simulates the execution of path generation logic. It recreates dynamic paths and verifies resource existence in a simulated testing environment, thus resolving the problems associated with static rules.

[0027] This resource location detection tool receives code snippets that call resources, parses the resource access statements within those snippets, and extracts various elements used to generate the path. For example, a code snippet might contain the statement `os.path.join(base_dir, filename)`, where `base_dir` might come from the environment variable `DATA_ROOT`, and `filename` might come from user input or a configuration file. The resource location detection tool simulates the path concatenation logic of this statement to reconstruct the expected complete resource path string at runtime.

[0028] This resource location detection tool can reconstruct paths by accessing various preset contextual information within the simulation testing environment. For example, in scenarios dependent on environment variables, the tool reads pre-configured environment variable values ​​from the simulation testing environment and substitutes them into the path expression. In scenarios dependent on configuration files, the tool reads the contents of the configuration files deployed in the simulation testing environment and extracts the corresponding configuration items.

[0029] Finally, the resource location detection tool constructs a resource access request based on the obtained complete resource path. The format of this request is consistent with the expected resource access method in the code. This request is then executed in a simulation test environment. During execution, the server generates a unique log trace identifier for this verification operation to record the detailed results of the resource access.

[0030] If the resource access request successfully returns the expected content, the implicit resource call is considered valid, and the resource actually exists at runtime; for example, a file is successfully opened or a directory existence check returns a true value. If the resource access request returns exception information such as a non-existent file, an incorrect path, or an access timeout, the implicit resource call is considered to be a resource illusion.

[0031] Furthermore, to address the issue of resources with valid paths but inaccessible due to insufficient permissions, a permission detection tool is provided. This tool simulates resource access with preset permissions in a simulation testing environment to verify resource accessibility.

[0032] Specifically, the permission detection tool also receives code snippets sent by the server, and first determines the expected access mode of the resource call based on the context of the code snippet, such as read operation, write operation or execution operation.

[0033] The permission verification tool then determines the simulated permissions used for verification based on the preset permission configurations in the simulation test environment. This simulation test environment typically has at least one pre-configured permission setting, and more commonly, multiple levels and their corresponding permission sets are configured. For example, ordinary user, administrator, or operations and maintenance accounts; the server can select the appropriate permission configuration for verification based on the typical execution scenario of the code. Simulated permissions refer to the permissions selected by the permission verification tool for verification based on the access patterns of the code snippet and the permission configurations in the simulation test environment.

[0034] Then, after determining the simulated permissions, the permission verification tool attempts to access the resources referenced in the code snippet within the simulation test environment using those simulated permissions. For example, for the resource access statement `open(' / data / share / config.json', 'r')` in the code snippet, the permission verification tool initiates a read operation on the file as a pre-defined ordinary user. The permission verification tool does not directly execute the original code; instead, it constructs a standard resource access request based on the access intent expressed by the code and executes the request with the specified user permissions. During execution, the server generates a unique log trace identifier for this verification operation to record the detailed results of the permission access.

[0035] Finally, after the resource access request is executed, the permission detection tool determines the accessibility of the resource based on the execution result. If the resource access request successfully returns the expected content, such as a file being successfully opened and its content read, it is determined that the resource can be accessed normally under the current permissions. If the resource access request returns an access denied exception due to insufficient permissions, it is determined that there is a permission-related resource illusion.

[0036] It should be noted that in scenarios where access fails due to insufficient permissions, even if the resource itself exists in the location specified in the path, the code will still fail to execute normally during actual runtime due to permission exceptions. Therefore, permission detection tools identify this as a manifestation of resource illusion.

[0037] Furthermore, in one or more embodiments of this specification, to address the issue that external interfaces may become inaccessible during code runtime due to service outages, address changes, or the need for temporary authentication, this specification provides an interface testing tool. This tool verifies the real-time reachability, authentication requirements, and response validity of the interface by sending test requests in a simulation testing environment.

[0038] Specifically, the interface testing tool also receives code snippets sent by the server, parses the code snippets, and extracts the information required for the interface call, including the request method, interface address, request header parameters, and the request body data carried.

[0039] For example, in the code snippet requests.get('https: / / api.weather.com / current', params={'city': 'Beijing'}), the API testing tool can extract that the request method is GET, the API address is https: / / api.weather.com / current, and the request parameter is the city name.

[0040] Next, the interface testing tool constructs a standard test request based on the parsed results to simulate the actual runtime call method of the code, including the same request method, request headers, query parameters, and request body content. For request parameters that may be dynamically generated in the code, such as query conditions concatenated by variables, the interface testing tool obtains the specific values ​​from the resource call parameters output by the large testing model and fills them into the request. After construction is complete, the interface testing tool actually sends the test request to the target interface in the simulation test environment.

[0041] Finally, after the test request is sent, the status of the interface is determined based on the returned response information. If the request successfully returns the expected status code and response content, the interface is confirmed to be currently accessible. If a 401 or 403 status code is returned, the interface requires valid authentication to access, and the simulated request was rejected due to a lack of a valid key. If a 404 status code or connection timeout is returned, the interface is confirmed to be offline or currently unreachable by the network. The server generates a unique log tracer for the verification operation to record the detailed results of the interface request.

[0042] S102: Obtain the code dataset, identify each code sample in the code dataset, determine the code segment that calls the resource in each code sample, and determine the detection tool corresponding to the code segment from the verification tool library.

[0043] In one or more embodiments of this specification, in addition to tools for detection, the server also needs to have the ability to identify code segments from complete code and determine the corresponding detection tools for detection, so as to achieve detection of code calls without running the complete code. Therefore, the server can also build training samples to train a large detection model that achieves the above-mentioned functions.

[0044] Specifically, the server first acquires a collected dataset of open-source code, which contains several code samples. A code sample can be a complete source code file or an independent function or method implementation. Its key feature is that it can be processed as a whole by the large detection model, and it may contain one or more code snippets that call external resources.

[0045] For each code sample, the server traverses all its statements through syntax parsing or static analysis, identifying code snippets involving external resource calls. For example, for Python code samples, the server identifies statements containing keywords such as `open`, `os.path.exists`, `import`, and `requests.get`, extracting these statements into independent code snippets. During the identification process, the server can also record the type of resource operated on by each code snippet, including file resources, library resources, or network interface resources.

[0046] After extracting the code snippets, the server further assigns a corresponding detection tool to each snippet. Since a pre-defined verification tool library contains multiple pre-configured detection tools, the server can match the specific characteristics of resource calls within the code snippet to determine the appropriate detection tool. For example, for a code snippet that generates a file path through string concatenation, the server associates it with a resource location detection tool. After matching, the server records each code snippet and its corresponding detection tool identifier as annotation data for subsequent training of the large-scale detection model.

[0047] S104: Using the code sample as the training sample, and the code fragments identified in the code sample and the corresponding detection tools as annotations, train a large detection model.

[0048] In the embodiments described in this specification, after determining the training samples, the server can perform supervised fine-tuning based on the general large model to obtain a detection large model, thereby transforming the general large model into a detection large model for resource illusion detection. Through fine-tuning training, the detection large model can learn which statements need to trigger verification and which detection tools should be invoked.

[0049] Specifically, the server first constructs a training dataset containing multiple training samples. Each training sample consists of a code sample and corresponding annotation information. The annotation information includes the code snippets identified as resource calls within the code sample, as well as the detection tool identifier for each code snippet. Code samples that do not involve resource access statements do not require annotation.

[0050] Subsequently, the server trains the general-purpose large model using supervised fine-tuning. During training, the server feeds code samples as input to the general-purpose large model, i.e., the detection large model to be trained. The detection large model to be trained then outputs the inference results for that code sample, including the identified code fragments and their corresponding detection tool identifiers. Based on the output inference results and annotations, the server determines the loss function and updates the model parameters using the backpropagation algorithm. The goal of training is to make the code fragment positions and detection tool identifiers output by the model as close as possible to the labeled information.

[0051] S106: In response to the code detection request, input the code to be detected into the trained detection model, determine the code segment that calls resources in the code to be detected and the corresponding detection tool, and determine the detection result of the code to be detected based on the verification result of the corresponding detection tool on the code segment.

[0052] In the embodiments described in this specification, after the large-scale detection model is trained, the server can respond to code detection requests by using the large-scale detection model to identify code segments and their corresponding detection tools, and then perform detection using the detection tools. In other words, the code segments and detection tools output by the large-scale detection model are only the basis for subsequent verification. The server needs to combine this with a simulation testing environment and perform resource access operations to obtain the verification results.

[0053] Specifically, the server first responds to the code detection request and identifies the code to be detected. This code can be submitted by the user or generated by a large code generation model. The server inputs the code to be detected into the trained detection model, which then identifies the code line by line according to its original line order. Based on the semantic features learned during training, the detection model determines whether there is a code segment that calls external resources in each line of code. If so, it outputs the code segment and its corresponding detection tool identifier.

[0054] Furthermore, based on the output of the large detection model, the server sequentially verifies the resource calls of each identified code segment. For the currently identified code segment, the server calls the corresponding detection tool from the verification tool library and performs the verification operation in the simulation test environment.

[0055] For example, if the large detection model identifies a code snippet that generates a file path by concatenating strings and outputs a resource location detection tool identifier, the server will call the resource location detection tool to restore the path and verify its existence. If it identifies a code snippet that references an internal network configuration file and outputs a permission detection tool identifier, the server will call the permission detection tool to access the file with preset simulated permissions and verify its accessibility. If it identifies a resource access statement and outputs an interface detection tool identifier, the server will call the interface detection tool to send a test request to the interface and determine the interface status.

[0056] After each code snippet verification is completed, the server determines the subsequent process based on the verification result. If the verification result of the current code snippet is successful, it indicates that the resource call is normal. The server then continues to input the next line of code into the detection model, which identifies the subsequent code snippets and repeats the above verification process until all lines of code to be detected have been processed.

[0057] If the verification result of a code snippet fails, the server can stop subsequent detection operations and determine and return the detection result based on the information output by the detection model or detection tool. For example, based on the detection tool output by the detection model, the server can determine the type of error and then return the corresponding code snippet and error type. Alternatively, based on the output of the detection tool, the server can output the specific error content. For example, when a resource location detection tool verifies a code snippet, if the parsed path does not contain a corresponding file, it will return "resource not found" along with the code snippet.

[0058] Figure 2 The flowchart for the generation code detection provided in this specification describes the process as follows: In response to a code detection request, the code to be detected is determined; the code to be detected is input into a trained detection model, which identifies the code line by line; for each identified code segment, resource call verification is performed on the code segment according to the detection tool output by the detection model; the verification result is determined to be successful; if the verification result is successful, the next code segment is identified and the detection continues until the code to be detected is detected; if the verification result is unsuccessful, the detection stops and an error message is output.

[0059] based on Figure 1 The generated code detection method described herein acquires a verification tool library containing at least implicit dependency detection tools to support detection. It identifies code samples in the code dataset, pinpointing code fragments that call resources, and determines the corresponding detection tools from the verification tool library. Using code samples as training samples and code fragments and their corresponding detection tools as annotations, a large-scale detection model is fine-tuned. Thus, when facing generated code detection, the large-scale detection model can identify the tool-calling fragments and their corresponding detection tools in the code, enabling code detection through these tools. This method effectively detects implicit dependency resource anomalies that rule-matching methods cannot cover, and eliminates the need for manually written test cases, improving detection efficiency and reducing resource overhead.

[0060] In the embodiments of this specification, when acquiring the code dataset in step S102, the quality and diversity of the data samples affect the training effect and also determine the generalization ability of the detection model. If only code samples from a single programming language are used for training, the detection model may have difficulty recognizing resource call patterns that are semantically similar but syntactically different in other languages. For example, the file open statement in Python and the file input stream statement in Java have the same function but significantly different forms.

[0061] Furthermore, if the dataset contains a large number of ordinary code samples that do not involve external resource calls, it can also lead to low efficiency in building training samples. Therefore, this server can acquire code datasets covering multiple programming languages ​​and identifying those containing resource loading scenarios. This allows the large detection model to learn common patterns and language-specific keywords for calling external resources such as files, libraries, and interfaces in different language environments, thereby improving the detection efficiency of the large detection model.

[0062] Specifically, the server can first obtain code sets in at least two preset programming languages, which can be obtained from open-source code hosting platforms or publicly available code datasets. The specific preset programming languages ​​are configured as needed and are not limited in this specification. Examples include Python, Java, JavaScript, Go, and C++. Obtaining code in multiple languages ​​increases the application scenarios that the dataset can cover. External resources include three types: file resources, library resources, and interface resources. File resources refer to data files or configuration files accessed through file read / write operations; library resources refer to third-party code modules loaded through import or reference statements; and interface resources refer to external service interfaces accessed through HTTP requests or remote procedure calls.

[0063] After obtaining the code set, the server can filter each code sample in the set based on whether it contains resource access statements. The server can identify resource access statements in the code samples using methods such as keyword matching or syntax parsing. Of course, this specification does not restrict the specific filtering method used, as long as it identifies code samples that load external resources. The server iterates through the code samples in each code set to filter out code samples containing resource access statements.

[0064] The server then retains the selected code samples, discards the other code samples, and uses the selected code samples corresponding to each language to form the final code dataset.

[0065] Furthermore, in one or more embodiments of this specification, in step S106, when the server receives the detection tool output by the detection model as a resource location detection tool, it can also determine the location parameters also output by the detection model. Then, the server can perform detection based on the code snippet output by the detection model, using the resource location detection tool and the location parameters.

[0066] Specifically, after receiving the code snippet and location parameters output by the large detection model, the server calls the resource location detection tool in the verification tool library, passing the location parameters as input to the tool. This resource location detection tool can then construct a resource access path based on these location parameters in the test environment. Specifically, the tool first reads the preset environment variable values ​​in the simulation test environment based on the environment variable names in the location parameters; for example, it reads the value of PROJECT_ROOT as / home / user / project. Then, it concatenates the fixed path fragments to obtain / home / user / project / data. Finally, it combines the identifier of the variable name filename to obtain its specific value from the simulation context of the simulation test environment; for example, if filename is set to config.json, the resource location detection tool obtains the complete resource path / home / user / project / data / config.json through these steps.

[0067] Then, the resource location and detection tool continues to attempt to access the resource corresponding to the path in the test environment to verify whether the resource exists. For example, for file resources, it performs a file existence check; for directory resources, the tool performs a directory traversal operation.

[0068] During the above access process, the server can generate a unique log tracking identifier for the verification operation, which is used to record the path construction process and access results.

[0069] If the resource access is successful and returns the expected content, it is determined that the resource exists, and the server determines the verification result of the code snippet as normal.

[0070] If the resource access returns an error message such as "file does not exist," "path is incorrect," or "access timed out," it is determined that the resource does not exist. The server can then identify the verification result of the code snippet as an error and extract detailed information based on the log tracking identifier for output.

[0071] Furthermore, in one or more embodiments of this specification, in step S106, when the detection tool output by the detection large model is the permission detection tool, the detection large model also outputs permission verification parameters.

[0072] Specifically, after receiving the code snippet and permission verification parameters output by the large detection model, the server calls the permission detection tool in the verification tool library and inputs the permission verification parameters into the permission detection tool.

[0073] In the test environment, this permission detection tool selects the corresponding simulated permissions for resource access based on the received permission verification parameters. In the real test environment, various simulated user roles and their corresponding permission sets can be pre-configured. For example, a regular user role only has read and write permissions to the user directory, an administrator role has read permissions to the system configuration directory, and a service account role has access to specific service interfaces. The permission detection tool selects the appropriate simulated identity based on the user role identifier in the permission verification parameters and determines the specific operation type based on the access mode identifier.

[0074] Then, the permission detection tool, using this simulated permission, attempts to access the resources referenced in the code snippet in the test environment. For example, in a system configuration file reading scenario, the permission detection tool, acting as an administrator, initiates a file read operation on ` / etc / config / app.conf`. In a user data writing scenario, the permission detection tool, acting as a regular user, initiates a file write operation on `. / user_data / report.pdf`. During the access process, the server can also log the results of the permission access.

[0075] Finally, if the resource access request successfully returns the expected content, such as the file being successfully opened and its content read, it is determined that the resource can be accessed normally under the current simulated permissions, and the server determines the verification result of the code snippet as normal.

[0076] If a resource access request returns an access denied exception due to insufficient permissions—for example, an administrator role should be able to read a system configuration file but returns a permission error, or a regular user role fails to write to another user's directory—then it is determined that there is a permission problem with the resource. The server will then recognize the verification result of the code snippet as an exception and extract the specific error information from the logs and return it.

[0077] Furthermore, in one or more embodiments of this specification, in step S106, when the detection tool output by the large detection model is the interface detection tool, the large detection model also outputs interface call parameters.

[0078] Specifically, after receiving the code snippet and interface call parameters output by the large detection model, the server calls the interface detection tool in the verification tool library and inputs the interface call parameters into the interface detection tool.

[0079] The API testing tool constructs test requests in the test environment based on the received API call parameters. First, it builds a complete request URL based on the API address template and query parameter list. Then, it sets the request action according to the request method type and adds necessary header information based on the request header field set. For example, it specifies the content type or acceptable format and fills in simulated data according to the request body structure. For key placeholders, it can choose to obtain a test key from the test environment's key pool or use a preset test credential. For example, for a user creation API, it constructs a JSON request body containing the simulated username `test_user` and adds an `Authorization` field carrying the test token to the request header.

[0080] After the test request is constructed, the interface testing tool sends the request to the target external interface in the test environment. During the sending process, the server records the log content, sending time, and subsequent received response information.

[0081] Finally, wait for and receive the response from the external interface, and determine the status of the interface based on the response status code, response header, and response body content.

[0082] For example, if the response status code is 200 and the response body contains the expected data structure, the interface is determined to be accessible, and the server classifies the verification result of this code snippet as normal. If the response returns a 401 or 403 status code, the interface requires valid authentication, and the current test request is rejected due to a lack of a valid key. The server classifies the verification result of this code snippet as abnormal and includes an error type of "authentication required". If the response returns a 404 status code or a connection timeout, the interface is determined to be offline or its address has changed. The server classifies the verification result of this code snippet as abnormal and includes an error type of "interface does not exist".

[0083] Additionally, it should be noted that in scenarios where high-level knowledge programs are generated through large models, the main logic of the program reuses traditional programming languages ​​such as Python, while natural language is used to describe knowledge concepts, such as "reading yesterday's weather data" or "calling user profile services".

[0084] While this hybrid approach enhances the flexibility and cognitive empathy of expression, it also introduces implicit uncertainty in resource invocation. Resource paths, interfaces, or dependencies described in natural language often cannot be directly located through static syntax parsing, and their actual availability dynamically changes with the runtime environment. In this specification, the server can be trained to detect a large model, enabling it to understand the resource invocation intent described in natural language and automatically match the appropriate implicit invocation detection tool.

[0085] For example, for the phrase "read yesterday's weather data" in an advanced program, the detection model can identify that this description corresponds to an external interface call and output the interface detection tool and the required interface call parameters. The tool then constructs a test request in a simulation testing environment to verify whether the interface exists, requires authentication, or has been taken offline. Compared to static rule matching, which can only handle explicit code, the detection process in this specification can perform resource call detection on advanced programs, avoiding resource dependency blind spots caused by natural language knowledge concepts. Furthermore, it verifies resource validity without manually writing test cases, improving the reliability and development efficiency of advanced programs, while also supporting detection for large-scale application deployments.

[0086] The above is an example of a code generation detection method provided in this specification. Based on the same idea, this specification also provides corresponding devices, storage media and electronic devices.

[0087] Figure 3 This is a schematic diagram of a code generation detection device provided in an embodiment of this specification. The device includes: Tool building module 201 is used to determine a verification tool library, which includes at least an implicit call detection tool. The implicit call detection tool responds to a resource access statement and constructs a resource access request to perform resource call verification. The sample construction module 202 is used to acquire a code dataset, identify each code sample in the code dataset, determine the code segment that calls resources in each code sample, and determine the detection tool corresponding to the code segment from the verification tool library; Training module 203 is used to train a large detection model using the code sample as a training sample, and the code fragments identified in the code sample and the corresponding detection tools as annotations. The detection module 204 is used to respond to a code detection request by inputting the code to be detected into a trained detection model, determining the code segment that calls resources in the code to be detected and the corresponding detection tool, and determining the detection result of the code to be detected based on the verification result of the corresponding detection tool on the code segment.

[0088] Optionally, the implicit call detection tool includes a resource location detection tool; the resource location detection tool is used to generate dynamic resource access paths based on code snippets and detect the calls to resources corresponding to the resource access paths.

[0089] Optionally, the implicit call detection tool includes a permission detection tool; the permission detection tool is used to access resources referenced in the code snippet with preset permissions and to detect resource calls.

[0090] Optionally, the implicit call detection tool includes an interface detection tool; the interface detection tool is used to send test requests to external interfaces referenced in the code snippet and determine the status of the external interfaces based on the returned response.

[0091] Optionally, the detection module 204 is configured to, in response to a code detection request, determine the code to be detected; input the code to be detected into a trained detection model, which then identifies the code line by line; sequentially, for each identified code segment, perform resource call verification on the code segment according to the detection tool corresponding to that code segment output by the detection model; if the verification result is successful, continue to identify the next code segment and continue detection until the code to be detected is detected; if the verification result is unsuccessful, stop detection and output error information.

[0092] Optionally, the sample construction module 202 is used to obtain code sets of at least two preset programming languages; filter out code samples containing loading external resources from the code sets, wherein the external resources include at least one of files, libraries or interfaces; and determine a code dataset based on the filtered code samples.

[0093] Optionally, when the detection tool output by the large detection model is the resource location detection tool, the large detection model also outputs location parameters; the detection module 204 is used to input the location parameters into the resource location detection tool, so that the resource location detection tool parses the resource access path based on the location parameters and verifies whether the resource corresponding to the parsed path exists; if it exists, the verification result of the code segment is determined to be normal; if it does not exist, the verification result of the code segment is determined to be abnormal.

[0094] Optionally, when the detection tool output by the detection model is the permission detection tool, the detection model also outputs permission verification parameters; the detection module 204 is used to input the permission verification parameters into the permission detection tool, so that the permission detection tool accesses the resources referenced in the code segment with simulated permissions corresponding to the permission verification parameters; if the resource is accessed normally, the verification result of the code segment is determined to be normal; if the access is abnormal, the verification result of the code segment is determined to be abnormal.

[0095] Optionally, when the detection tool output by the large detection model is the interface detection tool, the large detection model also outputs interface call parameters. The detection module 204 is used to input the interface call parameters into the interface detection tool, so that the interface detection tool constructs a test request based on the interface call parameters and sends the test request to the external interface referenced in the code snippet; and determines the verification result of the code snippet based on the response information returned by the external interface. This specification also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can be used to perform the code generation detection method provided above.

[0096] based on Figure 1 The generated code detection method shown in this specification also provides embodiments that... Figure 4 The diagram shows the structure of the electronic device. Figure 4 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the aforementioned code generation detection method.

[0097] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for detecting generated code, the method comprising: A verification tool library is determined, which includes at least an implicit call detection tool. The implicit call detection tool constructs a resource access request in response to a resource access statement to perform resource call verification. Obtain a code dataset, identify each code sample in the code dataset, determine the code fragments that call resources in each code sample, and determine the detection tools corresponding to the code fragments from the verification tool library; Using the code sample as the training sample, and the code fragments identified in the code sample and the corresponding detection tools as annotations, a large detection model is trained; In response to a code detection request, the code to be detected is input into a trained detection model to determine the code segments that call resources in the code to be detected and the corresponding detection tools. Based on the verification results of the code segments by the corresponding detection tools, the detection result of the code to be detected is determined.

2. The method as described in claim 1, wherein the implicit call detection tool includes a resource location detection tool; The resource location detection tool is used to generate dynamic resource access paths based on code snippets and detect the calls to resources corresponding to the resource access paths.

3. The method as described in claim 1, wherein the implicit call detection tool includes a permission detection tool; The permission detection tool is used to access resources referenced in the code snippet with preset permissions and to detect resource calls.

4. The method as described in claim 1, wherein the implicit call detection tool includes an interface detection tool; The interface testing tool is used to send test requests to the external interfaces referenced in the code snippet and determine the status of the external interfaces based on the returned response.

5. The method as described in claim 1, in response to a code detection request, inputting the code to be detected into a trained detection model, determining the code segments that call resources in the code to be detected and their corresponding detection tools, and determining the detection result of the code to be detected based on the verification result of the corresponding detection tool on the code segments, specifically including: In response to a code inspection request, determine the code to be inspected; The code to be detected is input into the trained detection model, which then identifies the code line by line. For each identified code segment, resource call verification is performed on the code segment according to the detection tool corresponding to the code segment output by the large detection model. If the verification result is successful, the next code segment will be identified and the detection will continue until the code to be detected is detected. If the verification result fails, the detection will stop and an error message will be output.

6. The method as described in claim 1, specifically including: obtaining the code dataset, further comprising: Obtain code sets in at least two preset programming languages; From the code collection, code samples containing loading external resources are selected, the external resources including at least one of files, libraries or interfaces; Based on the selected code samples, the code dataset is determined.

7. The method as described in claim 2, wherein when the detection tool output by the large detection model is the resource location detection tool, the large detection model also outputs location parameters; The method further includes: The location parameters are input into the resource location detection tool, which then parses the resource access path based on the location parameters and verifies whether the resource corresponding to the parsed path exists. If it exists, then the verification result of the code snippet is considered normal; If it does not exist, then the verification result of the code snippet is determined to be abnormal.

8. The method as described in claim 3, wherein when the detection tool output by the large detection model is the permission detection tool, the large detection model also outputs permission verification parameters; The method further includes: The permission verification parameters are input into the permission detection tool, so that the permission detection tool can access the resources referenced in the code snippet with simulated permissions corresponding to the permission verification parameters; If the resource is accessed normally, then the verification result of the code snippet is confirmed to be normal. If an access error occurs, it indicates that the verification result of the code snippet is abnormal.

9. The method as described in claim 4, wherein when the detection tool output by the large detection model is the interface detection tool, the large detection model also outputs interface call parameters; The method further includes: The interface call parameters are input into the interface testing tool, which then constructs a test request based on the interface call parameters and sends the test request to the external interface referenced in the code snippet. The verification result of the code snippet is determined based on the response information returned by the external interface.

10. A code generation detection device, the device comprising: A tool building module is used to determine a verification tool library, which includes at least an implicit call detection tool. The implicit call detection tool responds to a resource access statement and constructs a resource access request to perform resource call verification. The sample construction module is used to acquire a code dataset, identify each code sample in the code dataset, determine the code fragments that call resources in each code sample, and determine the detection tools corresponding to the code fragments from the verification tool library; The training module is used to train a large detection model using the code sample as the training sample, and the code fragments identified in the code sample and the corresponding detection tools as annotations. The detection module is used to respond to code detection requests by inputting the code to be detected into the trained detection model, determining the code segments that call resources in the code to be detected and the corresponding detection tools, and determining the detection result of the code to be detected based on the verification results of the corresponding detection tools on the code segments.

11. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of claims 1-9.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1-9.

Images