Data warehouse code review method and device, equipment and storage medium

Abstract

The application discloses a kind of data warehouse code inspection method, device, equipment and storage medium, the method includes: the code to be inspected is parsed, and a plurality of logic fragments with different semantic functions are obtained;Based on the preset inspection rule set, each logic fragment is distributed to the corresponding rule detection queue, and the logic fragment in each rule detection queue is split into a statement unit to be detected;Concurrent processing is carried out to each rule detection queue, a large model is called to detect the rule matching of the statement unit to be detected, and a plurality of rule detection results are obtained;Each rule detection result is aggregated and processed to generate the inspection report corresponding to the code to be inspected.Compared with the prior art, the application improves the efficiency and accuracy of data warehouse code inspection by parsing the code to be inspected into multiple logic fragments and distributing them to the corresponding rule detection queue, concurrently calling a large model for rule matching detection, and generating an inspection report through aggregation processing.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data warehouse code inspection method, apparatus, device, and storage medium. Background Technology

[0002] As a core infrastructure for data asset management in banks and large enterprises, the code quality of a data warehouse directly affects the stability of batch data processing tasks and the reliability of business decisions. Currently, the development and quality assurance of data warehouse code mainly rely on manual code review, where developers or technical experts read the code line by line to check for business logic, syntax errors, and performance issues.

[0003] However, traditional manual review methods have many inherent flaws. On the one hand, the review process relies heavily on the personal experience of the reviewers. The varying skill levels of different technical personnel make it difficult to standardize review criteria, and insufficient experience can easily lead to overlooking spelling errors, grammatical errors, and potential performance bottlenecks. On the other hand, when faced with data warehouse scripts with complex business logic and large amounts of code, manual line-by-line review is time-consuming and labor-intensive, with extremely low efficiency. Moreover, the review focus often leans towards business logic while neglecting technical details, resulting in inconsistent code style, frequent missed performance issues, and ultimately, production environment failures and other incidents.

[0004] With the development of artificial intelligence technology, using large models for automated code inspection has become a technological trend. However, directly applying large models to data warehouse code inspection still faces significant technical obstacles: data warehouse scripts are usually long and easily exceed the input limits of large models; the high complexity of multi-rule concurrent inspection leads to a decrease in recognition accuracy; at the same time, multi-rule parallel processing is too time-consuming and cannot meet the timeliness requirements of business orchestration.

[0005] Therefore, there is an urgent need for a data warehouse code inspection method that can improve the efficiency and accuracy of data warehouse code inspection. Summary of the Invention

[0006] The main objective of this invention is to provide a data warehouse code inspection method, apparatus, device, and storage medium, aiming to solve the technical problem of low efficiency and accuracy in data warehouse code inspection in the prior art.

[0007] To achieve the above objectives, the present invention provides a data warehouse code inspection method, the method comprising the following steps: The code to be inspected is parsed to obtain multiple logical segments with different semantic functions; Based on a preset set of inspection rules, each logical segment is distributed to a corresponding rule detection queue, and the logical segments in each rule detection queue are split into units of statements to be detected. Concurrent processing is performed on each of the rule detection queues, and the large model is called to perform rule matching detection on the statement unit to be detected, thereby obtaining multiple rule detection results; The detection results of each rule are aggregated to generate an inspection report corresponding to the code to be inspected.

[0008] Optionally, the step of concurrently processing each of the rule detection queues, calling the large model to perform rule matching detection on the statement unit to be detected, and obtaining multiple rule detection results includes: For each rule detection queue, determine the number of sub-detection queues for each rule detection queue based on the corresponding inspection rule type; According to the number of sub-detection queues, each rule detection queue is evenly divided into multiple sub-detection queues; The large model is invoked in parallel for each of the sub-detection queues. Based on the inspection rule type corresponding to each sub-detection queue, rule matching detection is performed on the statement unit to be detected to obtain the sub-rule detection results corresponding to each sub-detection queue. The detection results of all sub-rules corresponding to the same rule detection queue are summarized to obtain the rule detection results corresponding to each rule detection queue.

[0009] Optionally, the step of determining the number of sub-detection queues for each rule detection queue based on the inspection rule type corresponding to each rule detection queue includes: Identify the type of inspection rule corresponding to each of the rule detection queues; The call frequency of each of the aforementioned inspection rule types in historical code inspection tasks is statistically analyzed, and the weight coefficient corresponding to each of the aforementioned inspection rule types is determined based on the call frequency. The call frequency and the weight coefficient are positively correlated. By combining the number of statement units to be detected in each rule detection queue and the weight coefficient of the corresponding inspection rule type, the number of sub-detection queues corresponding to each rule detection queue is determined.

[0010] Optionally, the step of concurrently calling the large model for each of the sub-detection queues, performing rule matching detection on the statement unit to be detected based on the inspection rule type corresponding to each sub-detection queue, and obtaining the sub-rule detection results corresponding to each sub-detection queue includes: Obtain the rule description information, input / output format description, and reference sample data for the inspection rule type corresponding to each of the sub-detection queues; Based on the rule description information, the input and output format specifications, and the reference sample data, a prompt word template corresponding to each of the inspection rule types is generated according to the preset template results. The large model is invoked based on each of the prompt word templates. The large model performs rule matching detection on the sentence unit to be detected according to the corresponding single inspection rule type, and obtains the sub-rule detection results corresponding to each of the sub-detection queues.

[0011] Optionally, the step of parsing the code to be inspected to obtain multiple logical segments with different semantic functions includes: The code to be inspected is parsed, and the module separators in the code to be inspected are identified based on the parsing results. Based on the module separator, the code to be inspected is divided into a variable definition module, a temporary table definition module, and a business logic module. The variable definition module is stored as a string, and the temporary table definition module and the business logic module are stored as a list, resulting in multiple logical fragments with different semantic functions.

[0012] Optionally, the step of distributing each logical segment to a corresponding rule detection queue based on a preset inspection rule set, and splitting the logical segments in each rule detection queue into statement units to be detected, includes: Based on each inspection rule in the preset inspection rule set, extract the corresponding rule feature keywords; Traverse each of the logical segments and assign the logical segments containing the rule feature keywords to the corresponding rule detection queue; The logical segments in each rule detection queue are split into statements to obtain the statement units to be detected corresponding to each rule detection queue.

[0013] Optionally, the step of aggregating the detection results of each rule to generate an inspection report corresponding to the code to be inspected includes: The detection results of each rule are deduplicated to obtain the deduplicated rule detection results. The deduplicated rule detection results are merged to obtain merged rule detection results. The merged rule detection results are formatted and arranged according to a preset display format to generate an inspection report corresponding to the code to be inspected.

[0014] Furthermore, to achieve the above objectives, the present invention also proposes a data warehouse code inspection device, the device comprising: The code parsing module is used to parse the code to be inspected and obtain multiple logical fragments with different semantic functions. The fragment splitting module is used to distribute each logical fragment to a corresponding rule detection queue based on a preset inspection rule set, and to split the logical fragments in each rule detection queue into statement units to be detected; The concurrent processing module is used to perform concurrent processing on each of the rule detection queues, call the large model to perform rule matching detection on the statement unit to be detected, and obtain multiple rule detection results; The result aggregation module is used to aggregate the detection results of each rule and generate an inspection report corresponding to the code to be inspected.

[0015] Furthermore, to achieve the above objectives, the present invention also proposes a data warehouse code inspection device, the device comprising: a memory, a processor, and a data warehouse code inspection program stored on the memory and executable on the processor, the data warehouse code inspection program being configured to implement the steps of the data warehouse code inspection method described above.

[0016] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a data warehouse code inspection program, wherein the data warehouse code inspection program, when executed by a processor, implements the steps of the data warehouse code inspection method as described above.

[0017] This invention discloses a method for parsing code to be inspected to obtain multiple logical segments with different semantic functions; based on a preset inspection rule set, distributing each logical segment to a corresponding rule detection queue, and further splitting the logical segments in each rule detection queue into statement units to be inspected; performing concurrent processing on each rule detection queue, calling a large model to perform rule matching detection on the statement units to be inspected, obtaining multiple rule detection results; and aggregating the rule detection results to generate an inspection report corresponding to the code to be inspected. Because this invention parses the code to be inspected into multiple logical segments and distributes them to corresponding rule detection queues, then concurrently calls a large model for rule matching detection, and finally generates an inspection report through aggregation processing, it improves the efficiency and accuracy of data warehouse code inspection compared to existing technologies. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the first embodiment of the data warehouse code inspection method of the present invention; Figure 2 This is a flowchart illustrating the code parsing process in the data warehouse code inspection method of the present invention; Figure 3 This is a flowchart illustrating the second embodiment of the data warehouse code inspection method of the present invention; Figure 4 This is a flowchart illustrating the third embodiment of the data warehouse code inspection method of the present invention; Figure 5 This is a structural block diagram of the first embodiment of the data warehouse code inspection device of the present invention; Figure 6This is a schematic diagram of the structure of a data warehouse code viewing device in the hardware operating environment involved in the embodiments of the present invention.

[0019] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0021] This invention provides a data warehouse code inspection method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the data warehouse code inspection method of the present invention.

[0022] In this embodiment, the data warehouse code inspection method includes steps S10 to S40: Step S10: Parse the code to be inspected to obtain multiple logical segments with different semantic functions.

[0023] It should be noted that the executing entity in this embodiment can be a computer server device with data processing, network communication, and program execution functions applied in code inspection scenarios, such as a server, tablet computer, or personal computer, or an electronic device capable of performing the above functions (such as a data warehouse code inspection device). The following uses a system including a data warehouse code inspection device (hereinafter referred to as the system) as an example to illustrate this embodiment and the following embodiments.

[0024] It should be understood that the code to be inspected mentioned above can refer to source code that needs to undergo code quality inspection. In a data warehouse development scenario, the code to be inspected can specifically be data warehouse code, such as SQL script code. The parsing mentioned above can refer to the process of identifying and decomposing code text according to its syntactic structure or predetermined delimitation rules.

[0025] It should be explained that the aforementioned logical segments can refer to code units with independent semantic functions and identifiable boundaries, such as variable definition units, temporary table definition units, or business logic implementation units. For example, in a complete SQL job script, the above logical segments may include variable definition parts, temporary table definition parts, and Request statement parts, each carrying different semantic functions.

[0026] In its implementation, the system first acquires the code text to be inspected during code inspection. Responding to a user-submitted code inspection request, the system reads the complete script content from a code repository or development tools. Next, the system parses the code. Based on the code's syntactic structure, the system identifies the boundaries of each module within the code. Subsequently, based on the identified module boundaries, the system segments the code into multiple logical segments with different semantic functions. After segmentation, the system obtains multiple independent logical segments, each corresponding to a different semantic function within the code being inspected.

[0027] For ease of understanding, the following explanations are provided using examples, but these examples do not impose specific limitations on this embodiment. For example, refer to... Figure 2 Suppose the code to be inspected is an SQL job script, whose content sequentially includes a variable definition area, a temporary table definition area, and a business logic implementation area. After obtaining this script, the system parses the keywords and structure in the script to identify the start and end positions of the variable definition area, the temporary table definition area, and the business logic implementation area. Based on the identified boundaries, the system divides the script into three logical segments: the variable definition section, the temporary table definition section, and the request statement section, resulting in three logical segments with different semantic functions.

[0028] To effectively address the issue of long scripts exceeding the input limit of large models, the step of parsing the code to be inspected to obtain multiple logical fragments with different semantic functions may include: parsing the code to be inspected and identifying module separators in the code based on the parsing results; dividing the code to be inspected into variable definition modules, temporary table definition modules, and business logic modules based on the module separators; storing the variable definition modules as strings and storing the temporary table definition modules and the business logic modules as lists to obtain multiple logical fragments with different semantic functions.

[0029] It should be explained that the above module separators can refer to specific syntax marks or keywords used to identify the boundaries of different modules in the code to be inspected. In data warehouse SQL scripts, the above module separators can specifically be represented as the starting mark of the variable definition area, the starting mark of the temporary table definition area, and the starting mark of the business logic area.

[0030] It should be noted that the above variable definition module can refer to the code area in the code to be inspected that is used to define job variables, the above temporary table definition module can refer to the code area in the code to be inspected that is used to define temporary tables, and the above business logic module can refer to the main code area in the code to be inspected that is used to implement business logic processing, such as the Request statement area.

[0031] Understandably, the above-mentioned string-based storage can refer to storing the code content in memory or cache as a continuous string data format, while the above-mentioned list-based storage can refer to storing the code content as a list data structure after breaking it down by line or by statement unit, with each element in the list corresponding to a line of code or a complete statement unit.

[0032] In its implementation, after obtaining the code to be inspected, the system parses the code and identifies module separators based on the parsing results. The system can use regular expression matching to match predetermined module keywords with the code to locate the start and end positions of each module. Next, based on the identified module separators, the system segments the code into variable definition modules, temporary table definition modules, and business logic modules. Subsequently, the system stores the variable definition modules as strings. Furthermore, the system stores the temporary table definition modules and the business logic modules as lists. After completing these segmentation and storage operations, the system obtains multiple logical fragments with different semantic functions.

[0033] To facilitate understanding, the following examples illustrate the concept, but do not impose specific limitations on this embodiment. For instance, suppose the code to be inspected is an SQL script. The variable definition section in the script begins with the keyword "DEFINE," the temporary table definition section begins with the keyword "CREATE VOLATILE TABLE," and the business logic section begins with the keyword "REQUEST." The system uses regular expressions to match these keywords, identifying the location of "DEFINE" as the starting point of the variable definition module, the location of "CREATE VOLATILE TABLE" as the starting point of the temporary table definition module, and the location of "REQUEST" as the starting point of the business logic module. The system then divides the script into three modules according to these starting points. The complete content of the variable definition module is stored as a string in a memory variable. Each line of code in the temporary table definition module is sequentially stored in a list object, and each line of code in the business logic module is sequentially stored in another list object, thus obtaining three logical segments with different semantic functions.

[0034] It should be understood that by identifying module separators, the code is structurally divided into variable definitions, temporary table definitions, and business logic modules. Differentiated storage methods are adopted for the data characteristics of different modules, which realizes fine-grained code parsing. This provides a foundation for subsequent modular cyclic input into large models and effectively solves the problem of long scripts exceeding the input limit of large models.

[0035] Step S20: Based on the preset inspection rule set, distribute each logical segment to the corresponding rule detection queue, and split the logical segments in each rule detection queue into statement units to be detected.

[0036] It should be noted that the aforementioned preset inspection rule set can refer to a predefined set of rules for detecting code quality. The preset inspection rule set contains multiple inspection rules, and each inspection rule corresponds to a type of code problem, such as a specification problem, a performance problem, or a syntax problem.

[0037] It should be explained that the rule detection queue mentioned above can refer to a cache queue that is independently allocated to each inspection rule. The rule detection queue is used to store the logical fragments to be inspected. Different rule detection queues correspond to different inspection rule types. A logical fragment can be distributed to multiple rule detection queues.

[0038] Understandably, the aforementioned unit of statement to be detected can refer to the smallest unit of code that can be independently detected by rule matching, which is separated from a logical segment, such as a complete SQL statement or an independent code clause.

[0039] In its implementation, after obtaining multiple logical segments, the system acquires a preset set of inspection rules. Based on these rules, the system distributes each logical segment to a rule detection queue corresponding to that rule. Specifically, the system iterates through each logical segment and determines whether it matches a feature of interest to a particular inspection rule. If it does, the segment is distributed to the rule detection queue corresponding to that rule. A logical segment may be distributed to multiple rule detection queues simultaneously or not at all. Next, the system processes each rule detection queue separately, splitting the logical segments into units to be detected. The system can further segment the logical segments based on statement boundary identifiers to obtain multiple smaller, structurally complete units to be detected. After splitting, each rule detection queue stores its corresponding unit to be detected.

[0040] For ease of understanding, the following explanation uses examples, but does not impose specific limitations on this embodiment. For example, assume that the preset inspection rule set contains two inspection rules: a GROUP BY detection rule and an NDS table COALESCE detection rule. The system pre-creates a GROUP BY rule inspection queue and an NDS rule inspection queue. The system traverses each obtained logical segment. When it encounters a Request logical segment, it identifies that the segment contains both the GROUP BY keyword and the NDS keyword, and then distributes the segment to both the GROUP BY rule inspection queue and the NDS rule inspection queue. Subsequently, the system segments the logical segments in the GROUP BY rule inspection queue using semicolons or newlines, splitting them into multiple single-statement units to be inspected. Similarly, the system performs the same segmentation process on the logical segments in the NDS rule inspection queue, obtaining the corresponding units to be inspected in the NDS rule inspection queue. After completing the above operations, each rule inspection queue stores its own independent list of units to be inspected.

[0041] Step S30: Perform concurrent processing on each rule detection queue, call the large model to perform rule matching detection on the statement unit to be detected, and obtain multiple rule detection results.

[0042] Understandably, the aforementioned large model may refer to a large-scale language model built based on deep learning technology. This large model has the ability to understand natural language and analyze code syntax, and can perform semantic understanding and rule determination on the input code text based on the input prompt words.

[0043] It should be noted that the above rule matching detection can refer to comparing the statement unit to be detected with the preset inspection rules to determine whether the statement unit to be detected violates the inspection rules, such as whether necessary keywords are missing, whether there is a non-standard syntax structure, or whether there are potential performance risks.

[0044] It should be explained that the above rule detection results can refer to the judgment conclusions output by the large model for each rule matching detection. The above rule detection results include at least the problem description information, the problem location information, and the problem type information. For example, for a GROUP BY statement, the above rule detection results can indicate that there is a problem in the SELECT clause of the statement where a non-aggregate field does not appear in GROUP BY.

[0045] In its implementation, after the system breaks down the logical segments in each rule detection queue into statement units to be detected, it processes each rule detection queue concurrently. The system can launch multiple processing processes or threads simultaneously, each corresponding to a rule detection queue. For each rule detection queue, the system calls the large model to perform rule matching detection on the statement units to be detected in that queue.

[0046] Specifically, the system can organize the statement unit to be detected and the inspection rule information corresponding to the rule detection queue into input content, and send it to the interface of the large model. The large model then outputs a judgment result on whether the statement unit to be detected violates the inspection rule. The system sequentially performs the above-mentioned operation of calling the large model for rule matching detection on all statement units to be detected in each rule detection queue until all statement units to be detected in the rule detection queue have been detected. After the concurrent processing of all rule detection queues has been completed, the system obtains multiple rule detection results, each rule detection result corresponding to one detection output of a rule detection queue.

[0047] To facilitate understanding, the following explanation uses examples, but does not impose specific limitations on this embodiment. For example, assume there is a GROUP BY rule detection queue and an NDS table COALESCE rule detection queue, each storing several statement units to be detected. The system simultaneously starts two parallel threads: the first thread processes the GROUP BY rule detection queue, and the second thread processes the NDS table COALESCE rule detection queue. In the first thread, the system concatenates the GROUP BY rule description information with the first statement unit to be detected in the queue to form a prompt word, calls the large model interface, and the large model returns a determination result indicating whether the statement has a GROUP BY issue. The system continues processing the next statement until the queue is empty. In the second thread, the system calls the large model in the same way to detect the NDS table COALESCE rule. After the two threads execute concurrently, the system obtains two rule detection results, corresponding to the complete detection outputs of the GROUP BY detection and the NDS table COALESCE detection, respectively.

[0048] Step S40: Aggregate the detection results of each rule to generate an inspection report corresponding to the code to be inspected.

[0049] It should be noted that the above aggregation process refers to the operation of merging, organizing and summarizing multiple independent rule detection results according to predetermined rules. The above aggregation process includes deduplication of multiple detection results, merging similar problems and unifying data format.

[0050] Understandably, the inspection report can refer to a summary document of the code quality inspection results finally presented to the user. The aforementioned inspection report includes a list of all issues found, the code location information corresponding to each issue, the issue type classification, and the issue severity indicator.

[0051] To facilitate understanding, the following explanation uses examples, but does not impose specific limitations on this embodiment. For instance, suppose the preset inspection rule set contains 26 inspection rules. After the system processes all rule detection queues concurrently, it obtains 26 rule detection results, some of which are empty, and some contain problem records. The system collects all 26 rule detection results, summarizes the problem records in all non-empty detection results into a temporary data set, and performs format standardization and problem classification and merging on the temporary data set to eliminate duplicate records. After completing the above aggregation process, the system generates a structured document as an inspection report based on the aggregated data set. The report displays each problem record according to problem type, and each problem record includes a problem description, the code location, and the severity of the problem.

[0052] To improve the readability of the inspection report and user efficiency, and to facilitate developers in quickly locating and fixing code issues, the step of aggregating the detection results of each rule to generate an inspection report corresponding to the code to be inspected may include: deduplicating the detection results of each rule to obtain deduplicated rule detection results; merging the deduplicated rule detection results to obtain merged rule detection results; and formatting the merged rule detection results according to a preset display format to generate an inspection report corresponding to the code to be inspected.

[0053] Understandably, the deduplication process described above can refer to the process of identifying and removing duplicate records from multiple rule detection results. For example, if the same code statement is reported by multiple rule detection results because it hits multiple inspection rules at the same time, the deduplication process described above is used to retain each record but eliminate completely identical duplicate entries.

[0054] The above-mentioned merging process can refer to the operation of integrating the detection results of various rules after deduplication into the same dataset. The above-mentioned merging process includes grouping the detection results from different sources according to a unified data field structure.

[0055] It should be noted that the aforementioned preset display format refers to a pre-defined output template used to organize the structure of the inspection report content. This preset display format specifies the order of the test results in the report, the display method of fields, and the classification organization method. The aforementioned formatting and arrangement refers to the process of converting the merged test data into a structured display document according to the preset display format.

[0056] In its implementation, after obtaining multiple rule detection results output by each rule detection queue, the system first performs deduplication on each rule detection result. The system iterates through each issue record in all rule detection results, combining the code location information, issue description information, and issue type information in the issue record into a unique identifier. By comparing the unique identifier, duplicate issue records are identified and removed, thus obtaining the deduplicated rule detection results.

[0057] Next, the system merges the deduplicated rule detection results. The system aggregates the problem records from each rule detection result into the same dataset and standardizes the fields of each problem record according to a unified data field definition to obtain the merged rule detection results.

[0058] Subsequently, the system formats and arranges the merged rule detection results according to a preset display format. The system reads the problem classification method, field order, and display style requirements defined in the preset display format, reorganizes and transforms the merged rule detection results according to the preset display format, and generates a review report corresponding to the code to be reviewed.

[0059] To facilitate understanding, the following example illustrates the concept, but does not impose specific limitations on this embodiment. For instance, suppose the system executes concurrent detection corresponding to 26 inspection rules and obtains 26 rule detection results. Among these, the GROUP BY detection results include a record regarding the issue that "field A in the SELECT clause does not appear in the GROUP BY clause," and this same record was also output by another duplicate detection rule. When performing deduplication, the system identifies two record problems with identical line numbers, descriptions, and types. Therefore, it removes one of the duplicate records, retaining only one, thus obtaining the deduplicated rule detection results. Next, the system aggregates all deduplicated record problems into a list, unifying the field structure of each record to four fields: "line number, problem type, problem description, and suggested modification," obtaining the merged rule detection results. Subsequently, the system reads a preset display format, which requires displaying problems categorized by type and arranged in ascending order of line number within each category. The system then arranges the merged test results according to this format, generating a review report that includes a categorization directory and a detailed list of issues.

[0060] It should be understood that by eliminating duplicate inspection results through deduplication, merging and integrating multi-rule detection information, and formatting and arranging the results according to a preset display format, the standardized output and structured presentation of inspection results are achieved, improving the readability of inspection reports and user access efficiency, and making it easier for developers to quickly locate and fix code problems.

[0061] This embodiment discloses a method for parsing the code to be inspected to obtain multiple logical segments with different semantic functions; based on a preset inspection rule set, distributing each logical segment to a corresponding rule detection queue, and splitting the logical segments in each rule detection queue into statement units to be inspected; performing concurrent processing on each rule detection queue, calling a large model to perform rule matching detection on the statement units to be inspected, obtaining multiple rule detection results; and aggregating the rule detection results to generate an inspection report corresponding to the code to be inspected. Because this embodiment parses the code to be inspected into multiple logical segments and distributes them to corresponding rule detection queues, then concurrently calls a large model for rule matching detection, and finally generates an inspection report through aggregation processing, compared to existing technologies, this embodiment improves the efficiency and accuracy of data warehouse code inspection.

[0062] refer to Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the data warehouse code inspection method of the present invention.

[0063] Based on the first embodiment described above, in this embodiment, step S30 includes steps S301 to S304: Step S301: For each rule detection queue corresponding to the inspection rule type, determine the number of sub-detection queues for each rule detection queue.

[0064] Step S302: Divide each rule detection queue evenly into multiple sub-detection queues according to the number of sub-detection queues.

[0065] Step S303: Invoke the large model in parallel for each of the sub-detection queues, and perform rule matching detection on the statement unit to be detected based on the inspection rule type corresponding to the sub-detection queue to obtain the sub-rule detection result corresponding to each of the sub-detection queues.

[0066] Step S304: Summarize the detection results of all sub-rules corresponding to the same rule detection queue to obtain the rule detection results corresponding to each rule detection queue.

[0067] It should be noted that the above inspection rule types can refer to the problem category to which each inspection rule belongs. The above inspection rule types include specification problem types, performance problem types, and syntax problem types. Different inspection rule types correspond to different inspection focus points and judgment criteria.

[0068] It should be understood that the aforementioned sub-detection queues can refer to multiple sub-queues obtained by dividing a rule detection queue according to the number of divisions. Each sub-detection queue contains a portion of the statement units to be detected from the original rule detection queue, and each sub-detection queue is independent of the others. The aforementioned number of sub-detection queue divisions can refer to the number of sub-detection queues into which a rule detection queue is divided. Different rule detection queues can have different numbers of sub-detection queue divisions.

[0069] It should be explained that the aforementioned sub-rule detection results can refer to the independent detection results output by each sub-detection queue after being detected by the large model, or multiple sub-detection queues under the same rule detection queue outputting their respective sub-rule detection results. The aforementioned aggregation can refer to the operation of collecting and merging all sub-rule detection results belonging to the same rule detection queue.

[0070] In the specific implementation, for each rule detection queue, the number of sub-detection queues corresponding to that rule detection queue is first determined. The system described above can allocate a number of sub-detection queues to each rule detection queue according to the inspection rule type corresponding to each rule detection queue, and different rule detection queues can correspond to different numbers of sub-detection queues.

[0071] Next, the system divides the statement units to be detected in each rule detection queue into multiple sub-detection queues according to the number of sub-detection queues corresponding to each rule detection queue. For each rule detection queue, the system divides the total number of statement units to be detected in that rule detection queue by the number of sub-detection queues for that rule detection queue, and allocates the statement units to be detected to each sub-detection queue according to the calculation result, so that the number of statement units to be detected in each sub-detection queue is basically the same.

[0072] Subsequently, the system invokes the large model in parallel across all sub-detection queues under each rule detection queue. For each sub-detection queue, the system performs rule matching detection on each statement unit to be detected in that sub-detection queue based on the inspection rule type corresponding to the rule detection queue to which it belongs. After each sub-detection queue is detected, it outputs a sub-rule detection result. After the system completes the parallel invocation and detection of all sub-detection queues, for each rule detection queue, the system summarizes all sub-rule detection results corresponding to that rule detection queue, merging multiple sub-rule detection results under that rule detection queue into a complete rule detection result, thereby obtaining the rule detection results corresponding to each rule detection queue.

[0073] To facilitate understanding, the following explanation uses examples, but does not impose specific limitations on this embodiment. For example, assume that the above system contains a GROUP BY rule detection queue and an NDS table COALESCE rule detection queue. The GROUP BY rule detection queue stores 200 statement units to be detected, whose corresponding inspection rule type is a specification problem type. The system determines the number of sub-detection queues for this queue to be 4. The NDS table COALESCE rule detection queue stores 20 statement units to be detected, whose corresponding inspection rule type is also a specification problem type. The system determines the number of sub-detection queues for this queue to be 2. Next, the system evenly divides the 200 statement units to be detected in the GROUP BY rule detection queue into 4 sub-detection queues, each containing 50 statement units to be detected, and evenly divides the 20 statement units to be detected in the NDS table COALESCE rule detection queue into 2 sub-detection queues, each containing 10 statement units to be detected. Subsequently, the system launches six parallel threads to simultaneously call the large model for detection on the four sub-detection queues of the GROUP BY rule detection queue and the two sub-detection queues of the NDS table COALESCE rule detection queue. Each sub-detection queue outputs a sub-rule detection result after detection. After detection, the system aggregates the four sub-rule detection results of the GROUP BY rule detection queue into a single GROUP BY rule detection result, and aggregates the two sub-rule detection results of the NDS table COALESCE rule detection queue into a single NDS table COALESCE rule detection result.

[0074] To improve the overall resource utilization efficiency and load balancing accuracy of concurrent detection, the step of determining the number of sub-detection queues for each rule detection queue, based on the inspection rule type corresponding to each rule detection queue, may include: identifying the inspection rule type corresponding to each rule detection queue; calculating the call frequency of each inspection rule type in historical code inspection tasks, and determining the weight coefficient corresponding to each inspection rule type based on the call frequency, wherein the call frequency is positively correlated with the weight coefficient; and determining the number of sub-detection queues corresponding to each rule detection queue by combining the number of statement units to be detected in each rule detection queue and the weight coefficient of the corresponding inspection rule type.

[0075] It is understood that the aforementioned historical code inspection tasks may refer to all or part of the code inspection tasks processed by the system before executing the current code inspection operation. The aforementioned historical code inspection tasks record the inspection rule types involved in each inspection task and the number of times they were called.

[0076] It should be explained that the aforementioned call frequency refers to the statistical frequency of each inspection rule type being actually triggered in historical code inspection tasks. This call frequency can be quantified in the form of absolute counts or relative proportions. The aforementioned weight coefficient refers to a numerical factor used to measure the priority of different inspection rule types in allocating processing resources. The magnitude of the weight coefficient is positively correlated with the call frequency; that is, the higher the call frequency, the larger the weight coefficient, and the lower the call frequency, the smaller the weight coefficient.

[0077] In its implementation, when determining the number of sub-detection queues for each rule detection queue, the system first identifies the type of inspection rule corresponding to each rule detection queue. The system can then read the type label of each inspection rule from a preset set of inspection rules and associate that type label with the corresponding rule detection queue.

[0078] Next, the system calculates the call frequency of each inspection rule type in historical code inspection tasks. The system can read stored historical inspection log data, count the inspection rule types called in each historical inspection task, obtain the cumulative call count for each inspection rule type, and calculate the call frequency corresponding to each inspection rule type based on the cumulative call count.

[0079] Subsequently, the system determines the weight coefficient for each view rule type based on its call frequency. The weight coefficient is determined by the call frequency; a higher call frequency results in a larger weight coefficient, meaning the weight coefficient is positively correlated with the call frequency.

[0080] Next, the system obtains the number of statement units to be detected in each rule detection queue, and determines the number of sub-detection queues corresponding to each rule detection queue by combining the weight coefficient of the inspection rule type corresponding to each rule detection queue. For each rule detection queue, the system multiplies the number of statement units to be detected in the rule detection queue by the weight coefficient corresponding to the queue, calculates a partition value based on the product, and uses this partition value as the number of sub-detection queues for that rule detection queue. The system performs the above calculation process on all rule detection queues to obtain the number of sub-detection queues corresponding to each rule detection queue.

[0081] To facilitate understanding, the following explanation uses examples, but does not impose specific limitations on this embodiment. For example, assume that the system has a GROUP BY detection queue and an NDS table COALESCE detection queue. The former corresponds to a "grouping specification" rule type, and the latter corresponds to a "null value handling" rule type. The system queries historical inspection logs and finds that the "grouping specification" rule was called 80 times in the past 100 inspection tasks, with a call frequency of 80%, while the "null value handling" rule was called 20 times, with a call frequency of 20%. The system determines weight coefficients based on the call frequencies, setting the weight coefficient for the "grouping specification" rule to 0.8 and the weight coefficient for the "null value handling" rule to 0.2. Simultaneously, the system obtains that the number of statement units to be inspected in the GROUP BY detection queue is 200, and the number of statement units to be inspected in the NDS table COALESCE detection queue is 20. For the GROUP BY detection queue, the system multiplies the quantity 200 by a weighting coefficient of 0.8, resulting in 160. Then, using a preset normalization mapping rule, 160 is mapped to a sub-detection queue division number of 4. For the NDS table COALESCE detection queue, the quantity 20 is multiplied by a weighting coefficient of 0.2, resulting in 4, which is mapped to a sub-detection queue division number of 2. Ultimately, the number of sub-detection queue divisions for the GROUP BY detection queue is determined to be 4, and the number of sub-detection queue divisions for the NDS table COALESCE detection queue is determined to be 2.

[0082] It should be understood that by statistically examining the historical call frequency of rule types to determine the weight coefficient, and combining it with the number of statement units to be detected to dynamically determine the number of sub-detection queues, fine-grained allocation of concurrent resources is achieved. This avoids response delays caused by insufficient splitting of high-frequency rules and resource waste caused by excessive splitting of low-frequency rules, thereby improving the overall resource utilization efficiency and load balancing accuracy of concurrent detection.

[0083] To reduce the complexity of understanding the large model and improve the accuracy of rule recognition, the step of calling the large model in parallel for each of the sub-detection queues, performing rule matching detection on the statement unit to be detected based on the inspection rule type corresponding to each sub-detection queue, and obtaining the sub-rule detection results corresponding to each of the sub-detection queues may include: obtaining rule description information, input / output format description, and reference sample data for the inspection rule type corresponding to each of the sub-detection queues; generating prompt word templates that correspond one-to-one with each of the inspection rule types based on the rule description information, the input / output format description, and the reference sample data, according to preset template results; calling the large model based on each prompt word template, and having the large model perform rule matching detection on the statement unit to be detected according to the corresponding single inspection rule type, thereby obtaining the sub-rule detection results corresponding to each of the sub-detection queues.

[0084] It should be noted that the rule description information mentioned above can refer to explanatory text that describes the detection target, judgment criteria and applicable scenarios of each inspection rule. The rule description information is used to clarify the detection task to be performed to the large model.

[0085] The above input / output format specification can refer to the format definition text that specifies the structural form of the data received by the large model and the data structure form of the output detection results. The above input / output format specification includes the name and order of the input fields, the name and order of the output fields, and the data type of each field.

[0086] The aforementioned reference sample data may refer to example data pairs provided to the large model, which contain input statements and corresponding correct detection output results. The aforementioned reference sample data is used to guide the large model to output detection results in the expected manner.

[0087] Understandably, the aforementioned prompt word template can refer to a complete prompt text framework that combines rule description information, input / output format specifications, and reference sample data according to a predetermined structure. This prompt word template is used as input content when calling a large model. The aforementioned single inspection rule type can mean that only one inspection rule type is detected during each large model detection process, rather than processing multiple inspection rule types simultaneously.

[0088] In the specific implementation, for each sub-detection queue, the system obtains the rule description information, input / output format specifications, and reference sample data corresponding to the inspection rule type of that sub-detection queue. The system can read these three types of information associated with the inspection rule type from a preset rule base.

[0089] Next, based on the acquired rule description information, input / output format specifications, and reference sample data, the system generates prompt word templates corresponding one-to-one with each inspection rule type, according to a preset template structure. The system fills the task description information into the task description area of ​​the prompt word template, the input / output format specifications into the format specification area, and the reference sample data into the example display area, forming a complete prompt word template. The system generates a corresponding prompt word template for each inspection rule type.

[0090] Subsequently, the system invokes the large model based on the generated prompt word templates. For each sub-detection queue, the system combines the prompt word template corresponding to the sub-detection queue and the sentence units to be detected in the sub-detection queue into the input for a single large model call, and sends it to the large model. The large model then performs rule matching detection on the sentence units to be detected according to the single inspection rule type specified in the prompt word template. The large model outputs a judgment result for each sentence unit to be detected according to the input-output format specified in the prompt word template. The system receives the detection results returned by the large model and obtains the sub-rule detection results corresponding to each sub-detection queue.

[0091] It should be understood that by customizing independent prompt word templates for each inspection rule, the complex multi-rule detection is decomposed into a single rule-specific recognition task, reducing the understanding complexity of the large model. At the same time, by using the constraints of input and output formats and reference examples, the output structure is standardized, which not only significantly improves the accuracy of rule recognition, but also provides a unified data foundation for the subsequent automated aggregation of multi-source detection results.

[0092] This embodiment discloses a method for determining the number of sub-detection queues for each rule detection queue based on the inspection rule type corresponding to each rule detection queue; uniformly dividing each rule detection queue into multiple sub-detection queues according to the number of sub-detection queues; in parallel calling the large model for each sub-detection queue; performing rule matching detection on the statement unit to be detected based on the inspection rule type corresponding to each sub-detection queue; obtaining the sub-rule detection results corresponding to each sub-detection queue; and summarizing all sub-rule detection results corresponding to the same rule detection queue to obtain the rule detection results corresponding to each rule detection queue. Because this embodiment determines the number of sub-detection queues based on the inspection rule type and uniformly splits the rule detection queues before invoking the large model in parallel for detection, compared to existing technologies, this embodiment effectively shortens the overall response time of multi-rule concurrent detection, solves the problem of linearly increasing detection time due to the increase in the number of rules, and significantly improves code inspection efficiency while ensuring detection accuracy.

[0093] refer to Figure 4 , Figure 4This is a flowchart illustrating the third embodiment of the data warehouse code inspection method of the present invention.

[0094] Based on the above embodiments, in this embodiment, step S20 includes steps S201 to S203: Step S201: Extract the corresponding rule feature keywords according to each inspection rule in the preset inspection rule set.

[0095] Step S202: Traverse each of the logical segments and assign the logical segments containing the rule feature keywords to the corresponding rule detection queue.

[0096] Step S203: Perform statement splitting on the logical segments in each rule detection queue to obtain the statement unit to be detected corresponding to each rule detection queue.

[0097] It should be explained that the above-mentioned rule feature keywords can refer to specific strings or syntax patterns associated with each inspection rule, used to identify the code that may trigger the inspection rule. The above-mentioned rule feature keywords are used to quickly locate the code range that needs to be inspected in the logical segment. For example, the rule feature keyword corresponding to the GROUP BY inspection rule can be the string "GROUP BY", and the rule feature keyword corresponding to the COALESCE inspection rule of the NDS table can be the string "NDS".

[0098] Understandably, the above traversal can refer to the process of sequentially accessing each logical segment and performing the same judgment operation on each logical segment.

[0099] It should be noted that the above statement splitting can refer to the operation of dividing a logical segment into multiple independent and structurally complete code statements according to a predetermined statement boundary separator. The above statement boundary separator can include semicolons, newline characters, or specific keywords. For example, a logical segment containing multiple SQL statements can be split into multiple independent SQL statements by semicolons.

[0100] In its implementation, the system extracts the corresponding rule feature keywords based on each view rule in the preset view rule set. For each view rule, the system can read the pre-associated keyword list from its rule configuration information to obtain the rule feature keywords corresponding to each view rule.

[0101] Next, the system iterates through each logical segment, assigning segments containing rule feature keywords to their corresponding rule detection queues. The system sequentially retrieves each logical segment, matches its text content against the rule feature keywords of each inspection rule, and if a match is found, assigns the logical segment to the rule detection queue corresponding to that inspection rule. A logical segment can contain multiple different rule feature keywords, and this segment will be simultaneously assigned to multiple corresponding rule detection queues. The system performs the above matching and assignment operations on all logical segments until all logical segments have been processed.

[0102] Subsequently, the system performs statement splitting on the logical segments in each rule detection queue. For each rule detection queue, the system retrieves all logical segments stored in the queue, splits each logical segment into multiple independent statement units to be detected according to statement boundary delimiters, and stores the split statement units to be detected in the rule detection queue, thus obtaining the set of statement units to be detected corresponding to each rule detection queue.

[0103] This embodiment discloses a method for extracting corresponding rule feature keywords based on each inspection rule in a preset inspection rule set; traversing each logical segment and assigning logical segments containing the rule feature keywords to corresponding rule detection queues; and performing statement splitting on the logical segments in each rule detection queue to obtain the statement units to be detected corresponding to each rule detection queue. Because this embodiment extracts rule feature keywords, traverses and matches logical segments, and performs statement splitting on the logical segments in each rule detection queue, compared to existing technologies, this embodiment achieves accurate pre-association between code segments and detection rules, reducing the semantic complexity of single-processing of large models, reducing unnecessary computational power consumption, and further improving the accuracy of single-rule detection.

[0104] Furthermore, this embodiment of the invention also proposes a storage medium storing a data warehouse code inspection program, which, when executed by a processor, implements the steps of the data warehouse code inspection method described above.

[0105] Reference Figure 5 , Figure 5 This is a structural block diagram of the first embodiment of the data warehouse code inspection device of the present invention.

[0106] like Figure 5 As shown, the data warehouse code inspection device proposed in this embodiment of the invention includes: a code parsing module 501, a fragment splitting module 502, a concurrent processing module 503, and a result aggregation module 504.

[0107] The code parsing module 501 is used to parse the code to be examined and obtain multiple logical segments with different semantic functions.

[0108] The segment splitting module 502 is used to distribute each logical segment to a corresponding rule detection queue based on a preset inspection rule set, and to split the logical segments in each rule detection queue into statement units to be detected.

[0109] The concurrent processing module 503 is used to perform concurrent processing on each of the rule detection queues, call the large model to perform rule matching detection on the statement unit to be detected, and obtain multiple rule detection results.

[0110] The result aggregation module 504 is used to aggregate the detection results of each rule and generate an inspection report corresponding to the code to be inspected.

[0111] The code parsing module 501 is further configured to parse the code to be inspected, identify module separators in the code to be inspected based on the parsing results, divide the code to be inspected into variable definition modules, temporary table definition modules, and business logic modules based on the module separators, and store the variable definition modules as strings, and store the temporary table definition modules and the business logic modules as lists to obtain multiple logical fragments with different semantic functions.

[0112] The result aggregation module 504 is further configured to perform deduplication processing on each rule detection result to obtain deduplicated rule detection results; merge the deduplicated rule detection results to obtain merged rule detection results; format and arrange the merged rule detection results according to a preset display format to generate an inspection report corresponding to the code to be inspected.

[0113] This device embodiment discloses a method for parsing code to be inspected to obtain multiple logical segments with different semantic functions; based on a preset inspection rule set, distributing each logical segment to a corresponding rule detection queue, and splitting the logical segments in each rule detection queue into statement units to be inspected; performing concurrent processing on each rule detection queue, calling a large model to perform rule matching detection on the statement units to be inspected, obtaining multiple rule detection results; and aggregating the rule detection results to generate an inspection report corresponding to the code to be inspected. Because this device embodiment parses the code to be inspected into multiple logical segments and distributes them to corresponding rule detection queues, then concurrently calls a large model for rule matching detection, and finally generates an inspection report through aggregation processing, compared to existing technologies, this device embodiment improves the efficiency and accuracy of data warehouse code inspection.

[0114] Based on the first embodiment of the data warehouse code inspection device of the present invention, a second embodiment of the data warehouse code inspection device of the present invention is proposed.

[0115] In this embodiment, the concurrent processing module 503 is further configured to: determine the number of sub-detection queues for each rule detection queue based on the inspection rule type corresponding to each rule detection queue; evenly divide each rule detection queue into multiple sub-detection queues according to the number of sub-detection queues; call the large model in parallel for each sub-detection queue; perform rule matching detection on the statement unit to be detected based on the inspection rule type corresponding to the sub-detection queue; obtain the sub-rule detection results corresponding to each sub-detection queue; and summarize all sub-rule detection results corresponding to the same rule detection queue to obtain the rule detection results corresponding to each rule detection queue.

[0116] The concurrent processing module 503 is further configured to identify the inspection rule type corresponding to each rule detection queue; count the call frequency of each inspection rule type in the historical code inspection task, and determine the weight coefficient corresponding to each inspection rule type based on the call frequency, wherein the call frequency is positively correlated with the weight coefficient; and determine the number of sub-detection queues corresponding to each rule detection queue by combining the number of statement units to be detected in each rule detection queue and the weight coefficient of the corresponding inspection rule type.

[0117] The concurrent processing module 503 is further configured to acquire rule description information, input / output format description, and reference sample data for each of the sub-detection queues corresponding to the inspection rule types; based on the rule description information, the input / output format description, and the reference sample data, generate prompt word templates that correspond one-to-one with each of the inspection rule types according to preset template results; and call the large model based on each prompt word template, so that the large model performs rule matching detection on the statement unit to be detected according to the corresponding single inspection rule type, thereby obtaining the sub-rule detection results corresponding to each of the sub-detection queues.

[0118] Other embodiments or specific implementations of the data warehouse code inspection device of the present invention can be referred to the above-described method embodiments, and will not be repeated here.

[0119] This application provides a data warehouse code inspection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the data warehouse code inspection method in the first embodiment described above.

[0120] The following is for reference. Figure 6The diagram illustrates a structural schematic of a data warehouse code viewing device suitable for implementing embodiments of this application. The data warehouse code viewing device in embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The data warehouse code viewing device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0121] like Figure 6 As shown, the data warehouse code viewing device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the data warehouse code viewing device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the data warehouse code viewing device to communicate wirelessly or wiredly with other devices to exchange data. While the figure shows data warehouse code viewing devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0122] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0123] The data warehouse code inspection device provided in this application, employing the data warehouse code inspection method described in the above embodiments, can solve the technical problem of low efficiency and accuracy in data warehouse code inspection in the prior art. Compared with the prior art, the beneficial effects of the data warehouse code inspection device provided in this application are the same as those of the data warehouse code inspection method provided in the above embodiments, and other technical features in this data warehouse code inspection device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0124] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0125] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0126] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0127] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0128] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0129] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the protection scope of the present invention.

Claims

1. A data warehouse code inspection method, characterized in that, The method includes: The code to be inspected is parsed to obtain multiple logical segments with different semantic functions; Based on a preset set of inspection rules, each logical segment is distributed to a corresponding rule detection queue, and the logical segments in each rule detection queue are split into units of statements to be detected. Concurrent processing is performed on each of the rule detection queues, and the large model is called to perform rule matching detection on the statement unit to be detected, so as to obtain multiple rule detection results; The detection results of each rule are aggregated to generate an inspection report corresponding to the code to be inspected.

2. The data warehouse code inspection method as described in claim 1, characterized in that, The steps of concurrently processing each of the rule detection queues, calling the large model to perform rule matching detection on the statement unit to be detected, and obtaining multiple rule detection results include: For each rule detection queue, determine the number of sub-detection queues for each rule detection queue based on the corresponding inspection rule type; According to the number of sub-detection queues, each rule detection queue is evenly divided into multiple sub-detection queues; The large model is invoked in parallel for each of the sub-detection queues. Based on the inspection rule type corresponding to each sub-detection queue, rule matching detection is performed on the statement unit to be detected to obtain the sub-rule detection results corresponding to each sub-detection queue. The detection results of all sub-rules corresponding to the same rule detection queue are summarized to obtain the rule detection results corresponding to each rule detection queue.

3. The data warehouse code inspection method as described in claim 2, characterized in that, The step of determining the number of sub-detection queues for each rule detection queue based on the inspection rule type corresponding to each rule detection queue includes: Identify the inspection rule type corresponding to each of the rule detection queues; The call frequency of each of the aforementioned inspection rule types in historical code inspection tasks is statistically analyzed, and the weight coefficient corresponding to each of the aforementioned inspection rule types is determined based on the call frequency. The call frequency is positively correlated with the weight coefficient. By combining the number of statement units to be detected in each rule detection queue and the weight coefficient of the corresponding inspection rule type, the number of sub-detection queues corresponding to each rule detection queue is determined.

4. The data warehouse code inspection method as described in claim 2, characterized in that, The step of concurrently calling the large model for each of the sub-detection queues, performing rule matching detection on the statement unit to be detected based on the inspection rule type corresponding to each sub-detection queue, and obtaining the sub-rule detection results corresponding to each sub-detection queue includes: Obtain the rule description information, input / output format description, and reference sample data for the inspection rule type corresponding to each sub-detection queue; Based on the rule description information, the input and output format specifications, and the reference sample data, a prompt word template corresponding to each of the inspection rule types is generated according to the preset template results. The large model is invoked based on each of the prompt word templates. The large model performs rule matching detection on the sentence unit to be detected according to the corresponding single inspection rule type, and obtains the sub-rule detection results corresponding to each of the sub-detection queues.

5. The data warehouse code inspection method as described in claim 1, characterized in that, The step of parsing the code to be inspected to obtain multiple logical fragments with different semantic functions includes: The code to be inspected is parsed, and the module separators in the code to be inspected are identified based on the parsing results. Based on the module separator, the code to be inspected is divided into a variable definition module, a temporary table definition module, and a business logic module. The variable definition module is stored as a string, and the temporary table definition module and the business logic module are stored as a list, resulting in multiple logical fragments with different semantic functions.

6. The data warehouse code inspection method as described in claim 1, characterized in that, The step of distributing each logical segment to a corresponding rule detection queue based on a preset inspection rule set, and splitting the logical segments in each rule detection queue into statement units to be detected, includes: Based on each inspection rule in the preset inspection rule set, extract the corresponding rule feature keywords; Traverse each of the logical segments and assign the logical segments containing the rule feature keywords to the corresponding rule detection queue; The logical segments in each rule detection queue are split into statements to obtain the statement units to be detected corresponding to each rule detection queue.

7. The data warehouse code inspection method as described in claim 1, characterized in that, The step of aggregating the detection results of each rule to generate an inspection report corresponding to the code to be inspected includes: The detection results of each rule are deduplicated to obtain the deduplicated rule detection results. The deduplicated rule detection results are merged to obtain merged rule detection results. The merged rule detection results are formatted and arranged according to a preset display format to generate an inspection report corresponding to the code to be inspected.

8. A data warehouse code viewing device, characterized in that, The device includes: The code parsing module is used to parse the code to be inspected and obtain multiple logical fragments with different semantic functions. The fragment splitting module is used to distribute each logical fragment to a corresponding rule detection queue based on a preset inspection rule set, and to split the logical fragments in each rule detection queue into statement units to be detected; The concurrent processing module is used to perform concurrent processing on each of the rule detection queues, call the large model to perform rule matching detection on the statement unit to be detected, and obtain multiple rule detection results; The result aggregation module is used to aggregate the detection results of each rule and generate an inspection report corresponding to the code to be inspected.

9. A data warehouse code inspection device, characterized in that, The device includes: a memory, a processor, and a data warehouse code inspection program stored on the memory and executable on the processor, the data warehouse code inspection program being configured to implement the steps of the data warehouse code inspection method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a data warehouse code inspection program, which, when executed by a processor, implements the steps of the data warehouse code inspection method as described in any one of claims 1 to 7.

Images