Code analysis method, device, electronic device and computer readable medium

Abstract

The embodiments of the present application provide a code analysis method, device, electronic device and computer-readable medium, which relate to the field of Internet technology. The method includes: receiving a code analysis request sent by a client through a data lake analysis service, wherein the code analysis request is used to indicate code analysis based on the carried code analysis requirement information; performing code analysis on the code in the code engineering file based on the code analysis requirement information through the data lake analysis service to obtain an analysis result that matches the code analysis requirement information. Through the embodiments of the present application, code analysis can be performed on the code in the code engineering file conveniently and effectively, thereby effectively improving the development efficiency and release efficiency of the code in the code engineering file.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of Internet technology, and in particular, to a code analysis method, device, electronic device, and computer-readable medium. Background Art

[0002] Most general-purpose large-scale code development languages ​​are born to solve complex problems. These complex problems are separated into independent modules through step-by-step abstraction, decoupling, modularization and other software engineering ideas to solve problems in various scenarios. With the development of the open source community, there are often a large number of open source libraries and open source codes to solve these problems. Software is constantly evolving and iterating with business needs, so a large number of versions have appeared. At the same time, the business scenarios that developers themselves have to face are also very complex, and they will also do a lot of modular splitting, referencing various different libraries and classes, functions, methods, etc. in the libraries. In actual work, developers often face many library dependency conflicts and are in pain and have no choice. For example, library A and library B both depend on library C, but they depend on different versions, and the two versions are very different. A piece of code depends on function f of library A, but the interface definition of function f is different from the parameters in the code. In addition, there are various other compatibility, same-name conflicts and other problems. In addition to dependency conflicts, there are many other language analysis needs. Therefore, how to effectively analyze the code has become a technical problem that needs to be solved.

[0003] In the prior art, the various problems described above often occur in the use scenarios of Java, a very general, object-oriented high-level language, and have even become typical problems of the Java language. Java is a pure object-oriented language that loads various class libraries and related dependencies at runtime. Therefore, it cannot be encountered at compile time, and it will always encounter such problems as conflicts of classes with the same name and different method signatures in classes of different versions at runtime. There are some code analysis tools on the market to analyze the code developed in Java. These tools are excellent tools in the industry and bring a good experience to developers. However, as tools, they have some shortcomings, mainly as follows: 1. Building relevant memory indexes through the preprocessing process to accelerate the analysis occupies a large amount of memory resources and computing resources, the preprocessing delay is relatively large, and the on-demand analysis and calculation to obtain the data that users want will be very slow each time The interaction speed is very poor, and the experience is very poor. 2. Many tools are not free. 3. These tools can only run on personal computers, and cannot execute analysis codes anytime and anywhere. 4. The analysis capabilities of the tools themselves are limited. 5. The very valuable result set after analysis is not easy to save, and the effect and value of the analysis data are discounted. 6. It is impossible to analyze multiple project files at the same time in one process, so it is difficult to do correlation analysis. Therefore, how to analyze the code conveniently and effectively has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The purpose of this application is to propose a code analysis method, device, electronic device and computer-readable medium to solve the technical problem of how to analyze code conveniently and effectively in the prior art.

[0005] According to a first aspect of an embodiment of the present application, a code analysis method is provided. The method comprises: receiving a code analysis request sent by a client through a data lake analysis service, wherein the code analysis request is used to indicate that code analysis is performed based on the carried code analysis requirement information; and performing code analysis on the code in the code engineering file based on the code analysis requirement information through the data lake analysis service to obtain an analysis result that matches the code analysis requirement information.

[0006] According to a second aspect of an embodiment of the present application, a code analysis device is provided. The device includes: a first receiving module, which is used to receive a code analysis request sent by a client through a data lake analysis service, wherein the code analysis request is used to indicate that code analysis is performed based on the carried code analysis requirement information; an analysis module, which is used to perform code analysis on the code in the code engineering file based on the code analysis requirement information through the data lake analysis service to obtain an analysis result that matches the code analysis requirement information.

[0007] According to the third aspect of the embodiment of the present application, a code analysis system is provided. The system includes: a client, a data lake analysis service connected to the client, and a data source connected to the data lake analysis service, wherein the client is used to generate a code analysis request based on the code analysis requirement information input by the user, and send the code analysis request to the data lake analysis service, wherein the code analysis request is used to indicate that code analysis is performed based on the carried code analysis requirement information; the code analysis request sent by the client is received through the data lake analysis service, and based on the code analysis requirement information, code analysis is performed on the code in the data table pulled from the data source to obtain an analysis result that matches the code analysis requirement information.

[0008] According to a fourth aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; and a computer-readable medium configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the code analysis method as described in the first aspect of the above embodiment.

[0009] According to a fifth aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the code analysis method described in the first aspect of the above embodiment is implemented.

[0010] According to the code analysis solution provided in the embodiment of the present application, a code analysis request sent by a client is received through a data lake analysis service, wherein the code analysis request is used to indicate that code analysis is to be performed based on the carried code analysis requirement information; and through the data lake analysis service, based on the code analysis requirement information carried in the code analysis request, code analysis is performed on the code in the code engineering file to obtain an analysis result that matches the code analysis requirement information. Compared with other existing methods, through the data lake analysis service, code analysis is performed on the code in the code engineering file based on the code analysis requirement information carried in the received code analysis request, so that code analysis can be performed on the code in the code engineering file conveniently and effectively, thereby effectively improving the development efficiency and release efficiency of the code in the code engineering file. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0012] Figure 1A A schematic diagram of the structure of a code analysis system provided in an embodiment of the present application;

[0013] Figure 1BThis is a flowchart of the steps of the code analysis method in Example 1 of the present application;

[0014] Figure 1C A schematic diagram of a scenario provided according to Embodiment 1 of the present application;

[0015] Figure 2A This is a flowchart of the steps of the code analysis method in Example 2 of this application;

[0016] Figure 2B A schematic diagram of a code analysis process provided according to Embodiment 2 of the present application;

[0017] Figure 3 This is a schematic diagram of the structure of the code analysis device in the third embodiment of the present application;

[0018] Figure 4 This is a schematic diagram of the structure of the code analysis device in the fourth embodiment of the present application;

[0019] Figure 5 This is a schematic diagram of the structure of an electronic device in Embodiment 5 of the present application;

[0020] Figure 6 This is the hardware structure of the electronic device in Example 6 of the present application. DETAILED DESCRIPTION

[0021] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only configured to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0022] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0023] The embodiment of the present application proposes a code analysis method, which can be based on a code analysis system including a client, a data lake analysis service that is communicatively connected to the client, and a data source that is communicatively connected to the data lake analysis service. The client is used to generate a code analysis request based on the code analysis requirement information input by the user, and send the code analysis request to the data lake analysis service, wherein the code analysis request is used to indicate that code analysis is to be performed based on the carried code analysis requirement information; the code analysis request sent by the client is received through the data lake analysis service, and based on the code analysis requirement information, code analysis is performed on the code in the data table pulled from the data source to obtain an analysis result that matches the code analysis requirement information. The data lake analysis service includes a front-end node pool composed of front-end nodes (also called front-end servers) and a compute node pool composed of compute nodes (also called compute servers).

[0024] See also Figure 1A As shown, it is a schematic diagram of the structure of a code analysis system provided by an embodiment of the present application. In the front-end node pool of the front-end node, one or more front-end nodes are included. Figure 1A Take three front-end nodes as an example. In the computing node pool of the computing node, there are one or more computing nodes. Figure 1A Take five computing nodes as an example. In actual applications, the number of front-end nodes may be increased or decreased, and the number of computing nodes may be increased or decreased.

[0025] Among them, the client can be used to analyze the code in the data source, such as an APP (Application) included in a terminal device (such as a PC (Personal Computer), a laptop, a mobile terminal, etc.), or a browser included in the terminal device. There is no specific restriction on the type of this client.

[0026] The load balancing device is used to load balance the code analysis requests. For example, after receiving a large number of code analysis requests, the load of these code analysis requests can be balanced to each front-end node. There is no restriction on this process.

[0027] Among them, the data source is used to store various types of data, and the data stored in the data source can be provided to the client or the data lake analysis service. The type of data stored in the data source is not limited in the embodiments of the present application, such as user data, product data, map data, video data, image data, audio data, code engineering files, etc. The data source can be an object storage system (OSS), a table storage service, a cloud database, etc., and the embodiments of the present application do not impose any restrictions on this.

[0028] Among them, multiple front-end nodes in the front-end node pool can be used to provide the same function, that is, to provide peer-to-peer services. Specifically, the front-end node is used to receive the code analysis request sent by the client, and perform SQL (Structured Query Language) analysis on the code analysis request, generate a distributed execution plan using the SQL analysis result, and distribute the distributed execution plan to the computing node. The code analysis request is used to request analysis of the corresponding code. Then, the front-end node is also used to receive the code analysis result returned by the computing node, and send the code analysis result to the client.

[0029] Among them, multiple computing nodes in the computing node pool can be used to provide the same function, that is, to provide peer-to-peer services. Specifically, the computing node is used to receive the distributed execution plan distributed by the front-end node, and analyze the code corresponding to the code analysis request pulled from the data source based on the distributed execution plan to obtain the code analysis result, and send the code analysis result to the front-end node.

[0030] Based on the above code analysis system, see Figure 1B As shown, it is a flowchart of the steps of the code analysis method provided in the first embodiment of the present application. The method can be applied to an application or system with a data lake analysis service, and the embodiment of the present application does not limit this. Specifically, the method may include the following steps:

[0031] In step S101, a code analysis request sent by a client is received through the data lake analysis service.

[0032] In an embodiment of the present application, the data lake analysis service is a new cloud data computing and analysis product. Provide users with serverless interactive query and analysis services on the cloud. Without ETL, this service can be used to query and analyze data in object storage systems, table storage services, and other cloud storage and cloud databases directly on the cloud through standard SQL, and seamlessly integrate business analysis tools and applications. The code analysis request is used to indicate code analysis based on the carried code analysis requirement information, which can be an SQL code analysis request. The code analysis requirement information can be understood as information used to express the user's analysis requirements for the code. For example, the user can use the SQL language to write various operators and various functions to express the analysis requirements for the code. For example, the code analysis requirement information can be to find all unreasonable naming specifications, find all methods with too long parameter lists, find repeated code segments, find complete class path conflicts, find classes and class names and method names with the same name methods, and count all dependent classes (deduplication), etc. Among them, the code can be code written in Java, code written in CPP, and code written in C, etc. It should be understood that the above description is merely exemplary and the embodiments of the present application do not impose any limitation thereto.

[0033] In some optional embodiments, when receiving a code analysis request sent by a client through the data lake analysis service, the code analysis request sent by the client can be received through a front-end node in the data lake analysis service. It can be understood that the above description is only exemplary and the embodiments of the present application do not make any limitation to this.

[0034] In step S102, the data lake analysis service performs code analysis on the code in the code engineering file based on the code analysis requirement information to obtain an analysis result that matches the code analysis requirement information.

[0035] In some optional embodiments, when code analysis is performed on the code in the code engineering file based on the code analysis requirement information through the data lake analysis service, the code in the code engineering file is parsed through the data lake analysis service to obtain the grammatical structure information of the code in the code engineering file; and the grammatical structure information of the code in the code engineering file is analyzed through the data lake analysis service based on the code analysis requirement information to obtain an analysis result that matches the code analysis requirement information. Thus, by analyzing the grammatical structure information of the code in the code engineering file based on the code analysis requirement information through the data lake analysis service, an analysis result that matches the code analysis requirement information can be accurately obtained. It will be understood that the above description is merely exemplary and the embodiments of the present application do not impose any limitation on this.

[0036] In a specific example, when the code in the code engineering file is parsed through the data lake analysis service, the corresponding code parsing tool is called through the data lake analysis service to parse the code in the code engineering file to obtain the grammatical structure information of the code in the code engineering file. It can be understood that the above description is only exemplary and the embodiments of the present application do not make any limitation to this.

[0037] In a specific example, when the corresponding code parsing tool is called to parse the code in the code engineering file through the data lake analysis service, the corresponding code parsing tool is called to parse the code in the code engineering file through the computing node in the data lake analysis service to obtain the grammatical structure information of the code in the code engineering file. The grammatical structure information may include code fields and code content corresponding to the code fields. It is understandable that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0038] In a specific example, when the syntax structure information of the code in the code engineering file is analyzed based on the code analysis requirement information by the data lake analysis service, the syntax structure information of the code in the code engineering file is analyzed based on the code analysis requirement information by the computing node in the data lake analysis service to obtain an analysis result that matches the code analysis requirement information. It is understandable that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0039] In some optional embodiments, when the code in the code engineering file is analyzed based on the code analysis requirement information through the data lake analysis service, the code analysis request is parsed through the front-end node in the data lake analysis service to obtain the parsing result of the code analysis request, and based on the parsing result, a distributed execution plan matching the code analysis requirement information is generated, and then the distributed execution plan is distributed to the computing nodes in the data lake analysis service for execution; the computing nodes in the data lake analysis service receive the distributed execution plan distributed by the front-end node, and based on the distributed execution plan, the code in the code engineering file is parsed to obtain the grammatical structure information of the code in the code engineering file, and the grammatical structure information of the code in the code engineering file is analyzed to obtain the analysis result matching the code analysis requirement information. It can be understood that the above description is only exemplary, and the embodiments of the present application do not impose any limitation on this.

[0040] In one example, the execution order of the above step S101 and step S102 is just an example given for the convenience of description. In practical applications, the execution order between the steps can also be changed, and there is no limitation on this execution order. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0041] In actual applications, users can import the original code engineering files into the data lake analysis service through the client, and then use the data lake analysis service to parse the syntax structure information of the code from the original code engineering files, and then use the SQL engine technology (distributed computing nodes) of the data lake analysis service to perform analysis. Users can use the SQL language to write various operators and functions to express analysis requirements, use the parallel computing capabilities of the data lake analysis service to achieve rapid analysis, use the multi-data source capabilities of the data lake analysis service to connect other related data for correlation analysis, and use the data lake analysis service to output the analysis results to any place.

[0042] For various languages ​​for writing code, many language designs and language concepts are interlinked. For example, taking CPP and Java, two object-oriented languages, as an example, they have concepts such as classes, attributes, various types, methods / functions, method signatures, code segments, keywords, parent classes and child classes, abstract classes / virtual classes, as well as code structures such as loops, branches, conditions, exceptions, and function returns. Moreover, through the parser tool of the corresponding language, the grammatical structure information can be fully obtained. Therefore, the file analysis, dependency analysis, and code content analysis of codes written in various languages ​​can all be solved and accelerated using similar methods and ideas provided in the embodiments of the present application, thereby greatly improving the efficiency of code development and release, reducing time costs, and expanding the value of analysis.

[0043] A scenario of the above process is shown as follows Figure 1C As shown, a data lake analysis service platform (providing Figure 1A The platform can be used to provide data lake analysis services. Figure 1A As shown in , it is implemented through multiple nodes. Figure 1CThe client sends a code analysis request to the data lake analysis service platform, and the data lake analysis service platform receives the code analysis request through its data lake analysis service; then, according to the code analysis requirement information, it sends a data pull request to OSS to pull the required code engineering file from OSS, and then the data lake analysis service performs code analysis on the code engineering file to obtain the corresponding analysis result; then, the analysis result is fed back to the client.

[0044] According to the code analysis method provided in the embodiment of the present application, a code analysis request sent by a client is received through a data lake analysis service, wherein the code analysis request is used to indicate that code analysis is to be performed based on the carried code analysis requirement information; and through the data lake analysis service, based on the code analysis requirement information carried in the code analysis request, code analysis is performed on the code in the code engineering file to obtain an analysis result that matches the code analysis requirement information. Compared with other existing methods, through the data lake analysis service, code analysis is performed on the code in the code engineering file based on the code analysis requirement information carried in the received code analysis request, so that code analysis can be performed on the code in the code engineering file conveniently and effectively, thereby effectively improving the development efficiency and release efficiency of the code in the code engineering file.

[0045] The code analysis method of this embodiment can be executed by any appropriate device with data processing capabilities, including but not limited to: cameras, terminals, mobile terminals, PCs, servers, vehicle-mounted devices, entertainment devices, advertising devices, personal digital assistants (PDAs), tablet computers, laptop computers, handheld game consoles, smart glasses, smart watches, wearable devices, virtual display devices or display enhancement devices (such as Google Glass, Oculus Rift, Hololens, Gear VR), etc.

[0046] Reference Figure 2A , showing a step flow chart of the code analysis method of embodiment 2 of the present application.

[0047] Specifically, the code analysis method provided in this embodiment includes the following steps:

[0048] In step S201, a data table creation request sent by the client is received through the data lake analysis service.

[0049] In the embodiment of the present application, the data table creation request is used to instruct to create a data table for code analysis of the code in the code engineering file based on the carried data table type information and the address information of the code engineering file. It can be understood that the above description is only exemplary and the embodiment of the present application does not make any limitation to this.

[0050] In some optional embodiments, when the data lake analysis service receives the data table creation request sent by the client, the data table creation request sent by the client is received by the front-end node in the data lake analysis service. It can be understood that the above description is only exemplary and the embodiments of the present application do not make any limitation to this.

[0051] In a specific example, after the user uploads the code engineering file to the relevant data source through the client, if the user has the need to perform code analysis on the code engineering file uploaded to the relevant data source, the user can click the data table creation control of the client. In response to the click operation on the data table creation control of the client, the client sends a data table creation request to the data lake analysis service associated with it. Then, click the code analysis control of the client again. In response to the click operation on the code analysis control of the client, the client sends a code analysis request to the data lake analysis service associated with it. It can be understood that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0052] In step S202, the data table is created through the data lake analysis service based on the data table type information and the address information carried in the data table creation request, so as to perform code analysis on the code in the code engineering file.

[0053] In the embodiment of the present application, the data table can be created by the front-end node in the data lake analysis service based on the data table type information and the address information carried by the data table creation request, so as to perform code analysis on the code in the code engineering file. It can be understood that the above description is only exemplary and the embodiment of the present application does not make any limitation to this.

[0054] In some optional embodiments, when creating the data table through the data lake analysis service based on the data table type information and the address information carried by the data table creation request, a first data table with a code field is created through the data lake analysis service based on the data table type information carried by the data table creation request; the code engineering file is obtained through the data lake analysis service based on the address information carried by the data table creation request, and the code engineering file is parsed to obtain the grammatical structure information of the code in the code engineering file; through the data lake analysis service, a mapping relationship between the code field in the first data table and the grammatical structure information is established, and a second data table having both the code field and the code content of the code field is obtained, so as to perform code analysis on the code in the code engineering file. Thus, by establishing a mapping relationship between the code field in the first data table and the grammatical structure information, a second data table having both the code field and the code content of the code field can be obtained, so as to be used for subsequent rapid code analysis of the code in the code engineering file. It can be understood that the above description is only exemplary, and the embodiments of the present application do not make any limitation to this.

[0055] In some optional embodiments, when creating a first data table with a code field through the data lake analysis service based on the data table type information carried by the data table creation request, the data lake analysis service queries the mapping relationship between the pre-configured data table type and the data table structure based on the data table type information carried by the data table creation request to obtain the data table structure information corresponding to the data table type information, and determines the first data table with the code field based on the data table structure information. Thereby, through the mapping relationship between the pre-configured data table type and the data table structure, the first data table with the code field can be quickly created without the need to additionally write or generate a table creation statement to create the first data table with the code field. It will be understood that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0056] In a specific example, when creating a first data table with a code field through the data lake analysis service based on the data table type information carried by the data table creation request, the mapping relationship between the pre-configured data table type and the data table structure is queried through the front-end node in the data lake analysis service based on the data table type information carried by the data table creation request to obtain the data table structure information corresponding to the data table type information, and based on the data table structure information, the first data table with the code field is determined. It can be understood that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0057] In some optional embodiments, when parsing the code engineering file to obtain the grammatical structure information of the code in the code engineering file, the corresponding code parsing tool is called through the front-end node in the data lake analysis service to parse the code engineering file to obtain the grammatical structure information of the code in the code engineering file. It can be understood that the above description is only exemplary and the embodiments of the present application do not make any limitation to this.

[0058] In some optional embodiments, when establishing a mapping relationship between the code field in the first data table and the grammatical structure information through the data lake analysis service, the code field in the first data table is compared with the code field in the grammatical structure information through the front-end node in the data lake analysis service. If the code field in the first data table is the same as the code field in the grammatical structure information, a mapping relationship is established between the code field in the first data table and the code content corresponding to the code field in the grammatical structure information, and a second data table having both the code field and the code content of the code field is obtained, so as to perform code analysis on the code in the code engineering file. It can be understood that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0059] In some optional embodiments, after obtaining the second data table having both the code field and the code content of the code field, the method further includes: using the data lake analysis service, using a preset data format, storing the second data table having both the code field and the code content of the code field in the corresponding data source. Among them, the preset data format may be xml format, json format, Parquet format, arrow format, etc. Thereby, by adopting a preset data format, storing the second data table having both the code field and the code content of the code field in the corresponding data source, it is possible to quickly and conveniently perform code analysis on the code in the code engineering file based on the second data table. It will be understood that the above description is only exemplary, and the embodiments of the present application do not impose any limitation on this.

[0060] In step S203, a code analysis request sent by the client is received through the data lake analysis service.

[0061] Since the specific implementation of step S203 is similar to the specific implementation of step S101 above, it will not be repeated here.

[0062] In step S204, code analysis is performed on the code in the data table based on the code analysis requirement information through the data lake analysis service to obtain an analysis result that matches the code analysis requirement information.

[0063] In some optional embodiments, when the code in the code engineering file is analyzed by the data lake analysis service based on the code analysis requirement information, the code in the second data table pulled from the data source is analyzed by the data lake analysis service based on the code analysis requirement information to obtain an analysis result that matches the code analysis requirement information. Thus, by performing code analysis on the code in the second data table pulled from the data source, code analysis can be quickly implemented. It will be understood that the above description is only exemplary and the embodiments of the present application do not limit this in any way.

[0064] In a specific example, when the code analysis is performed on the code in the second data table pulled from the data source based on the code analysis requirement information through the data lake analysis service, the code analysis request is parsed through the front-end node in the data lake analysis service to obtain the parsing result of the code analysis request, and based on the parsing result, a distributed execution plan matching the code analysis requirement information is generated, and then the distributed execution plan is distributed to the computing nodes in the data lake analysis service for execution; the computing nodes in the data lake analysis service receive the distributed execution plan distributed by the front-end node, and based on the distributed execution plan, code analysis is performed on the code in the second data table pulled from the data source to obtain the analysis result matching the code analysis requirement information. It can be understood that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0065] In some optional embodiments, after obtaining the analysis results that match the code analysis requirement information, the method further includes: dumping the analysis results to the target data source through the data lake analysis service. Thus, the analysis results can be dumped to the target data source through the data lake analysis service. It is to be understood that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0066] In one example, the execution order of the above steps S201 to S204 is just an example given for the convenience of description. In practical applications, the execution order between the steps can also be changed, and there is no limitation on this execution order. Moreover, in other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification, and the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0067] In a specific example, Figure 2B As shown, the user uploads the code engineering file to the object storage system through the client, and the object storage system returns the address information of the code engineering file to the client. In response to the click operation of the data table creation control for the client, the client sends a data table creation request to the data lake analysis service associated with it. The data lake analysis service receives the data table creation request, and based on the data table creation request, parses the corresponding code engineering file obtained from the object storage system to obtain the grammatical structure information of the code in the code engineering file, and creates a data table with code fields, and then establishes a mapping relationship between the code fields and the grammatical structure information in the data table to obtain a data table with both code fields and code contents corresponding to the code fields. After obtaining a data table with both code fields and code contents corresponding to the code fields, the data lake analysis service stores the data table in the object storage system in a specific data format. When the data lake analysis service receives the code analysis request sent by the client, the data lake analysis service pulls the corresponding data table from the object storage system for distributed computing to obtain an analysis result that matches the code analysis requirement information carried by the code analysis request, and returns the analysis result to the client. Of course, the analysis result can also be dumped in the object storage system. By pre-creating a data table with both code fields and code content corresponding to the code fields, the data lake analysis service can perform simple and fast calculations and analysis. The entire computing architecture of the data lake analysis service is distributed, so whether it is pulling data, associative calculations, complex calculations, rich functions, etc., it is very efficient, and based on the relationship of the database, it can easily express various complex calculations such as transformations, mappings, associations, sorting, and grouping. It can be understood that the above description is only exemplary, and the embodiments of this application do not impose any limitations on this.

[0068] In actual applications, after the data lake analysis service creates a data table in advance, users can use the database method to quickly analyze the project code, such as dependencies, conflicts, irregular naming, duplication, etc. The SQL language has strong expressive power and can meet a variety of needs. In addition, the distributed computing in the data lake analysis service can realize very complex, very resource-intensive, and large-scale input analysis and calculations. At the same time, it can connect and integrate various platforms to participate in the calculation, greatly improving the analysis ability and the value after the analysis.

[0069] According to the code analysis method provided in the embodiment of the present application, a data table creation request sent by a client is received through a data lake analysis service, wherein the data table creation request is used to indicate creation of a data table for performing code analysis on the code in the code engineering file based on the carried data table type information and the address information of the code engineering file, and through the data lake analysis service, based on the data table type information and address information carried in the data table creation request, the data table is created for performing code analysis on the code in the code engineering file, and then through the data lake analysis service, a code analysis request sent by the client is received, and then through the data lake analysis service, based on the code analysis requirement information carried in the code analysis request, code analysis is performed on the code in the data table to obtain an analysis result matching the code analysis requirement information. Compared with other existing methods, through the data lake analysis service, code analysis is performed on the code in the data table for performing code analysis on the code in the code engineering file based on the code analysis requirement information carried in the received code analysis request, so that the code in the code engineering file can be further quickly and effectively analyzed, thereby effectively improving the development efficiency and release efficiency of the code in the code engineering file.

[0070] The code analysis method of this embodiment can be executed by any appropriate device with data processing capabilities, including but not limited to: cameras, terminals, mobile terminals, PCs, servers, vehicle-mounted devices, entertainment devices, advertising devices, personal digital assistants (PDAs), tablet computers, laptop computers, handheld game consoles, smart glasses, smart watches, wearable devices, virtual display devices or display enhancement devices (such as Google Glass, Oculus Rift, Hololens, Gear VR), etc.

[0071] Reference Figure 3 , showing a schematic diagram of the structure of the code analysis device in Example 3 of the present application.

[0072] The code analysis device provided in this embodiment includes: a first receiving module 301, used to receive a code analysis request sent by a client through a data lake analysis service, wherein the code analysis request is used to indicate code analysis based on the carried code analysis requirement information; an analysis module 302, used to perform code analysis on the code in the code engineering file based on the code analysis requirement information through the data lake analysis service to obtain an analysis result matching the code analysis requirement information.

[0073] The code analysis device of this embodiment is used to implement the corresponding code analysis methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0074] Reference Figure 4 , showing a schematic diagram of the structure of the code analysis device in Example 4 of the present application.

[0075] The code analysis device provided in this embodiment includes: a first receiving module 403, which is used to receive a code analysis request sent by a client through a data lake analysis service, wherein the code analysis request is used to indicate code analysis based on the carried code analysis requirement information; an analysis module 404, which is used to perform code analysis on the code in the code engineering file based on the code analysis requirement information through the data lake analysis service to obtain an analysis result matching the code analysis requirement information.

[0076] Optionally, before the first receiving module 403, the device also includes: a second receiving module 401, used to receive a data table creation request sent by the client through the data lake analysis service, wherein the data table creation request is used to indicate creation of a data table for performing code analysis on the code in the code engineering file based on the carried data table type information and the address information of the code engineering file; a creation module 402, used to create the data table through the data lake analysis service based on the data table type information and the address information carried in the data table creation request, so as to perform code analysis on the code in the code engineering file.

[0077] Optionally, the creation module 402 includes: a creation sub-module 4021, which is used to create a first data table having a code field based on the data table type information carried by the data table creation request through the data lake analysis service; a parsing sub-module 4022, which is used to obtain the code engineering file through the data lake analysis service based on the address information carried by the data table creation request, and parse the code engineering file to obtain the grammatical structure information of the code in the code engineering file; an establishment sub-module 4023, which is used to establish a mapping relationship between the code field in the first data table and the grammatical structure information through the data lake analysis service, and obtain a second data table having both the code field and the code content of the code field, so as to perform code analysis on the code in the code engineering file.

[0078] Optionally, the creation submodule 4021 is specifically used to: through the data lake analysis service, based on the data table type information carried by the data table creation request, query the mapping relationship between the pre-configured data table type and the data table structure to obtain the data table structure information corresponding to the data table type information, and based on the data table structure information, determine the first data table having the code field.

[0079] Optionally, after the establishment submodule 4023, the creation module 402 also includes: a storage submodule 4024, which is used to store the second data table having both the code field and the code content of the code field in a corresponding data source through the data lake analysis service and in a preset data format.

[0080] Optionally, the analysis module 404 includes: an analysis submodule 4041, which is used to perform code analysis on the code in the second data table pulled from the data source based on the code analysis requirement information through the data lake analysis service to obtain an analysis result that matches the code analysis requirement information.

[0081] Optionally, the analysis submodule 4041 is specifically used to: parse the code analysis request through the front-end node in the data lake analysis service to obtain the parsing result of the code analysis request, and based on the parsing result, generate a distributed execution plan that matches the code analysis requirement information, and then distribute the distributed execution plan to the computing nodes in the data lake analysis service for execution; receive the distributed execution plan distributed by the front-end node through the computing nodes in the data lake analysis service, and based on the distributed execution plan, perform code analysis on the code in the second data table pulled from the data source to obtain an analysis result that matches the code analysis requirement information.

[0082] Optionally, after the analysis module 404, the device further includes: a dump module 405, configured to dump the analysis result to a target data source through the data lake analysis service.

[0083] The code analysis device of this embodiment is used to implement the corresponding code analysis methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0084] Figure 5 This is a schematic diagram of the structure of an electronic device in Embodiment 5 of the present application; the electronic device may include:

[0085] One or more processors 501;

[0086] The computer readable medium 502 may be configured to store one or more programs,

[0087] When the one or more programs are executed by the one or more processors, the one or more processors implement the code analysis method as described in the above-mentioned embodiment 1 or embodiment 2.

[0088] Figure 6 The hardware structure of the electronic device in the sixth embodiment of the present application; Figure 6 As shown, the hardware structure of the electronic device may include: a processor 601, a communication interface 602, a computer readable medium 603 and a communication bus 604;

[0089] The processor 601, the communication interface 602, and the computer readable medium 603 communicate with each other via a communication bus 604;

[0090] Optionally, the communication interface 602 may be an interface of a communication module, such as an interface of a GSM module;

[0091] Among them, the processor 601 can be specifically configured to: receive a code analysis request sent by a client through a data lake analysis service, wherein the code analysis request is used to instruct code analysis based on the carried code analysis requirement information; through the data lake analysis service, based on the code analysis requirement information, perform code analysis on the code in the code engineering file to obtain an analysis result that matches the code analysis requirement information.

[0092] The processor 601 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It may implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0093] The computer readable medium 603 can be, but is not limited to, a random access memory medium (RAM), a read-only memory medium (ROM), a programmable read-only memory medium (PROM), an erasable programmable read-only memory medium (EPROM), an electrically erasable read-only memory medium (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

[0094] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code configured to execute the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are executed. It should be noted that the computer-readable medium described in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access storage medium (RAM), a read-only storage medium (ROM), an erasable programmable read-only storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only storage medium (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, an apparatus or a device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program configured to be used by or in combination with an instruction execution system, an apparatus or a device. The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0095] Computer program code configured to perform the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0096] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of the code contains one or more executable instructions configured to implement the specified logical function. There are specific sequential relationships in the above-mentioned specific embodiments, but these sequential relationships are only exemplary. When the specific implementation is implemented, these steps may be less, more, or the execution order may be adjusted. That is, in some implementations as replacements, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0097] The modules involved in the embodiments of the present application may be implemented in software or in hardware. The modules described may also be provided in a processor. For example, they may be described as follows: a processor includes a first receiving module and an analysis module. The names of these modules do not, in some cases, constitute limitations on the modules themselves. For example, the first receiving module may also be described as a "module for receiving code analysis requests sent by a client through a data lake analysis service."

[0098] As another aspect, the present application further provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the code analysis method described in the above-mentioned embodiment 1 or embodiment 2 is implemented.

[0099] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the device, the device: receives a code analysis request sent by a client through a data lake analysis service, wherein the code analysis request is used to indicate code analysis based on the carried code analysis requirement information; through the data lake analysis service, based on the code analysis requirement information, performs code analysis on the code in the code engineering file to obtain an analysis result that matches the code analysis requirement information.

[0100] The expressions "first", "second", "the first" or "the second" used in various embodiments of the present disclosure may modify various components regardless of order and / or importance, but these expressions do not limit the corresponding components. The above expressions are only configured for the purpose of distinguishing an element from other elements. For example, a first user device and a second user device represent different user devices, although both are user devices. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the present disclosure.

[0101] When one element (e.g., a first element) is referred to as being "(operably or communicatively) coupled" or "(operably or communicatively) coupled to" or "connected to" another element (e.g., a second element), it is understood that the one element is directly connected to the other element or the one element is indirectly connected to the other element via yet another element (e.g., a third element). Conversely, it is understood that when an element (e.g., a first element) is referred to as being "directly connected" or "directly coupled" to another element (the second element), no element (e.g., a third element) is interposed between the two.

[0102] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A code analysis method, the method comprising: Receiving, through the data lake analysis service, a data table creation request sent by a client, wherein the data table creation request is used to instruct to create a data table for performing code analysis on the code in the code engineering file based on the carried data table type information and address information of the code engineering file; The data table is created by the data lake analysis service based on the data table type information and the address information carried in the data table creation request, so as to perform code analysis on the code in the code engineering file, including: Creating, by the data lake analysis service, a first data table having a code field based on the data table type information carried in the data table creation request; By using the data lake analysis service, based on the address information carried in the data table creation request, the code engineering file is obtained, and the code engineering file is parsed to obtain grammatical structure information of the code in the code engineering file; By using the data lake analysis service, a mapping relationship between the code field in the first data table and the grammatical structure information is established to obtain a second data table having both the code field and the code content of the code field, so as to perform code analysis on the code in the code engineering file; Receiving, through the data lake analysis service, a code analysis request sent by a client, wherein the code analysis request is used to instruct to perform code analysis based on the carried code analysis requirement information; Through the data lake analysis service, based on the code analysis requirement information, code analysis is performed on the code in the code engineering file to obtain an analysis result that matches the code analysis requirement information.

2. The method according to claim 1, wherein: The step of creating, by the data lake analysis service, a first data table having a code field based on the data table type information carried in the data table creation request includes: Through the data lake analysis service, based on the data table type information carried in the data table creation request, a mapping relationship between a pre-configured data table type and a data table structure is queried to obtain data table structure information corresponding to the data table type information, and based on the data table structure information, the first data table having the code field is determined.

3. The method according to claim 1, wherein: After obtaining the second data table having both the code field and the code content of the code field, the method further comprises: Through the data lake analysis service, a second data table having both the code field and the code content of the code field is stored in a corresponding data source using a preset data format.

4. The method according to claim 3, wherein: The performing code analysis on the code in the code engineering file based on the code analysis requirement information by the data lake analysis service to obtain an analysis result matching the code analysis requirement information includes: Through the data lake analysis service, based on the code analysis requirement information, code analysis is performed on the code in the second data table pulled from the data source to obtain an analysis result that matches the code analysis requirement information.

5. The method according to claim 4, wherein: The performing code analysis on the code in the second data table pulled from the data source based on the code analysis requirement information by the data lake analysis service to obtain an analysis result matching the code analysis requirement information includes: Parsing the code analysis request through the front-end node in the data lake analysis service to obtain a parsing result of the code analysis request, and generating a distributed execution plan matching the code analysis requirement information based on the parsing result, and then distributing the distributed execution plan to the computing nodes in the data lake analysis service for execution; The distributed execution plan distributed by the front-end node is received through the computing node in the data lake analysis service, and based on the distributed execution plan, code analysis is performed on the code in the second data table pulled from the data source to obtain an analysis result that matches the code analysis requirement information.

6. The method according to any one of claims 1 to 5, wherein: After obtaining the analysis result matching the code analysis requirement information, the method further includes: The analysis results are transferred to the target data source through the data lake analysis service.

7. A code analysis device, comprising: A second receiving module is used to receive a data table creation request sent by a client through a data lake analysis service, wherein the data table creation request is used to indicate the creation of a data table for performing code analysis on the code in the code engineering file based on the carried data table type information and address information of the code engineering file; A creation module, used to create the data table through the data lake analysis service based on the data table type information and the address information carried in the data table creation request, so as to perform code analysis on the code in the code engineering file, including: A creation submodule, configured to create, through the data lake analysis service, a first data table having a code field based on the data table type information carried in the data table creation request; a parsing submodule, configured to obtain the code engineering file through the data lake analysis service based on the address information carried in the data table creation request, and parse the code engineering file to obtain grammatical structure information of the code in the code engineering file; Establishing a submodule, used to establish a mapping relationship between the code field in the first data table and the grammatical structure information through the data lake analysis service, and obtain a second data table having both the code field and the code content of the code field, so as to perform code analysis on the code in the code engineering file; A first receiving module is used to receive a code analysis request sent by a client through a data lake analysis service, wherein the code analysis request is used to instruct to perform code analysis based on the carried code analysis requirement information; The analysis module is used to perform code analysis on the code in the code engineering file based on the code analysis requirement information through the data lake analysis service to obtain an analysis result matching the code analysis requirement information.

8. The device according to claim 7, wherein: The creation of submodules is specifically used for: Through the data lake analysis service, based on the data table type information carried in the data table creation request, a mapping relationship between a pre-configured data table type and a data table structure is queried to obtain data table structure information corresponding to the data table type information, and based on the data table structure information, the first data table having the code field is determined.

9. The device according to claim 7, wherein: After the submodule is established, the creation module further includes: The storage submodule is used to store the second data table having both the code field and the code content of the code field in a corresponding data source by using the data lake analysis service and adopting a preset data format.

10. The device according to claim 9, wherein: The analysis module comprises: The analysis submodule is used to perform code analysis on the code in the second data table pulled from the data source based on the code analysis requirement information through the data lake analysis service to obtain an analysis result matching the code analysis requirement information.

11. The device according to claim 10, wherein: The analysis submodule is specifically used for: Parsing the code analysis request through the front-end node in the data lake analysis service to obtain a parsing result of the code analysis request, and generating a distributed execution plan matching the code analysis requirement information based on the parsing result, and then distributing the distributed execution plan to the computing nodes in the data lake analysis service for execution; The distributed execution plan distributed by the front-end node is received through the computing node in the data lake analysis service, and based on the distributed execution plan, code analysis is performed on the code in the second data table pulled from the data source to obtain an analysis result that matches the code analysis requirement information.

12. The device according to any one of claims 7 to 11, wherein: After the analysis module, the device further includes: The dump module is used to dump the analysis result to the target data source through the data lake analysis service.

13. A code analysis system implemented using the method according to any one of claims 1 to 6, the system comprising: A client, a data lake analysis service connected to the client, and a data source connected to the data lake analysis service. The client is used to generate a code analysis request based on the code analysis requirement information input by the user, and send the code analysis request to the data lake analysis service, wherein the code analysis request is used to instruct to perform code analysis based on the carried code analysis requirement information; The code analysis request sent by the client is received through the data lake analysis service, and based on the code analysis requirement information, code analysis is performed on the code in the data table pulled from the data source to obtain an analysis result that matches the code analysis requirement information.

14. An electronic device, comprising: one or more processors; A computer readable medium configured to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the code analysis method according to any one of claims 1 to 6.

15. A computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the code analysis method according to any one of claims 1 to 6 is implemented.

Images