Conflict detection method, device and storage medium for big data applications

Abstract

The present application discloses a conflict detection method, device and storage medium for big data applications. The conflict detection method for big data applications includes: obtaining first relevant information of each dependency package corresponding to each basic component in the big data application and second relevant information of each third-party dependency package in the big data application; determining whether there is a dependency package conflict based on each first relevant information and each second relevant information; in response to the existence of a dependency package conflict, issuing a prompt message, the prompt message including the conflict cause and / or conflict solution of the dependency package conflict. The above scheme can improve the conflict detection efficiency of big data applications.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a conflict detection method, device and storage medium for big data applications. Background Art

[0002] When developing big data applications (Spark tasks, Flink tasks, etc.), it is inevitable to reference third-party toolkits (such as log packages, XML toolkits, JSON toolkits, etc.). These toolkits themselves and their dependencies are likely to conflict with Spark or Flink dependency packages, causing task failure.

[0003] At this time, a large amount of conflict package analysis and package sorting work is often required, which greatly affects development efficiency. In addition, there are currently no direct and effective big data application development conflict detection tools on the market. Summary of the invention

[0004] The present application at least provides a conflict detection method, device and storage medium for big data applications.

[0005] The present application provides a conflict detection method for big data applications, including: obtaining first relevant information of each dependency package corresponding to each basic component in the big data application and second relevant information of each third-party dependency package in the big data application; determining whether there is a dependency package conflict based on each first relevant information and each second relevant information; in response to the existence of a dependency package conflict, issuing a prompt information, the prompt information including the conflict cause and / or conflict resolution method of the dependency package conflict.

[0006] The present application provides a conflict detection device for a big data application, including: an information acquisition module, a conflict detection module and a prompt module; the information acquisition module is used to obtain first relevant information of each dependency package corresponding to each basic component in the big data application and second relevant information of each third-party dependency package in the big data application; the conflict detection module is used to determine whether there is a dependency package conflict based on each first relevant information and each second relevant information; the prompt module is used to issue a prompt message in response to the existence of a dependency package conflict, the prompt message including the conflict cause and / or conflict solution of the dependency package conflict.

[0007] The present application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-mentioned conflict detection method for big data applications.

[0008] The present application provides a computer-readable storage medium having program instructions stored thereon, which implement the above-mentioned conflict detection method for big data applications when the program instructions are executed by a processor.

[0009] The above scheme, during the conflict detection process of big data applications, can determine the conflicting dependency packages in the big data application based on the first relevant information and the second relevant information of the third-party dependency packages by obtaining the first relevant information of each dependency package corresponding to each basic component of the big data application, thereby issuing the cause of the conflict and / or the conflict solution, thereby improving the efficiency of conflict detection.

[0010] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0012] Figure 1 It is a flow chart of an embodiment of a conflict detection method for big data application of the present application;

[0013] Figure 2 yes Figure 1 Schematic diagram of the sub-process of step S12;

[0014] Figure 3 It is another flow chart showing an embodiment of a conflict detection method for big data application of the present application;

[0015] Figure 4 It is another flow chart showing an embodiment of the conflict detection method for big data application of the present application;

[0016] Figure 5 It is a structural diagram of an embodiment of a conflict detection device for big data applications of the present application;

[0017] Figure 6 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0018] Figure 7 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0019] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0020] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0021] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0022] When developing big data applications (Spark tasks, Flink tasks, etc.), third-party toolkits (such as log packages, XML toolkits, json toolkits, etc.) are inevitably referenced. These toolkits themselves and their dependencies are likely to conflict with Spark or Flink's dependency packages, causing task failures. At this time, if a large amount of conflict package analysis and package sorting work is performed, the development efficiency will be greatly affected. In addition, there is currently no direct and effective big data application development conflict detection tool on the market.

[0023] How to accurately find conflicting items and how to package them reasonably? Even more seriously, after developers have handled some dependency conflicts, they only use the test environment as the version baseline package if there is no error, leaving the potential conflict on site. As a result, on-site big data tasks report errors due to dependency conflicts, seriously affecting the stability of big data products and even causing economic losses.

[0024] To sum up, when packaging a project, how to accurately find the conflicting items has become an urgent, yet helpless and difficult task.

[0025] In this regard, the present application provides a conflict detection method for big data applications, which can detect conflicts between dependency packages of each basic component in the big data application and third-party dependency packages, and give prompt information. In the present application, the execution subject for implementing the conflict detection method for big data applications described in the present application can be a conflict detection device, electronic device, etc. for big data applications. For example, the conflict detection device for big data applications can be set in a terminal device or a server or other processing device, wherein the terminal device can be a security inspection device, an electronic device, a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the conflict detection method for big data applications can be implemented by a processor calling computer-readable instructions stored in a memory.

[0026] See also Figure 1 , Figure 1 1 is a flow chart of an embodiment of a conflict detection method for big data applications of the present application. Figure 1 As shown, the conflict detection method for big data applications provided by the embodiment of the present disclosure may include the following steps:

[0027] Step S11: Acquire first relevant information of each dependency package corresponding to each basic component in the big data application and second relevant information of each third-party dependency package in the big data application.

[0028] The big data application can specifically be a big data application program or a computing engine. For example, the big data application can be a spark type application or a flink type application. Among them, the third-party dependency package can be a direct dependency package or an indirect dependency package. Among them, the basic components in the big data application are a general term for a series of components in the common big data ecosystem, including hadoop, spark, hive, flink, kafka, zookeeper, etc. The third-party dependency package can be the above-mentioned third-party toolkit. The first related information and the second related information may include version information or other configuration information, such as access rights and other information.

[0029] Step S12: Based on each first related information and each second related information, determine whether there is a dependency package conflict.

[0030] Exemplarily, the third-party dependency packages may contain dependency packages with the same name corresponding to the basic components. Therefore, step S12 may be to determine whether there is a difference between the first related information and the second related information between the dependency packages with the same name. If there is a difference, it is determined that there is a dependency package conflict between the dependency packages with the same name.

[0031] Step S13: In response to the existence of a dependency package conflict, a prompt message is issued.

[0032] The prompt information includes the conflict cause and / or conflict solution of the dependency package conflict. In some application scenarios, the prompt information includes the conflict cause of the dependency package conflict. In some application scenarios, the prompt information includes the conflict solution of the dependency package conflict. In some application scenarios, the prompt information includes the conflict cause and conflict solution of the dependency package conflict. A corresponding relationship can be established between the conflict cause and the conflict solution in advance, and after the conflict cause is determined, the conflict solution can be determined according to the corresponding relationship.

[0033] The above scheme, during the conflict detection process of big data applications, can determine the conflicting dependency packages in the big data application based on the first relevant information and the second relevant information of the third-party dependency packages by obtaining the first relevant information of each dependency package corresponding to each basic component of the big data application, thereby issuing the cause of the conflict and / or the conflict solution, thereby improving the efficiency of conflict detection.

[0034] In some embodiments, the above step S11 may include the following steps: obtaining second relevant information of each third-party dependency package from a configuration file in the big data application, and obtaining first relevant information of the dependency package corresponding to each basic component from the target warehouse. Wherein, each third-party dependency package is a dependency package that is not added to the dependency conflict resolution plug-in.

[0035] Optionally, the configuration file of the big data application can be a pom file. The target warehouse can be a maven warehouse, and the maven warehouse can be a private warehouse or a central warehouse. The maven warehouse can also be used to store various jar packages, so that big data applications can obtain the required dependencies from these warehouses. The dependency conflict resolution plug-in can be a shade plug-in. Specifically, the full name of the shade plug-in can be a maven-shade-plugin plug-in. The maven-shade-plugin plug-in is mainly used to solve dependency conflict problems in maven project projects. It can package all dependencies into a jar package and rename the configured conflicting classes when packaging. That is, this embodiment can perform conflict detection on dependency packages that are not added to the maven-shade-plugin plug-in.

[0036] In some embodiments, the first relevant information includes the name and version number of the dependent package, and the second relevant information includes the name and version number of each third-party dependent package. Figure 2 On this basis, the above step S12 may include the following steps:

[0037] Step S121: Determine whether there is a target name in each first related information and each second related information.

[0038] The target name exists in at least one of the first related information and at least one of the second related information. That is, if the name of a dependency package exists in both the first related information and the second related information, the name of the dependency package is the target name. In other words, if a dependency package is a dependency package corresponding to each basic component and is also a third-party dependency package, the name of the dependency package is determined to be the target name, and the dependency package is the target dependency package.

[0039] Step S122: In response to the existence of the target name and the different version numbers of the target name in the first relevant information and the second relevant information, it is determined that a conflict exists in the dependent package corresponding to the target name.

[0040] That is, if the version numbers of two dependency packages with the same name are different, it is determined that there is a conflict between the two dependency packages with the same name. That is, if the version number of the target dependency package recorded in the first relevant information is different from the version number recorded in the second relevant information, it is determined that the target dependency package is a third-party dependency package and there is a conflict between the dependency packages of each basic component.

[0041] The prompt information obtained in the above step S13 may include the cause of the conflict, which includes the dependent packages in conflict and the sources of the dependent packages. For example, the prompt information for conflict 4 may be:

[0042] [Conflict 4] >>> Dependency [org.apache.avro:avro-ipc] detected versions are: [1.11.0, 1.10.0]

[0043] The specific sources are as follows:

[0044] |-- Spark dependencies

[0045] |--org.apache.avro:avro-ipc:1.11.0 : compile

[0046] |--Dependencies in the user's POM file (including direct and indirect dependencies)

[0047] |-- org.apache.avro:avro-mapred:jar:1.10.0:compile

[0048] |-- org.apache.avro: avro-ipc:1.10. 0 : compile

[0049] As shown in the above prompt information, the version number of the avro-ipc package that spark itself depends on is 1.11.0, but the version of the indirectly introduced third-party dependency package avro-ipc is 1.10.0. If the two version numbers are inconsistent, it is considered that there is a dependency package conflict.

[0050] In some embodiments, at least some of the basic components include multiple functional modules, and the conflict detection method for big data applications may further include the following steps:

[0051] Obtain version information of each functional module in each basic component. Based on the version information of each functional module in each basic component, determine whether there is a version conflict between the functional modules in each basic component. In response to the existence of a version conflict between at least two functional modules in at least one basic component, add the cause and / or solution of the conflict between the at least two functional modules to the prompt information.

[0052] Among them, each functional module included in the basic component can be specifically considered as a functional module used in the big data application. That is, there can be N functional modules of the basic component, but only M are used in the big data application, and M is less than or equal to N, then only consider whether there are conflicts between the M functional modules. The reason for the conflict between functional modules may be inconsistent versions or the parameters in one of the functional modules are fine-tuned, resulting in incompatibility between the two functional modules, etc.

[0053] For example, if the jar packages of two modules of the basic component spark are used in a big data application: spark-core and spark-mllib, but the versions of spark-core and spark-mllib used are inconsistent or the version difference is large, then it is considered that the two functional modules conflict with each other.

[0054] See also Figure 3 In some embodiments, the conflict detection method for big data applications may further include the following steps:

[0055] Step S21: Obtain the version information of each basic component.

[0056] Specifically, the second relevant information of the third-party dependency package and the relevant information of the basic components (the functional modules used and their version information, and the version information of the basic components) can be obtained from the configuration file (pom file) of the big data.

[0057] Step S22: Based on the version information of each basic component, determine whether each basic component is compatible.

[0058] Exemplarily, if the version difference between different basic components is large, it is determined that the basic components in the big data application are not compatible. Specifically, if the version difference between two basic components is greater than or equal to the preset version difference, it is determined that the version difference is large. For example, if the version of spark is 3.x, and the version of hadoop is 1.x, it is determined that they are not compatible. The determination of the specific preset version difference can be determined based on the normal or abnormal operation of big data.

[0059] Step S23: In response to the existence of at least two incompatible basic components, a solution to the incompatible basic components is added to the prompt information.

[0060] Optionally, relevant information of the incompatible basic components may be added to the prompt information, such as the name and version number of the incompatible basic components.

[0061] See also Figure 4 In some embodiments, the conflict detection method for big data applications may further include the following steps:

[0062] Step S31: Obtain the scope information of each basic component.

[0063] Exemplarily, the scope information may specifically be scope. For example, the acquired scope information may be:

[0064] <dependency>

[0065] <groupid>org.apache.spark< / groupId>

[0066] <artifactid> spark-core_2.12< / artifactid>

[0067] <version>3.3.2-HDP-24.06.3< / version>

[0068] <scope>compile< / scope>

[0069] < / dependency>

[0070] Step S32: Determine whether the parameters in the scope information of each basic component are correct.

[0071] Specifically, determine whether the scope of each basic component in the big data application is correct. For example, if the scope of the spark-core package configured in the pom file of the big data application uses "compile", it is considered a configuration error and should use "provided".

[0072] Step S33: In response to the parameter in the scope information of the basic component being incorrect, a method for modifying the parameter is added to the prompt information.

[0073] Exemplarily, the parameter may be modified by adjusting "compile" used in the scope to "provided".

[0074] In some embodiments, based on each first relevant information and each second relevant information, the step of determining whether there is a dependency package conflict is performed by the target plug-in. Exemplarily, the target plug-in can be a maven plug-in. On this basis, the conflict detection method for big data applications may also include the following steps: configuring the target plug-in into the configuration file of the big data application. Exemplarily, the target plug-in can be configured in the pom file. In response to receiving a packaging instruction for the big data application, the step of determining whether there is a dependency package conflict based on each first relevant information and each second relevant information is executed. Exemplarily, the packaging instruction for the big data application can be a build instruction for the project, that is, when the project is built, the execution of the target plug-in is triggered.

[0075] Among them, the prompt information includes a conflict resolution method, and the conflict resolution method includes an adjustment strategy for the packaging method of the big data application. In some application scenarios, by prompting the packaging method of the big data application, guiding the adjustment of the packaging method of the big data application, the big data application is successfully packaged. That is, the conflict detection method of the big data application provided in this embodiment performs conflict detection on the big data application during the development process of the big data application and guides the packaging method according to the conflict detection result, thereby improving the stability in the subsequent application process. On this basis, the conflict detection method of the big data application may also include the following steps: in response to receiving an adjustment instruction for the preset packaging method of the big data application, the big data application is packaged using the adjusted packaging method. Or, in response to the absence of a dependency package conflict, the big data application is packaged according to the preset packaging method.

[0076] If conflicting dependency packages, conflicting basic components of functional modules, two incompatible basic components and / or basic components with scope problems are detected, the prompt information will include solutions to each problem. For example, if the detection result is a version conflict (version conflict between dependency packages or version conflict between different functional modules in the same basic component), it will prompt you to use the shade plug-in for packaging; if the versions are incompatible, it will prompt you to use the matching version; if the scope is incorrectly configured, it will prompt you to change it to "provided". Exemplarily, the solution is as follows: 1. Use the maven-shade-plugin plug-in for packaging, replacing the packaging plug-in maven-assembly-plugino; 2. Add the dependency [org.apache.avro:avro-ipc] to the maven-shade-plugin plug-in <relocations>Under the tag, shade (redirection) is performed.

[0077] Optionally, the target plug-in provided by this solution can have the following functions: obtain the version information of each basic component in the big data application; detect whether there is a version conflict among multiple functional modules (jar packages) in each basic component in the big data application, and if there is a conflict, prompt the cause and solution of the conflict; detect whether different basic components in the big data application are matched, and if not, prompt the source of the problem and solution; detect whether the scope of the version of each basic component in the big data application is correct, and if not, prompt the cause and solution of the problem; detect the conflict between the third-party dependency package and the dependency package of each basic component within the detection scope. Specifically, it can detect whether there is a conflict between the dependency package (jar package) directly referenced or indirectly referenced and the dependency package of the basic component.

[0078] Optionally, the above detection scope may specifically include all toolkits configured in the project pom file and referenced during compilation and runtime, including direct dependencies and indirect dependencies. Optionally, toolkits that have been added to the maven-shade-plugin plugin are no longer included in the detection scope of the target plugin.

[0079] In this solution, the target plug-in is configured in the pom configuration file of the big data application (spark / flink) maven project in the form of a maven plug-in. The configuration method is simple and convenient. When the project is built, the execution of this tool is triggered. When conflicts and problem items are detected, prompts such as the cause of the conflict, the cause of failure, and the solution are given, and the project build fails. When all conflicts are resolved according to the prompts, the build will be successful.

[0080] In some application scenarios, the conflict detection method for big data applications may include the following steps:

[0081] Get the dependent jar package information (including basic component packages), shade plug-in configuration, and other information from the pom file configuration of the Maven project of the big data application (Spark / Flink). Based on the information obtained, detect the version information of each basic component in the big data application. Based on the detected basic component version information, obtain the dependent package information of the basic component from the Maven warehouse (which can be a private warehouse or a central warehouse). Check whether there are version conflicts among multiple functional module packages of the basic components used in the big data application. For example, if the jars of two modules of spark are used in the pom file: spark-core and spark-mllib, but the versions used are inconsistent, it is considered a version conflict. Check whether the versions of the big data basic components used in the big data application are compatible. For example, the versions of hadoop and spark used in the pom file of the big data application are too different and cannot be compatible. Check whether the scope of the big data basic component version used by the big data application is correct. For example, if the scope of the spark-core package configured in the pom file of the big data application uses "compile", it is considered a configuration error and "provided" should be used. Check whether there is a version conflict between the jar package (including direct and indirect dependencies) used by the big data application and the dependency package of the big data basic component. The judgment standard for the conflict is that if the version numbers of the jar packages with the same name are inconsistent, it is considered a conflict. For example, the version number of the avro-ipc package that spark itself depends on is 1.11.0, but it indirectly introduces the 1.10.0 version of avro-ipc. If the two version numbers are inconsistent, it is considered a version conflict. According to the above test results, when there are conflict items and problem items in the test results, prompts such as the source of the conflict, the cause of failure, and the solution are given, and the project build fails; otherwise, the build succeeds. Prompt solution: According to the test results, if there is a version conflict, it prompts to use the shade plug-in for packaging (see the figure above); if the versions are not compatible, it prompts to use the compatible version; if the scope configuration is incorrect, it prompts to change it to "provided".

[0082] In the process of big data application development, the solution provided by this solution can quickly identify the conflicting items between the dependency packages configured in the Mavenpom file and the dependency packages of the big data basic components, as well as the usage problems of the big data basic components. At the same time, it provides corresponding prompts such as the source of the conflict, the cause of the failure, and the solution, guides the reasonable packaging of big data applications, and greatly improves the efficiency of big data development.

[0083] The above scheme, during the conflict detection process of big data applications, can determine the conflicting dependency packages in the big data application based on the first relevant information and the second relevant information of the third-party dependency packages by obtaining the first relevant information of each dependency package corresponding to each basic component of the big data application, thereby issuing the cause of the conflict and / or the conflict solution, thereby improving the efficiency of conflict detection.

[0084] See also Figure 5 , Figure 5 It is a structural diagram of an embodiment of a conflict detection device for big data applications of the present application. The conflict detection device 30 for big data applications can execute the above-mentioned conflict detection method for big data applications. The conflict detection device 30 for big data applications includes an information acquisition module 31, a conflict detection module 32 and a prompt module 33; the information acquisition module 31 is used to obtain the first relevant information of each dependency package corresponding to each basic component in the big data application and the second relevant information of each third-party dependency package in the big data application; the conflict detection module 32 is used to determine whether there is a dependency package conflict based on each first relevant information and each second relevant information; the prompt module 33 is used to issue a prompt message in response to the existence of a dependency package conflict, and the prompt message includes the conflict cause and / or conflict solution of the dependency package conflict.

[0085] The above scheme, during the conflict detection process of big data applications, can determine the conflicting dependency packages in the big data application based on the first relevant information and the second relevant information of the third-party dependency packages by obtaining the first relevant information of each dependency package corresponding to each basic component of the big data application, thereby issuing the cause of the conflict and / or the conflict solution, thereby improving the efficiency of conflict detection.

[0086] Among them, the functions of each module can be found in the embodiment of the conflict detection method for big data application, which will not be repeated here.

[0087] See also Figure 6 , Figure 6 4 is a schematic diagram of the structure of an embodiment of an electronic device of the present application. The electronic device 40 includes a memory 41 and a processor 42, and the processor 42 is used to execute program instructions stored in the memory 41 to implement the steps in the above-mentioned conflict detection method embodiment of any big data application. In a specific implementation scenario, the electronic device 40 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 40 may also include a carrier device such as a laptop computer and a tablet computer, which is not limited here.

[0088] Specifically, the processor 42 is used to control itself and the memory 41 to implement the steps in the above-mentioned conflict detection method embodiment of any big data application. The processor 42 can also be called a CPU (Central Processing Unit). The processor 42 may be an integrated circuit chip with signal processing capabilities. The processor 42 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 42 can be implemented by an integrated circuit chip.

[0089] The above scheme, during the conflict detection process of big data applications, can determine the conflicting dependency packages in the big data application based on the first relevant information and the second relevant information of the third-party dependency packages by obtaining the first relevant information of each dependency package corresponding to each basic component of the big data application, thereby issuing the cause of the conflict and / or the conflict solution, thereby improving the efficiency of conflict detection.

[0090] See also Figure 7 , Figure 7 The computer-readable storage medium 50 stores program instructions 51, which, when executed by a processor, implement the steps in any of the above-mentioned conflict detection method embodiments for big data applications.

[0091] The above scheme, during the conflict detection process of big data applications, can determine the conflicting dependency packages in the big data application based on the first relevant information and the second relevant information of the third-party dependency packages by obtaining the first relevant information of each dependency package corresponding to each basic component of the big data application, thereby issuing the cause of the conflict and / or the conflict solution, thereby improving the efficiency of conflict detection.

[0092] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0093] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0094] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. In another image position, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0095] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.< / relocations> < / scope> < / version> < / groupid> < / dependency>

Claims

1. A conflict detection method for big data applications, characterized in that: include: Obtaining first relevant information of each dependency package corresponding to each basic component in the big data application and second relevant information of each third-party dependency package in the big data application; Determine whether there is a dependency package conflict based on each of the first relevant information and each of the second relevant information; In response to the existence of a dependency package conflict, issuing a prompt message, the prompt message including a conflict cause and / or a conflict solution of the dependency package conflict; Wherein, the step of determining whether there is a dependency package conflict based on each of the first related information and each of the second related information is performed by a target plug-in, and the method further includes: configuring the target plug-in into a configuration file of the big data application; in response to receiving a packaging instruction of the big data application, performing the step of determining whether there is a dependency package conflict based on each of the first related information and each of the second related information; The prompt information includes a conflict resolution method, and the conflict resolution method includes an adjustment strategy for the packaging method of the big data application. The method also includes: in response to receiving an adjustment instruction for a preset packaging method of the big data application, packaging the big data application using the adjusted packaging method; or, in response to the absence of a dependency package conflict, packaging the big data application according to the preset packaging method, wherein the packaging plug-in used in the adjusted packaging method is different from the packaging plug-in used in the preset packaging method.

2. The method according to claim 1, characterized in that The first relevant information includes the name and version number of the dependency package, the second relevant information includes the name and version number of each of the third-party dependency packages, and the determining whether there is a dependency package conflict based on each of the first relevant information and each of the second relevant information includes: Determine whether a target name exists in each of the first related information and each of the second related information, the target name existing in at least one of the first related information and at least one of the second related information; In response to the existence of the target name and different version numbers of the target name in the first relevant information and the second relevant information, it is determined that a conflict exists in the dependent package corresponding to the target name.

3. The method according to claim 1, characterized in that: At least part of the basic components includes a plurality of functional modules, and the method further includes: Obtaining version information of each of the functional modules in each of the basic components; Based on the version information of each functional module in each basic component, determining whether there is a version conflict between each functional module in each basic component; In response to a version conflict between at least two functional modules in at least one of the basic components, a reason and / or a solution for the conflict between the at least two functional modules are added to the prompt information.

4. The method according to claim 1, characterized in that: The obtaining of first relevant information of each dependency package corresponding to each basic component in the big data application and second relevant information of each third-party dependency package in the big data application includes: Acquire the second relevant information of each of the third-party dependency packages from the configuration file in the big data application, and acquire the first relevant information of the dependency packages corresponding to each of the basic components from the target warehouse; Among them, each of the third-party dependency packages is a dependency package that has not been added to the dependency conflict resolution plug-in.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Obtaining version information of each of the basic components; Based on the version information of each of the basic components, determining whether each of the basic components is compatible; In response to at least two incompatible basic components, a solution to the incompatible basic components is added to the prompt information.

6. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Obtaining scope information of each of the basic components; Determining whether the parameters in the scope information of each of the basic components are correct; In response to the parameter in the scope information of the basic component being incorrect, a modification method of the parameter is added to the prompt information.

7. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images