Smart lighting task recommendation method, system and device and readable medium

CN118964736APending Publication Date: 2024-11-15SHENZHEN ZHONGFUNENG ELECTRIC EQUIPMENT CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411037679.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-15

Smart Images

  • Figure CN118964736A_ABST
    Figure CN118964736A_ABST
Patent Text Reader

Abstract

The invention provides an intelligent lighting task recommendation method, system and device and a readable medium, and relates to the technical field of big data, and the method comprises the steps: obtaining multi-dimensional metadata information of a to-be-created lighting task; processing the multi-dimensional metadata information to obtain a feature data set in sequence representation; based on the feature data set, obtaining a homologous calculation task to which the first data source belongs, and performing similarity matching calculation through a feature vector of a preset calculation task in a platform back-end service by using an operational formula of a related set of discrete mathematics; and judging whether the matching calculation result is in a preset interval or not, and then re-evaluating the necessity of creating the lighting task. The method has the advantages that the quality of collected data is high, a user is helped to quickly reuse existing calculation tasks and existing database tables, the situation that redundant calculation tasks are submitted to a system and calculation and storage resources are occupied is avoided, a coal mine enterprise data recommendation method is optimized, and the accuracy of coal mine enterprise big data calculation is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a smart lighting task recommendation method, system, device and readable medium. Background Art

[0002] With the development of science and technology, the technology of smart lighting system with informatization, automation and intelligence as the core has been further developed. However, there are great differences in the lighting deployment in various scenarios such as outdoor adventure lighting, natural gas lighting, underwater diving lighting, mining lighting, etc. Therefore, the technical means combining machine learning and big data processing technology have been promoted in the field of smart lighting. Through machine learning, the laws and trends in the data can be quickly summarized, and intelligent interaction and management tasks can be generated by identifying and perceiving data, so as to realize the lighting system with high efficiency, energy saving, safety and environmental protection.

[0003] However, as the business scale expands, the characteristic data of lighting scenes are different, the number of data applications increases, and there is repeated calculation and storage of data, which increases the storage cost. In addition, a large number of repeated calculations increase the computing cost for proposing intelligent lighting strategies. Summary of the invention

[0004] The purpose of the present invention is to provide a smart lighting task processing method, system, device and readable medium to reduce the increasing storage cost problem and the computing power problem of fast smart lighting strategy.

[0005] A smart lighting task recommendation method, system, device and readable medium are provided to improve the above problems. In order to achieve the above objectives, the technical solutions adopted by the present invention are as follows:

[0006] In a first aspect, the present application provides a method for intelligent lighting task recommendation, comprising:

[0007] Acquire multidimensional metadata information of the lighting task to be created, where the multidimensional metadata information includes a first data source selected by the data processing party, a table corresponding to the data source, and column fields in the table;

[0008] Process the multidimensional metadata information to obtain a feature data set represented as a sequence, wherein the process of the feature data set is to obtain the data table and column fields in the data table that the user needs to read from the data source when executing the calculation task, and merge the data table and column fields into a field data set;

[0009] Based on the feature data set, the homologous computing tasks to which the first data source belongs are obtained, and according to each homologous computing task, the table stored in each computing task and the set of column fields in the table are obtained, and the similarity matching calculation is performed through the feature vectors of the computing tasks preset in the platform backend business by using the operation formula of the relevant sets of discrete mathematics to obtain the matching calculation result;

[0010] Determine whether the matching calculation result is within the preset interval. If it is within the preset interval, the library table of the matching calculation result will be sent to the terminal for lighting task recommendation. If it is greater than the preset interval, the operation control of the platform backend will be triggered, and the necessity of creating the lighting task will be re-evaluated.

[0011] Preferably, the processing of the multidimensional metadata information includes:

[0012] Create a platform task framework; based on the platform task framework, connect the data warehouse system of the coal mining enterprise through the big data platform;

[0013] Input the multidimensional metadata information into the data warehouse system, and select the first data warehouse required for the multidimensional metadata information using the Web interface, wherein the first data warehouse includes a library, a table, and a column field in the table corresponding to the data;

[0014] The metadata in the first data warehouse is stored, wherein the metadata is used to support lifecycle management, security and authority management of the data.

[0015] Preferably, the use of the relevant set operation formula of discrete mathematics is used to perform similarity matching calculation through the feature vector of the computing task preset in the platform back-end business to obtain a matching calculation result, which includes: based on the set intersection and union method in discrete mathematics, the feature vector of the computing task preset by the system is matched to perform similarity calculation, and it is judged whether the tasks running with the existing task field set have an inclusion relationship according to the two field sets. If so, the inclusion relationship is marked as a match degree of 100%. If not, a matching rate calculation is performed, wherein the matching rate calculation rule is to divide the number of intersections and unions of the two column fields.

[0016] Preferably, obtaining the matching calculation result then includes:

[0017] Extract the data corresponding to the matching rate greater than 60% in the matching calculation results, summarize all the data into a list, and sort them from high to low according to the similarity to obtain the arrangement result;

[0018] Eliminate redundant data in the arrangement results and select the top three data for storage;

[0019] Loop through each record with similarity in the arrangement result, and query the data warehouse for relevant information, including data source, column field details in the table, and primary key status, and feed back the relevant information to the front end for display.

[0020] Preferably, the obtaining of the homologous computing tasks to which the first data source belongs, and obtaining, according to each homologous computing task, a table stored in each computing task and a set of column fields in the table, include:

[0021] Based on the feature data set, each computing task data corresponding to each homologous computing task is obtained respectively, wherein the computing task data includes a table storing the task and a set of column fields in the table;

[0022] Determine whether the computing task data corresponding to the same-source computing tasks obtained twice consecutively are the same task;

[0023] If the computing task data are not the same, determine whether the absolute value is less than the first parameter. If the absolute value is less than the first parameter, the two adjacent homologous computing tasks are combined task data, wherein the first parameter is the maximum interval parameter of the homologous combined tasks.

[0024] If the computing task data is the same, determine whether the absolute value is less than a second parameter. If the absolute value is less than the second parameter, the two adjacent homologous computing tasks are combined task data, wherein the second parameter is the maximum interval parameter of combined tasks of different sources.

[0025] In a second aspect, the present application also provides a big data recommendation system based on similarity, including an acquisition module, a processing module, a calculation module and a judgment module, wherein:

[0026] An acquisition module: used to acquire multidimensional metadata information of a lighting task to be created, wherein the multidimensional metadata information includes a first data source selected by a data processor, a table corresponding to the data source, and column fields in the table;

[0027] Processing module: used to process the multidimensional metadata information to obtain a feature data set represented as a sequence, wherein the process of the feature data set is to obtain the data table and column fields in the data table that the user needs to read from the data source when executing the calculation task, and merge the data table and column fields into a field data set;

[0028] Calculation module: used to obtain the same-source calculation tasks to which the first data source belongs based on the feature data set, and obtain the table stored in each calculation task and the set of column fields in the table according to each same-source calculation task, and perform similarity matching calculation through the feature vectors of the calculation tasks preset in the platform backend business using the calculation formulas of the relevant sets in discrete mathematics to obtain the matching calculation results;

[0029] Judgment module: used to determine whether the matching calculation result is within the preset interval. If it is within the preset interval, the library table of the matching calculation result will be sent to the terminal for lighting task recommendation. If it is greater than the preset interval, it will trigger the operation control of the platform backend, and then re-evaluate the necessity of creating a lighting task.

[0030] In a third aspect, the present application also provides a big data recommendation device based on similarity, including:

[0031] Memory for storing computer programs;

[0032] A processor is used to implement the steps of the smart lighting task recommendation method when executing the computer program.

[0033] In a fourth aspect, the present application further provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned method for recommending tasks based on smart lighting are implemented.

[0034] The beneficial effects of the present invention are:

[0035] The present invention aims to solve the problems of repeated calculations and a large number of redundant database tables in big data computing tasks. When users create big data computing tasks, the present invention helps users to quickly reuse existing computing tasks and existing database tables, and avoids submitting redundant computing tasks to the system to occupy computing and storage resources, thereby achieving the purpose of reducing costs and increasing efficiency of big data computing systems, obtaining more complete information, and collecting high-quality data. It optimizes the recommendation method for coal mining enterprise data and improves the accuracy of big data computing in coal mining enterprises.

[0036] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or be understood by implementing the embodiments of the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 A schematic diagram of the process flow of the smart lighting task recommendation method described in an embodiment of the present invention;

[0039] Figure 2Schematic diagram of the structure of a big data recommendation system based on similarity described in an embodiment of the present invention;

[0040] Figure 3 Schematic diagram of the structure of a big data recommendation device based on similarity described in an embodiment of the present invention.

[0041] In the figure: 701, acquisition module; 702, processing module; 7021, creation unit; 7022, connection unit; 7023, input unit; 7024, storage unit; 703, calculation module; 7031, first judgment unit; 7032, arrangement unit; 7033, elimination unit; 7034, query unit; 7035, acquisition unit; 7036, second judgment unit; 7037, third judgment unit; 7038, fourth judgment unit; 704, judgment module; 800, big data recommendation device based on similarity; 801, processor; 802, memory; 803, multimedia component; 804, I / O interface; 805, communication component. DETAILED DESCRIPTION

[0042] The operating method of the monitoring system provided by the embodiment of the present invention is applied to the monitoring system. Unless otherwise defined, all technical and scientific terms used in this document have the same meaning as those generally understood by technicians in the technical field of this application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned drawings and any variations thereof are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of this application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0043] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0044] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0045] This embodiment provides a method for recommending intelligent lighting tasks.

[0046] See also Figure 1, the figure shows that the method includes step S100, step S200, step S300 and step S400.

[0047] S100. Obtain multidimensional metadata information of a lighting task to be created, where the multidimensional metadata information includes a first data source selected by a data processor, a table corresponding to the data source, and column fields in the table.

[0048] In the prior art, in this step, coal mining enterprises usually rely on manual work and process specifications to solve the problem of repeated calculation and repeated storage. The problem is that the efficiency is relatively low, the data calculation task processing process is relatively long, and it often takes a lot of manpower to complete the relevant processes and specifications. Some database table data exist in multiple copies in the coal mining enterprise data warehouse. These duplicate data occupy a large amount of storage space, which invisibly increases the storage cost of the enterprise in building a big data warehouse. A large amount of repeated calculations will also increase the computing cost of the enterprise. Therefore, in the present invention, a method of recommending big data computing tasks based on similarity will be adopted, so that when users create big data computing tasks, they can help users quickly reuse existing computing tasks, thereby achieving the purpose of reducing costs and increasing efficiency of big data computing systems.

[0049] It can be understood that in this step, the multidimensional metadata information includes that when the computing task creator uses the computing platform to create a lighting task, the big data computing platform can realize the connection to the enterprise's data warehouse system, and can select the library, table and table field corresponding to the required data warehouse through the Web interface, and then store these metadata; secondly, after obtaining the multidimensional metadata information, it is necessary to analyze and preprocess, and the steps include but are not limited to data cleaning and preparation: first, the original data needs to be cleaned and prepared. This includes removing duplicate data, processing missing values ​​and outliers, and converting data into a suitable format; secondly, data analysis target definition: before performing multidimensional data analysis, it is necessary to clearly define the analysis target as multidimensional metadata information in coal mines. For example, in exploring the main patterns in the data set and discovering potential associations, or identifying abnormal points. Next is data set modeling: it is necessary to select an appropriate data model to describe the characteristics and relationships of the data set. Commonly used data models include clustering algorithms, classification algorithms, association rule mining, etc. In addition, in this embodiment, dimension selection and selective projection: in multidimensional data analysis, the problem of dimensional disaster will be faced, that is, the data dimension is too high. To overcome this problem, dimension selection and selective projection techniques can be used to reduce dimensions and extract the most useful features. Secondly, data visualization: Data visualization is very important in multidimensional data analysis and can help understand the relationships and patterns between data. Common visualization techniques include scatter plots, histograms, heat maps, parallel coordinates plots, etc. Finally, pattern discovery and association analysis: Once the modeling and visualization of the data set are completed, various techniques can be used to discover patterns and associations in the data set, including association rule mining, cluster analysis, classification analysis, principal component analysis, etc. Therefore, by analyzing multidimensional metadata information, valuable information can be obtained from the data, and the efficiency of the next step of processing is improved, redundant information is eliminated, and accuracy is improved.

[0050] S200, processing the multidimensional metadata information to obtain a feature data set represented as a sequence, wherein the process of the feature data set is to obtain the data table and column fields in the data table that the user needs to read from the data source when executing the computing task, and merge the data table and column fields into a field data set.

[0051] It can be understood that the step S200 includes S201, S202, S203 and S204, wherein:

[0052] S201, create a platform task framework;

[0053] S202. Based on the platform task framework, the data warehouse system of the coal mining enterprise is connected through the big data platform;

[0054] S203, inputting the multidimensional metadata information into the data warehouse system, and selecting a first data warehouse required for the multidimensional metadata information using a Web interface, wherein the first data warehouse includes a library, a table, and column fields in the table corresponding to the data;

[0055] S204: Storing metadata in the first data warehouse, where the metadata is used to support lifecycle management, security, and authority management of data.

[0056] In this step, the multidimensional metadata information is processed into a feature set represented by a sequence. The main process is to obtain the data table and column fields of the table that need to read the data source selected by the user through the platform when executing the computing task to form a field set, and obtain the computing tasks using the same data source in the coal mining industry system. Through the data source selected in the first step, a homologous computing task set is obtained from the coal mining industry system, in order to further obtain the table and table field set stored in each computing task according to each homologous computing task, so that when users create coal mine big data computing tasks, they can help users quickly reuse existing computing tasks and existing database tables to improve efficiency.

[0057] S300. Based on the feature data set, obtain the homologous computing tasks belonging to the first data source, and obtain the table stored in each computing task and the set of column fields in the table according to each homologous computing task, and use the relevant set operation formulas of discrete mathematics to perform similarity matching calculations through the feature vectors of the computing tasks preset in the platform backend business to obtain the matching calculation results.

[0058] It can be understood that this step S300 includes S301, in which the operation formula of the relevant set in discrete mathematics is used to perform similarity matching calculation through the characteristic vector of the computing task preset in the platform back-end business to obtain the matching calculation result, including: based on the set intersection and union method in discrete mathematics, the characteristic vector of the computing task preset by the system is matched to perform similarity calculation, and it is judged whether the tasks running with the existing task field set have an inclusion relationship according to the two field sets. If so, the inclusion relationship is marked as a match degree of 100%, if not, a matching rate calculation is performed, wherein the matching rate calculation rule is to divide the number of intersections and unions of the two column fields.

[0059] In this step, when calculating the similarity, a common method is to calculate the distance between two vectors. The closer the distance between two vectors, the more similar the two vectors are. The Euclidean distance is a commonly used distance definition, which refers to the real distance between two points in n-dimensional space. The Euclidean distance in two-dimensional and three-dimensional space is the straight-line distance between two points, including but not limited to the Chebyshev distance or L∞ metric, which is a metric in vector space. The distance between two points is defined as the maximum absolute value of the difference between the numerical values ​​of their coordinates. Through the above calculation, the matching calculation result is obtained. Based on the formula of set intersection and union, that is, the inclusion relationship is reflexive, and it has for any set. The inclusion relation is transitive, and for any set A, B, C, and but The inclusion relation is antisymmetric. For any set A and B, as long as and Then A = B. Through the formula, it is determined whether there is a containment relationship, and then the matching rate is calculated.

[0060] It should be noted that step S300 also includes S302, S303 and S304, wherein:

[0061] S302, extracting data corresponding to a matching rate greater than 60% in the matching calculation results, aggregating all the data into a list, and sorting them from high to low according to similarity to obtain a ranking result;

[0062] S303, remove redundant data in the arrangement result, and select the top three data for storage;

[0063] S304, loop through each record with similarity in the arrangement result, and query the data warehouse for relevant information, where the relevant information includes data source, column field details in the table and primary key status, and feed back the relevant information to the front end for display.

[0064] Specifically, in this step, computing tasks that use the same data source in the system are obtained, and computing tasks of the same source are obtained from the system through the data source selected in the first step; the backend puts the result tables with a matching rate exceeding 60% into a list and sorts them from high to low according to the similarity; then, specifically, the backend sets the limit on the number of returned result sets through configuration parameters to filter out redundant tables. It should be noted that the first three are usually sufficient to avoid unnecessary recommendation results causing trouble to users; secondly, each record in the similarity result list is processed in a loop, and the table data source, table field details, primary key and other information are queried and returned to the front end for display; finally, a threshold is set. If there are tasks with a matching rate exceeding 90% or above, if computing tasks still need to be created, the approval audit operation logic will be triggered to evaluate the necessity of creating computing tasks.

[0065] It should be noted that step S300 also includes S305, S306, S307 and S308, wherein:

[0066] S305, based on the feature data set, respectively obtain each computing task data corresponding to each homologous computing task, wherein the computing task data includes a table in which the task is stored and a set of column fields in the table;

[0067] S306, determining whether the computing task data corresponding to the same-source computing tasks acquired twice consecutively are the same task;

[0068] It can be understood that in this step, the computing task data corresponding to the same-source computing tasks acquired twice consecutively are the table corresponding to each task and the column fields in the table.

[0069] S307, if the computing task data are not the same, determine whether the absolute value is less than the first parameter, if the absolute value is less than the first parameter, the two adjacent homologous computing tasks are combined task data, wherein the first parameter is the maximum interval parameter of the homologous combined tasks;

[0070] It can be understood that in this step, when the computing task data corresponding to the two consecutive homologous computing tasks are not the same, it is further determined whether the absolute value between the data of the two computing tasks is less than the first parameter. If so, it means that all the task data of the user in the task is the user's potential demand data, which is used as the combined task data. If the absolute value of the computing task data corresponding to the homologous computing tasks obtained is greater than or equal to the first parameter, it means that all the task data of the user in the task is the user's potential demand data and not the demand data, and it cannot be the combined task data, so the combined task data of the data is abandoned.

[0071] S308. If the computing task data is the same, determine whether the absolute value is less than a second parameter. If the absolute value is less than the second parameter, the two adjacent same-source computing tasks are combined task data, wherein the second parameter is the maximum interval parameter of different-source combined tasks.

[0072] It can be understood that in this step, when the computing task data corresponding to the two consecutive homologous computing tasks are the same, it is further determined whether the absolute value between the data of the two computing tasks is less than the second parameter. If so, it means that all the task data of the user in the task is the user's potential demand data, which is used as the combined task data. If the absolute value of the computing task data corresponding to the homologous computing tasks obtained is greater than or equal to the second parameter, it means that all the task data of the user in the task is the user's potential demand data and not the demand data, and it cannot be the combined task data, so the combined task data of the data is abandoned.

[0073] It can be understood that in this step, the similarity between the existing task field set and other field sets is calculated, and all fields corresponding to similarity values ​​higher than the similarity threshold (the preset similarity threshold is 0.5, which can be adjusted without restriction) are classified into the same category. Based on the above method, all fields are classified to obtain a similar group set. The calculation formula is as follows:

[0074]

[0075] Where: r ij is the similarity; H(i) is the combined task data of the existing task field set; H(j) is the combined task data of other field sets.

[0076] It is understandable that in this step, the similar group set to which the target customer belongs can be determined based on the target customer, and based on the combined task data corresponding to the user in the similar group set, the screening user containing the current operation data can be determined based on the user's current operation data (such as the searched task data, etc.), and then the combined task data containing the current operation data can be sorted in descending order based on the similarity of the screening users to obtain a combined task recommendation table. The calculation method of the similarity between the current product and other products includes the Pearson correlation coefficient algorithm or the Manhattan distance, etc.

[0077] S400, determine whether the matching calculation result is within the preset interval. If it is within the preset interval, send the library table of the matching calculation result to the terminal for lighting task recommendation. If it is greater than the preset interval, it will trigger the operation control of the platform backend, and then re-evaluate the necessity of creating the lighting task.

[0078] It can be understood that in this embodiment, in combination with steps S100-S300, in a second possible implementation of the first aspect, data accuracy scoring is performed based on multidimensional metadata information to determine the comprehensive score of each data table and column field in the data table, including: obtaining user evaluation information associated with the current data table and the column field in the data table, wherein each user evaluation information includes an indicator score for evaluating the current data table and the column field in the data table on multiple indicators; based on the indicator score of each user evaluation information associated with the current data table and the column field in the data table, calculating the information score corresponding to the column field in the current data table and the data table, and finally obtaining the scoring result of each data table and the column field in the data table. Calculate the similarity between the homologous computing task to which the first data source belongs and the computing task to which the preset data source belongs; based on the similarity between the homologous computing task to which the first data source belongs and the computing task to which the preset data source belongs, determine the data with the highest similarity to the current homologous computing task. Then, the back-end processing is performed based on the comparison between the matching result and the threshold. That is to say, if it is within the preset interval, the library table of the matching calculation result will be sent to the terminal for public announcement of the lighting task recommendation; if it is greater than the preset interval, if the calculation task still needs to be created, the approval and audit operation logic will be triggered to evaluate the necessity of creating the calculation task.

[0079] like Figure 2 As shown, this embodiment provides a big data recommendation system based on similarity, see Figure 2 The system includes an acquisition module 701, a processing module 702, a calculation module 703 and a judgment module 704, wherein:

[0080] Acquisition module 701: used to acquire multi-dimensional metadata information of the lighting task to be created, where the multi-dimensional metadata information includes the first data source selected by the data processing party, the table corresponding to the data source, and the column fields in the table;

[0081] Processing module 702: used to process the multidimensional metadata information to obtain a feature data set represented in a sequence, wherein the process of the feature data set is to obtain the data table and the column fields in the data table that the user needs to read from the data source when executing the computing task, and merge the data table and the column fields into a field data set;

[0082] Calculation module 703: used to obtain the same-source computing tasks to which the first data source belongs based on the feature data set, and obtain the table stored in each computing task and the set of column fields in the table according to each same-source computing task, and perform similarity matching calculation through the feature vectors of the computing tasks preset in the platform backend business using the operation formula of the relevant set of discrete mathematics to obtain the matching calculation result;

[0083] Judgment module 704: is used to judge whether the matching calculation result is within the preset interval. If it is within the preset interval, the library table of the matching calculation result will be sent to the terminal for lighting task recommendation. If it is greater than the preset interval, it will trigger the operation control of the platform backend, and then re-evaluate the necessity of creating the lighting task.

[0084] Specifically, the processing module 702 includes: a creation unit 7021, a connection unit 7022, an input unit 7023 and a storage unit 7024, wherein:

[0085] Creation unit 7021: used to create a platform task framework;

[0086] Connection unit 7022: used to connect the data warehouse system of the coal mining enterprise through the big data platform based on the platform task framework;

[0087] Input unit 7023: used to input the multidimensional metadata information into the data warehouse system, and select the first data warehouse required for the multidimensional metadata information using the Web interface, wherein the first data warehouse includes the library, table and column fields in the table corresponding to the data;

[0088] Storage unit 7024: used to store metadata in the first data warehouse, where the metadata is used to support data lifecycle management, security and authority management.

[0089] Specifically, the calculation module 703 includes:

[0090] The first judgment unit 7031 is used to perform similarity calculation based on the set intersection and union method in discrete mathematics, and match the feature vector of the computing task preset by the system, and judge whether the tasks running with the existing task field set have an inclusion relationship according to the two field sets. If so, the inclusion relationship is marked as a match degree of 100%. If not, a matching rate calculation is performed, wherein the matching rate calculation rule is to divide the number of intersections and unions of the two column fields.

[0091] Specifically, the calculation module 703 further includes an arrangement unit 7032, a removal unit 7033 and a query unit 7034, wherein:

[0092] Arrangement unit 7032: used to extract data corresponding to the matching rate greater than 60% in the matching calculation results, summarize all the data into a list, and sort them from high to low according to the similarity to obtain the arrangement result;

[0093] Elimination unit 7033: used to eliminate redundant data in the arrangement result and select the top three data for storage;

[0094] Query unit 7034: used to loop through each record related to similarity in the arrangement result, and query the relevant information in the data warehouse, where the relevant information includes the data source, column field details in the table and the primary key status, and feed back the relevant information to the front end for display.

[0095] Specifically, the calculation module 703 further includes an acquisition unit 7035, a second judgment unit 7036, a third judgment unit 7037 and a fourth judgment unit 7038, wherein:

[0096] The acquisition unit 7035 is used to acquire each computing task data corresponding to each homologous computing task based on the feature data set, wherein the computing task data includes a table storing the task and a set of column fields in the table;

[0097] The second judgment unit 7036 is used to judge whether the computing task data corresponding to the same source computing tasks obtained twice consecutively are the same task;

[0098] The third judgment unit 7037 is used to judge whether the absolute value is less than the first parameter if the computing task data are not the same; if the absolute value is less than the first parameter, the two adjacent homologous computing tasks are combined task data, wherein the first parameter is the maximum interval parameter of the homologous combined tasks;

[0099] The fourth judgment unit 7038 is used to judge whether the absolute value is less than the second parameter if the computing task data is the same; if the absolute value is less than the second parameter, the two adjacent same-source computing tasks are combined task data, wherein the second parameter is the maximum interval parameter of different-source combined tasks.

[0100] It should be noted that, regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0101] Corresponding to the above method embodiment, this embodiment also provides a big data recommendation device based on similarity. The big data recommendation device based on similarity described below and the intelligent lighting task recommendation method described above can refer to each other.

[0102] Figure 3 8 is a block diagram of a big data recommendation device 800 based on similarity according to an exemplary embodiment. Figure 3 As shown, the big data recommendation device 800 based on similarity includes: a processor 801 and a memory 802. The big data recommendation device 800 based on similarity also includes one or more of a multimedia component 803, an I / O interface 804, and a communication component 805.

[0103] The processor 801 is used to control the overall operation of the similarity-based big data recommendation device 800 to complete all or part of the steps in the above-mentioned smart lighting task recommendation method. The memory 802 is used to store various types of data to support the operation of the similarity-based big data recommendation device 800. These data may include, for example, instructions for any application or method operating on the similarity-based big data recommendation device 800, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or optical disk. The multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 802 or sent via the communication component 805. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 804 provides an interface between the processor 801 and other interface modules, and the above-mentioned other interface modules may be a keyboard, a mouse or buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 805 is used for wired or wireless communication between the similarity-based big data recommendation device 800 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 805 may include: a Wi-Fi module, a Bluetooth module or an NFC module.

[0104] In an exemplary embodiment, the similarity-based big data recommendation device 800 can be implemented by one or more application-specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned smart lighting task recommendation method.

[0105] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned smart lighting task recommendation method are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 802 including program instructions, and the above-mentioned program instructions can be executed by the processor 801 of the big data recommendation device 800 based on similarity to complete the above-mentioned smart lighting task recommendation method.

[0106] Corresponding to the above method embodiment, a readable storage medium is also provided in this embodiment. The readable storage medium described below and the intelligent lighting task recommendation method described above can refer to each other.

[0107] A computer program is stored on the readable storage medium, and when the computer program is executed by the processor, the steps of the smart lighting task recommendation method of the above method embodiment are implemented.

[0108] The readable storage medium can be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc., which can store program codes. Obviously, the embodiments described above are only part of the embodiments of the present application, not all of them. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application.

[0109] The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacements on some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is also within the scope of patent protection of this application.

Claims

1. A method for intelligent lighting task recommendation, It is characterized in that include: Acquire multidimensional metadata information of the lighting task to be created, where the multidimensional metadata information includes a first data source selected by the data processing party, a table corresponding to the data source, and column fields in the table; The multidimensional metadata information is processed to obtain a feature data set represented as a sequence. The process of the feature data set is to obtain the data table and column fields in the data table that the user needs to read from the data source when executing the calculation task, and merge the data table and column fields into a field data set; Based on the feature data set, the homologous computing tasks to which the first data source belongs are obtained, and according to each homologous computing task, the table stored in each computing task and the set of column fields in the table are obtained, and the similarity matching calculation is performed through the feature vectors of the computing tasks preset in the platform backend business by using the operation formula of the relevant sets of discrete mathematics to obtain the matching calculation result; When the matching calculation record is within a preset recommendation interval, the creation of the lighting task to be created based on the feature data set triggers a lighting task recommendation instruction, Determine whether the matching calculation result is within the preset interval. If it is within the preset interval, the library table of the matching calculation result will be sent to the terminal for lighting task recommendation. If it is greater than the preset interval, the operation control of the platform backend will be triggered, and the necessity of creating the lighting task will be re-evaluated.

2. The smart lighting task recommendation method according to claim 1, It is characterized in that The multidimensional metadata information is processed, including: Create a platform task framework; Based on the platform task framework, the data warehouse system of the coal mining enterprise is connected through the big data platform; Input the multidimensional metadata information into the data warehouse system, and select the first data warehouse required for the multidimensional metadata information using the Web interface, wherein the first data warehouse includes a library, a table, and a column field in the table corresponding to the data; The metadata in the first data warehouse is stored, wherein the metadata is used to support lifecycle management, security and authority management of the data.

3. The smart lighting task recommendation method according to claim 1, It is characterized in that The method utilizes the operation formula of the relevant sets in discrete mathematics, performs similarity matching calculation through the feature vectors of the computing tasks preset in the platform backend business, and obtains the matching calculation results, which include: based on the set intersection and union method in discrete mathematics, performs similarity calculation with the feature vectors of the computing tasks preset by the system, and judges whether the tasks running with the existing task field sets have an inclusion relationship according to the two field sets. If so, the inclusion relationship is marked as a match of 100%. If not, a matching rate calculation is performed, wherein the matching rate calculation rule is to divide the number of intersections and unions of the two column fields.

4. The smart lighting task recommendation method according to claim 1, It is characterized in that The matching calculation result is obtained, and then includes: Extract the data corresponding to the matching rate greater than 60% in the matching calculation results, summarize all the data into a list, and sort them from high to low according to the similarity to obtain the arrangement result; Eliminate redundant data in the arrangement results and select the top three data for storage; Loop through each record with similarity in the arrangement result, and query the data warehouse for relevant information, including data source, column field details in the table, and primary key status, and feed back the relevant information to the front end for display.

5. The smart lighting task recommendation method according to claim 1, It is characterized in that The obtaining of the same-source computing tasks to which the first data source belongs, and obtaining, according to each of the same-source computing tasks, a table stored in each computing task and a set of column fields in the table, includes: Based on the feature data set, each computing task data corresponding to each homologous computing task is obtained respectively, wherein the computing task data includes a table storing the task and a set of column fields in the table; Determine whether the computing task data corresponding to the same-source computing tasks obtained twice consecutively are the same task; If the computing task data are not the same, determine whether the absolute value is less than the first parameter. If the absolute value is less than the first parameter, the two adjacent homologous computing tasks are combined task data, wherein the first parameter is the maximum interval parameter of the homologous combined tasks. If the computing task data is the same, determine whether the absolute value is less than a second parameter. If the absolute value is less than the second parameter, the two adjacent homologous computing tasks are combined task data, wherein the second parameter is the maximum interval parameter of combined tasks of different sources.

6. A big data recommendation system based on similarity, It is characterized in that include: An acquisition module: used to acquire multidimensional metadata information of a lighting task to be created, wherein the multidimensional metadata information includes a first data source selected by a data processor, a table corresponding to the data source, and column fields in the table; Processing module: used to process multidimensional metadata information to obtain a feature data set represented as a sequence. The process of the feature data set is to obtain the data table and column fields in the data table that the user needs to read from the data source when executing the calculation task, and merge the data table and column fields into a field data set; Calculation module: used to obtain the same-source calculation tasks to which the first data source belongs based on the feature data set, and obtain the table stored in each calculation task and the set of column fields in the table according to each same-source calculation task, and perform similarity matching calculation through the feature vectors of the calculation tasks preset in the platform backend business using the calculation formulas of the relevant sets in discrete mathematics to obtain the matching calculation results; Judgment module: used to determine whether the matching calculation result is within the preset interval. If it is within the preset interval, the library table of the matching calculation result will be sent to the terminal for lighting task recommendation. If it is greater than the preset interval, it will trigger the operation control of the platform backend, and then re-evaluate the necessity of creating a lighting task.

7. The big data recommendation system based on similarity according to claim 5, It is characterized in that The processing module includes: Creation unit: used to create the platform task framework; Connection unit: used to connect the data warehouse system of the coal mining enterprise through the big data platform based on the platform task framework; Input unit: used to input the multidimensional metadata information into the data warehouse system, and select the first data warehouse required for the multidimensional metadata information using the Web interface, wherein the first data warehouse includes the library, table and column fields in the table corresponding to the data; Storage unit: used to store metadata in the first data warehouse, where the metadata is used to support data lifecycle management, security and authority management.

8. The big data recommendation system based on similarity according to claim 5, It is characterized in that The computing module includes: The first judgment unit is used to perform similarity calculation based on the set intersection and union method in discrete mathematics, and match the feature vector of the computing task preset by the system, and judge whether the tasks running with the existing task field set have an inclusion relationship according to the two field sets. If so, the inclusion relationship is marked as a match degree of 100%. If not, a matching rate calculation is performed, wherein the matching rate calculation rule is to divide the number of intersections and unions of the two column fields.

9. A big data recommendation device based on similarity, It is characterized in that include: Memory for storing computer programs; A processor, configured to implement the intelligent lighting task recommendation method as described in any one of claims 1 to 5 when executing the computer program.

10. A readable storage medium, Features: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the smart lighting task recommendation method according to any one of claims 1 to 5 is implemented.