Log data wide table portrait extraction method based on cloud service quality evaluation, storage medium, equipment and computer program product

By performing wide table image extraction and metadata association relationship management on the log data of the cloud service quality evaluation system, the problems of timeliness and resource consumption of massive log data are solved, and more efficient cloud service quality evaluation is achieved.

CN119006069APending Publication Date: 2024-11-22CHINA TELECOM DIGITAL INTELLIGENCE TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411029593.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing cloud service quality evaluation system has poor timeliness when processing massive log data, and its resource consumption is too large, which affects performance.

Method used

By extracting wide table portraits of log data, using data pools for orderly storage and metadata association management, prioritizing the cache according to the correlation between keywords and metadata, and extracting wide table portraits of log data to improve retrieval timeliness and resources Utilization.

Benefits of technology

It improves the timeliness of cloud service quality evaluation, reduces cache resource consumption, and improves system performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006069A_ABST
    Figure CN119006069A_ABST
Patent Text Reader

Abstract

The invention discloses a log data wide table portrait extraction method based on cloud service quality evaluation, a storage medium, equipment and a computer program product, and the method comprises the steps: obtaining log data, corresponding metadata and meta-process data of a cloud service quality evaluation system, and orderly storing the log data, the corresponding metadata and the meta-process data in a data pool; when cloud service quality evaluation is carried out, according to the correlation between the keyword and the metadata, searching the metadata with the maximum correlation from the data pool, and preferentially putting the metadata into a cache; finding out log data related to the metadata according to the incidence relation of the metadata in the data pool; and extracting a corresponding log data wide table portrait in the cloud service quality evaluation system according to the related log data, wherein the log data wide table portrait is used for cloud service quality evaluation. According to the method, the wide table portrait of the log data is extracted, so that the resource consumption of the cache is reduced, and the timeliness of cloud service quality evaluation is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of cloud service quality evaluation, and specifically, relates to a method, storage medium, device and computer program product for extracting wide-table portraits of log data based on cloud service quality evaluation. Background Art

[0002] After enterprises or individual users go to the cloud, cloud service providers often conduct surveys on users in order to better provide cloud services. These surveys include evaluations of the technical quality of cloud hosts, processing capabilities, system processing capabilities, and industry application carrying capacity. These are all key information bases for service providers to adjust their services. Real-time user evaluations of these indicators are particularly important. Therefore, it is necessary to obtain the true user experience as soon as possible.

[0003] Before obtaining cloud service evaluation data each time, dynamically analyzing the cloud service quality evaluation data that cloud service providers focus on and ignoring non-focused parameters is a key factor in retrieval timeliness. At present, the online search function of cloud users mainly obtains keywords and related conditions and then analyzes and obtains from the log database. When massive data is evaluated online at the same time, the timeliness is poor, and too much log data is put into the cache, which also consumes a lot of performance and resources of the cloud service quality evaluation system. Summary of the invention

[0004] In response to the problems existing in the prior art, the present invention provides a method, storage medium, device and computer program product for extracting wide-table portraits of log data based on cloud service quality evaluation. By extracting wide-table portraits of log data, the resource consumption of the cache is reduced and the timeliness of the cloud service quality evaluation is improved.

[0005] To achieve the above technical objectives, the present invention adopts the following technical solution: a method for extracting wide-table portraits of log data based on cloud service quality evaluation, which specifically includes the following steps:

[0006] Step 1: Obtain the log data, corresponding metadata and meta-process data of the cloud service quality evaluation system, and store them in the data pool in an orderly manner;

[0007] Step 2: When evaluating the quality of cloud services, based on the correlation between keywords and metadata, the metadata with the greatest correlation is searched from the data pool and placed in the cache first;

[0008] Step 3: Find out the log data related to the metadata according to the association relationship of the metadata in the data pool;

[0009] Step 4: Extract the corresponding log data wide table portrait in the cloud service quality evaluation system based on the relevant log data for cloud service quality evaluation.

[0010] Furthermore, step 1 includes the following sub-steps:

[0011] Step 1.1, according to the log data type, the data pool is divided into analog signal data pool, application data pool and text data pool;

[0012] Step 1.2, each data pool is provided with a storage tank, and the storage tank is used to store the association relationship with other storage tanks;

[0013] Step 1.3, each storage tank is provided with a log data storage block, a metadata storage block and a meta-process data storage block, each log data storage block is used to store the log data under the corresponding data pool, each metadata storage block is used to store the metadata under the corresponding data pool, and each meta-process data storage block is used to store the meta-process data under the corresponding data pool.

[0014] Furthermore, the data pool also includes a sundry data pool, and the sundry data pool is used to store abnormal log data.

[0015] Furthermore, the calculation process of the correlation between the keywords and metadata is as follows:

[0016]

[0017] Among them, Pr represents the correlation between keywords and metadata, S1 represents keywords, S2 represents metadata, ∩ represents intersection, and ∪ represents union.

[0018] Furthermore, step 4 includes the following sub-steps:

[0019] Step 4.1, extracting the corresponding user information table, cloud service information table, user feedback information table and evaluation index table in the cloud service quality evaluation system according to the relevant log data;

[0020] Step 4.2: Connect the extracted user information table, cloud service information table, user feedback information table, and evaluation index table through join to obtain a wide table portrait of log data for cloud service quality evaluation.

[0021] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the method for extracting a wide-table portrait of log data based on cloud service quality evaluation.

[0022] Furthermore, the present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for extracting wide-table portraits of log data based on cloud service quality evaluation is implemented.

[0023] Furthermore, the present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the method for extracting wide-table portraits of log data based on cloud service quality evaluation.

[0024] Compared with the prior art, the present invention has the following beneficial effects: the log data wide table portrait extraction method based on cloud service quality evaluation of the present invention stores log data, corresponding metadata and meta-process data in order through a data pool, log data and metadata, meta-process data are stored in different data storage blocks in the data pool, and the association relationship between other storage tanks is stored in the storage tank, that is, the storage tank can know the association relationship between different log data, when it is necessary to find data, the metadata and the association relationship recorded in the storage tank can improve the timeliness of retrieval, and can save resource space; at the same time, according to the correlation between keywords and metadata, the metadata with the greatest correlation is searched from the data pool and put into the cache first, because metadata has fewer fields, through metadata search, resource space can be saved; in order to obtain the maximum range of evaluation data for keywords in the cache, the corresponding log data wide table portrait is extracted through the relevant log data, and the user information table, cloud service information table, user feedback information table and evaluation index table of the user cloud service quality evaluation in the wide table portrait are extracted and put into the cache for cloud service quality evaluation. In summary, the present invention reduces the resource consumption of the cache and improves the timeliness of cloud service quality evaluation by extracting the log data wide table portrait. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flow chart of the method for extracting wide-table portraits of log data based on cloud service quality evaluation of the present invention;

[0026] Figure 2 A schematic diagram of the association relationship of the data pool. DETAILED DESCRIPTION

[0027] The technical solution of the present invention is further explained below in conjunction with the accompanying drawings.

[0028] like Figure 1 The flowchart of the method for extracting a wide table image of log data based on cloud service quality evaluation of the present invention is as follows:

[0029] Step 1: Obtain the log data, corresponding metadata and meta-process data of the cloud service quality evaluation system, and store them in a data pool in order. The present invention stores different types of log data, metadata and meta-process data in different data pools to facilitate data management. The log data, metadata and meta-process data are stored in different data storage blocks in the data pool, and the association relationship between the storage tank and other storage tanks is stored in the storage tank, that is, the storage tank can know the association relationship between different log data. When it is necessary to find data, the metadata and the association relationship recorded in the storage tank can improve the timeliness of the retrieval and save resource space; Figure 2 , specifically including the following sub-steps:

[0030] Step 1.1, according to the log data type, the data pool is divided into analog signal data pool, application data pool and text data pool;

[0031] Step 1.2, each data pool is provided with a storage tank, and the storage tank is used to store the association relationship between other storage tanks. Specifically, metadata, meta-process data and log data are associated between data storage tanks, so that when performing data query, metadata can be used to quickly search in the storage tank, thereby improving the timeliness of data search;

[0032] Step 1.3, each storage tank is provided with a log data storage block, a metadata storage block and a meta-process data storage block, each log data storage block is used to store the log data under the corresponding data pool, each metadata storage block is used to store the metadata under the corresponding data pool, and each meta-process data storage block is used to store the meta-process data under the corresponding data pool.

[0033] In a technical solution of the present invention, the data pool further includes a sundry data pool, and the sundry data pool is used to store abnormal log data, thereby preventing the abnormal log data from occupying cache resource space and consuming query time.

[0034] Step 2: When evaluating the quality of cloud services, based on the correlation between keywords and metadata, the metadata with the greatest correlation is searched from the data pool and placed in the cache first; since metadata has fewer fields, resource space can be saved by searching through metadata.

[0035] The calculation process of the correlation between keywords and metadata in the present invention is:

[0036]

[0037] Among them, Pr represents the correlation between keywords and metadata, S1 represents keywords, S2 represents metadata, ∩ represents intersection, and ∪ represents union.

[0038] Step 3: Find out the log data related to the metadata according to the association relationship of the metadata in the data pool;

[0039] Step 4: In order to obtain the maximum range of evaluation data for the keywords in the cache, the corresponding log data wide table portrait is extracted in the cloud service quality evaluation system according to the relevant log data for cloud service quality evaluation. Compared with the traditional multi-table joint query method, the data wide table portrait extraction method only consumes resource space for the corresponding metadata, avoiding large-scale resource consumption, improving computing efficiency, and thus improving the timeliness of cloud service quality evaluation; specifically, the following sub-steps are included:

[0040] Step 4.1, extracting the corresponding user information table, cloud service information table, user feedback information table and evaluation index table in the cloud service quality evaluation system according to the relevant log data;

[0041] Step 4.2: Connect the extracted user information table, cloud service information table, user feedback information table, and evaluation index table through join to obtain a wide table portrait of log data for cloud service quality evaluation.

[0042] The log data wide table portrait extraction method based on cloud service quality evaluation of the present invention stores log data, metadata, and meta-process data in different data storage blocks in the data pool, and stores the association relationship between other storage tanks in the storage tank, that is, the storage tank can know the association relationship between different log data. When it is necessary to find data, the metadata and the association relationship recorded in the storage tank can improve the timeliness of retrieval, and can save resource space; at the same time, according to the correlation between keywords and metadata, the metadata with the greatest correlation is searched from the data pool and placed in the cache first. Since metadata has fewer fields, resource space can be saved by searching through metadata; in order to obtain the maximum range of evaluation data for keywords in the cache, the corresponding log data wide table portrait is extracted through the relevant log data, and the user information table, cloud service information table, user feedback information table and evaluation index table of the user cloud service quality evaluation in the wide table portrait are extracted and placed in the cache for cloud service quality evaluation. In summary, the present invention reduces the resource consumption of the cache and improves the timeliness of cloud service quality evaluation by extracting the log data wide table portrait.

[0043] In a technical solution of the present invention, a computer-readable storage medium is also provided, storing a computer program, wherein the computer program enables a computer to execute the method for extracting a wide-table portrait of log data based on cloud service quality evaluation.

[0044] In a technical solution of the present invention, an electronic device is also provided, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for extracting wide-table portraits of log data based on cloud service quality evaluation is implemented.

[0045] In a technical solution of the present invention, a computer program product is also provided, including a computer program, which, when executed by a processor, implements the method for extracting wide-table portraits of log data based on cloud service quality evaluation.

[0046] In the embodiments disclosed in the present application, the computer storage medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. The computer storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the above. More specific examples of computer storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0047] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0048] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A method for extracting wide-table portraits of log data based on cloud service quality evaluation, characterized in that: The specific steps include: Step 1: Obtain the log data, corresponding metadata and meta-process data of the cloud service quality evaluation system, and store them in the data pool in an orderly manner; Step 2: When evaluating the quality of cloud services, based on the correlation between keywords and metadata, the metadata with the greatest correlation is searched from the data pool and placed in the cache first; Step 3: Find out the log data related to the metadata according to the association relationship of the metadata in the data pool; Step 4: Extract the corresponding log data wide table portrait in the cloud service quality evaluation system based on the relevant log data for cloud service quality evaluation.

2. According to the method for extracting wide table image of log data based on cloud service quality evaluation according to claim 1, it is characterized in that: Step 1 includes the following sub-steps: Step 1.1, according to the log data type, the data pool is divided into analog signal data pool, application data pool and text data pool; Step 1.2, each data pool is provided with a storage tank, and the storage tank is used to store the association relationship with other storage tanks; Step 1.3, each storage tank is provided with a log data storage block, a metadata storage block and a meta-process data storage block, each log data storage block is used to store the log data under the corresponding data pool, each metadata storage block is used to store the metadata under the corresponding data pool, and each meta-process data storage block is used to store the meta-process data under the corresponding data pool.

3. According to the method for extracting wide table image of log data based on cloud service quality evaluation according to claim 2, it is characterized in that: The data pool also includes a sundry data pool, and the sundry data pool is used to store abnormal log data.

4. According to a method for extracting wide table image of log data based on cloud service quality evaluation according to claim 2, it is characterized in that: The calculation process of the correlation between the keywords and metadata is as follows: Among them, Pr represents the correlation between keywords and metadata, S1 represents keywords, S2 represents metadata, ∩ represents intersection, and ∪ represents union.

5. According to the method for extracting wide table image of log data based on cloud service quality evaluation according to claim 4, it is characterized in that: Step 4 includes the following sub-steps: Step 4.1, extracting the corresponding user information table, cloud service information table, user feedback information table and evaluation index table in the cloud service quality evaluation system according to the relevant log data; Step 4.2: Connect the extracted user information table, cloud service information table, user feedback information table, and evaluation index table through join to obtain a wide table portrait of log data for cloud service quality evaluation.

6. A computer-readable storage medium storing a computer program, characterized in that: The computer program enables the computer to execute the log data wide table portrait extraction method based on cloud service quality evaluation as described in any one of claims 1-5.

7. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for extracting wide-table portraits of log data based on cloud service quality evaluation as described in any one of claims 1 to 5 is implemented.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for extracting wide-table portraits of log data based on cloud service quality evaluation as described in any one of claims 1 to 5 is implemented.

Citation Information

Patent Citations

  • Client origin information associative perception based metadata pre-acquisition method and system

    CN105279240A

  • Cloud service quality monitoring method and system

    CN109905276A

  • Distributed log data analysis optimization method based on data lake

    CN115658621A