Data processing method and device, server and storage medium

Abstract

The invention discloses a data processing method and device, a server and a storage medium, and the data processing method comprises the steps: obtaining a power consumption log file uploaded by a target device, the power consumption log file comprising a power consumption tracking log file and a first power consumption database log file; combining the power consumption tracking log file with the first power consumption database log file to obtain a second power consumption database log file; and loading the second power consumption database log file into a column database according to target configuration information. According to the method, the power consumption tracking logs and the power consumption database logs generated by the equipment can be collected and merged, and the merged file is loaded into the column database, so that a user can conveniently analyze the power consumption data generated by the equipment.

Description

Technical Field

[0001] This application relates to the technical field of electronic devices, and more particularly, to a data processing method, apparatus, electronic device, and storage medium. Background Art

[0002] With the rapid progress of technology and living standards, electronic devices (such as smart phones, tablets, etc.) have become one of the commonly used electronic products in people's lives. Usually, during the R & D and testing of electronic devices, detailed power consumption data of the electronic devices is used to locate modules with abnormal power consumption or optimize power consumption. However, in related technologies, when R & D personnel obtain and analyze power consumption detail data, they need to perform cumbersome operations, so there are inconveniences. Summary of the Invention

[0003] This application provides a data processing method, apparatus, electronic device, and storage medium, which can facilitate users to analyze power consumption data generated by the device.

[0004] In a first aspect, an embodiment of this application provides a data processing method, which includes: obtaining a power consumption log file uploaded by a target device, where the power consumption log file includes a power consumption trace log file and a first power consumption database log file; merging the power consumption trace log file and the first power consumption database log file to obtain a second power consumption database log file; and loading the second power consumption database log file into a columnar database according to target configuration information.

[0005] In a second aspect, an embodiment of this application provides a data processing apparatus, which includes: a file acquisition module, a file merging module, and a file loading module. Among them, the file acquisition module is used to obtain a power consumption log file uploaded by a target device, where the power consumption log file includes a power consumption trace log file and a first power consumption database log file; the file merging module is used to merge the power consumption trace log file and the first power consumption database log file to obtain a second power consumption database log file; and the file loading module is used to load the second power consumption database log file into a columnar database according to target configuration information.

[0006] In a third aspect, an embodiment of this application provides a server, which includes: one or more processors; a memory; one or more applications, where the one or more applications are stored in the memory and are configured to be executed by the one or more processors, and the one or more applications are configured to execute the data processing method provided in the first aspect above.

[0007] Fourthly, an embodiment of the present application provides a computer-readable storage medium. Program codes are stored in the computer-readable storage medium and can be called by a processor to execute the data processing method provided in the first aspect above.

[0008] The solution provided by the present application obtains a power consumption log file uploaded by a target device. The power consumption log file includes a power consumption tracking log file and a first power consumption database log file, merges the power consumption tracking log file with the first power consumption database log file to obtain a second power consumption database log file, and loads the second power consumption database log file into a columnar database according to target configuration information. Thus, it is possible to collect and merge the power consumption tracking logs and power consumption database logs generated by the device, and load the merged file into the columnar database, so that the required power consumption detail data can be directly queried from the columnar database for analysis, which facilitates the user to analyze the power consumption data generated by the device. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative efforts.

[0010] Figure 1 FIG. shows a schematic diagram of the application environment provided by an embodiment of the present application.

[0011] Figure 2 FIG. shows a schematic flowchart of a data processing method according to an embodiment of the present application.

[0012] Figure 3 FIG. shows a schematic flowchart of a data processing method according to another embodiment of the present application.

[0013] Figure 4 FIG. shows a schematic flowchart of a data processing method according to still another embodiment of the present application.

[0014] Figure 5 FIG. shows a schematic flowchart of a data processing method according to yet another embodiment of the present application.

[0015] Figure 6 FIG. shows a block diagram of a data processing device according to an embodiment of the present application.

[0016] Figure 7 FIG. is a block diagram of an electronic device for executing the data processing method according to an embodiment of the present application.

[0017] Figure 8It is a storage unit for storing or carrying program codes for implementing the data processing method according to the embodiments of the present application. Detailed implementation manners

[0018] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.

[0019] During the research and development and testing of current electronic devices, when developers need to obtain detailed power consumption data of an electronic device to locate power consumption abnormal modules or optimize power consumption, they need to obtain the power consumption detail data stored in the electronic device, and then analyze the power consumption detail data to complete the required tasks. There are usually two data types of the power consumption detail data stored in the electronic device. Among them, summary and initialization data are stored in a database (DB) file, and streaming time data is stored in a trace file.

[0020] In the related art, when collecting the power consumption detail data of an electronic device, developers usually connect a test device to a mobile terminal through a data cable, and use an ADB tool or other similar tools to obtain the power consumption log data of the test device, including DB and trace files; then manually transfer the obtained DB and trace files to the developer's work computer; then, the developer uses a local program on the computer to merge the DB and trace files into unified DB data. After the merging is completed, the developer can use local tools to query or analysis tools to view the data for data analysis and processing. In such a way, it will consume a lot of workload and time in obtaining data during the research and development process.

[0021] In addition, after obtaining the power consumption log of the test device, if sharing analysis is required, then the file can only be manually transferred to other people's devices for analysis, which leads to data insecurity problems during the transfer process; and, after merging to obtain the DB log, when developers need to analyze the data, they can only open it locally through the DB database tool, and then write their own analysis SQL to process the data, which results in no intuitive analysis chart and cannot share the analysis method, thus bringing inconvenience to the developers' analysis of the power consumption detail data and affecting the developers' research and development efficiency.

[0022] In view of the above problems, the inventors propose a data processing method, apparatus, electronic device, and storage medium provided by the embodiments of the present application, which can collect and merge the power consumption tracking logs and power consumption database logs generated by the device, and load the merged file into a columnar database, so that the power consumption detail data required can be directly queried from the columnar database for analysis, thereby facilitating the user to analyze the power consumption data generated by the device. Among them, the specific data processing method will be described in detail in the subsequent embodiments.

[0023] First, the scenarios involved in the embodiments of the present application will be introduced below.

[0024] As Figure 1 shown, in the Figure 1 scenario shown, there is a server 100 and at least one electronic device 200 (only 2 are shown in the figure). Among them, the electronic device 200 and the server 100 can communicate through a network to upload the stored power consumption log file to the server 100; according to the power consumption log file uploaded by the electronic device 200, the server 100 can merge the power consumption tracking log file and the power consumption database log file in the power consumption log file, and then load the merged power consumption database log file into a columnar database, so that the power consumption detail data required can be directly queried from the columnar database for analysis, thereby facilitating the user to analyze the power consumption data generated by the device.

[0025] The server 100 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The electronic device 200 and the server 100 can be directly or indirectly connected through wired or wireless communication methods, and the present invention does not limit this here. The electronic device 100 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto.

[0026] Next, the data processing method provided by the embodiments of the present application will be introduced in detail with reference to the accompanying drawings.

[0027] Please refer to Figure 2 , Figure 2 which shows a schematic flowchart of a data processing method provided by an embodiment of the present application. In a specific embodiment, the data processing method is applied to a data processing apparatus 500 as Figure 6 shown and a server 100 configured with the data processing apparatus 500 ( Figure 7), the server 100 can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server. The following will be directed to Figure 2 The process shown will be elaborated in detail. The data processing method may specifically include the following steps:

[0028] Step S110: Obtain a power consumption log file uploaded by a target device. The power consumption log file includes a power consumption trace log file and a first power consumption database log file.

[0029] In an embodiment of the present application, the server can obtain the power consumption log file uploaded by the target device, so as to merge different types of log files in the power consumption log file, obtain a structured log file and then load it into a columnar database, enabling the user to directly query the required power consumption detail data from the columnar database for analysis, thereby facilitating the user's analysis of the power consumption data generated by the device. Among them, the target device can be a device that needs to collect power consumption log files for power consumption detail data analysis. For example, the target device can be a test device. When the test device is being tested, it can record the power consumption detail data and store it in the power consumption log file. The power consumption log file can include the power consumption trace file stored by the target device and the power consumption database (Data Base, DB) file (as the first DB log file).

[0030] In some embodiments, the target device can store the summary and initialization data related to power consumption in the power consumption DB log file, store the streaming time data related to power consumption in the trace log file, and retain the power consumption DB log file and the trace log file at a specified storage location. When the target device needs to upload the power consumption log file, it can collect the power consumption log file at the specified storage location and transmit the collected power consumption log file to the server through the information transmission channel between the target device and the server, thereby realizing the upload of the stored power consumption log file to the server. Correspondingly, the server can receive the above power consumption log file uploaded by the target device according to the information transmission channel between the target device and the server.

[0031] In a possible embodiment, the target device can obtain the power consumption log file it stores and upload the power consumption log file to the server when the power consumption data upload condition is met. Correspondingly, the server can receive the power consumption log file uploaded by the target device when the power consumption data upload condition is met.

[0032] Optionally, the above upload condition may be that the duration since the last upload of the power consumption log file is greater than or equal to the target duration. Among them, the target device can start timing after each upload of the power consumption log file, and when the timing duration reaches the target duration, the newly generated power consumption log file can be uploaded to the server.

[0033] Optionally, the above upload condition may also be receiving a log upload instruction sent by the server. That is to say, when the target device receives a data upload instruction sent by the server, it can determine that it meets the above upload condition and upload the stored power consumption log file to the server.

[0034] Optionally, the above upload condition may also be detecting a log upload operation input by the user. That is to say, when the target device detects a log upload operation input by the user, it can determine that it meets the above upload condition and actively upload the stored power consumption log file to the server. For example, the target device can display an upload management interface for the power consumption log file, and the upload management interface may include an upload control. When the target device detects a trigger operation for the upload control, it can determine that the above log upload operation has been detected, and then in response to the log upload operation, upload the stored power consumption log file to the server.

[0035] Step S120: Merge the power consumption tracking log file with the first power consumption database log file to obtain a second power consumption database log file.

[0036] In the embodiment of the present application, after the server obtains the above power consumption log file, it can merge the power consumption trace log file in the above power consumption log file with the first power consumption DB log file to obtain a new power consumption DB log file, which is used as the second power consumption DB file. It can be understood that the types of log files in the electronic device can include unstructured log files and structured log files. Structured log files adopt a unified data structure, including the time, type, level, event information, etc. of the event. Unstructured log files do not have a unified data structure, and the trace log file is an unstructured log file; the DB log file is a file in the electronic device used to store database data, generally used to store structured data, and can achieve efficient data management, retrieval, etc. When storing power consumption detail data, it is used to store the summary and initialization data related to power consumption; while the trace log file is an unstructured file, and when storing power consumption detail data, it is used to store the streaming time data related to power consumption, and unstructured files cannot be directly queried and read. Therefore, in order to load the data in the power consumption log file into the columnar database later, so that the user can directly query the required power consumption detail data from the columnar database for analysis, the power consumption trace log file can be merged with the above first power consumption DB log file, and then a new structured DB log file can be obtained, so as to convert the power consumption trace log file into a structured file, and then the data in the obtained power consumption trace log file and power consumption DB log file can be loaded into the columnar database later. Among them, the data in the structured log file can be data based on Structured Query Language (SQL). SQL data can be stored in the database, and through operations such as querying and statistics on the database, automated data analysis can be achieved.

[0037] In some embodiments, when the server merges the above power consumption trace log file with the first power consumption DB log file, it can transcode and serialize the power consumption trace log file through a corresponding conversion tool, then convert it into a specific data structure type based on the Android PROTO structure, and then merge the converted data with the first power consumption DB log file. Of course, the specific method for the server to merge the power consumption trace log file and the power consumption DB log file can be not limited.

[0038] In a possible implementation, after the server merges the above power consumption trace log file and the first power consumption DB log file to obtain the second power consumption DB log file, it can also obtain the summary information of the second power consumption DB log file; the server can store the obtained second power consumption DB log file and the summary information, so as to provide the summary information to the columnar database subsequently, so that the summary information can be used to query the power consumption detail data. Among them, the summary information may include the date of the file, the device information corresponding to the file, etc., and the device information may be the device serial number, etc.

[0039] Step S130: Load the second power consumption database log file into the columnar database according to the target configuration information.

[0040] In the embodiments of the present application, after merging the above power consumption trace log file and the first power consumption DB log file to obtain the second power consumption DB log file, the obtained second power consumption database log file can be loaded into the columnar database according to the pre-configured target configuration information, so as to facilitate the user to directly query the required power consumption detail data from the columnar database for analysis. Among them, the target configuration information may be the configuration information required when the power consumption DB log file is loaded into the columnar database. The target configuration information may include the storage path, the table name of the required data table, the name of the power consumption DB log file corresponding to each data table, etc. Of course, the specific target configuration information may not be limited. For example, the target configuration information may further include the import type, and the import type may be hash import, non-hash import, etc.

[0041] In some implementations, the columnar database may be a ClickHouse database, an online analytical database in the big data field, which can query and analyze massive data through SQL language. Before the online query and analysis of the power consumption detail data need to be implemented, the above-obtained second power consumption DB log file needs to be loaded into the ClickHouse database, so that the online query and analysis of the power consumption detail data can be performed.

[0042] In some embodiments, obtaining the above power consumption log file, merging the above power consumption trace log file with the first power consumption DB log file, and writing the merged power consumption DB log file into a columnar database can be performed by the same server. That is to say, the server can be deployed with both a DB database and a columnar database to obtain the above power consumption log file, merge to obtain a new power consumption DB log file, store the power consumption DB log file, and be able to load the obtained power consumption DB log file into the columnar database; obtaining the above power consumption log file, merging the above power consumption trace log file with the first power consumption DB log file, and writing the merged power consumption DB log file into a columnar database can also be performed by different servers. Specifically, the DB database and the columnar database can be deployed on different servers. The server deployed with the DB database can execute the above steps S110 and S120, and the server deployed with the columnar database can execute the above step S130.

[0043] The data processing method provided by the embodiments of the present application obtains a power consumption log file uploaded by a target device. The power consumption log file includes a power consumption trace log file and a first power consumption database log file, merges the power consumption trace log file with the first power consumption database log file to obtain a second power consumption database log file, and loads the second power consumption database log file into a columnar database according to target configuration information. Thereby, it can collect and merge the power consumption trace log and the power consumption database log generated by the device, and load the merged file into the columnar database, so that the required power consumption detail data can be directly queried and analyzed from the columnar database, without the user collecting the power consumption detail data or merging the log files, thus facilitating the user to analyze the power consumption data generated by the device.

[0044] Please refer to Figure 3 , Figure 3 which shows a schematic flowchart of a data processing method provided by another embodiment of the present application. This data processing method is applied to the above server, and the following will elaborate on the Figure 3 shown process in detail. The data processing method can specifically include the following steps:

[0045] Step S210: Obtain a power consumption log file uploaded by a target device, where the power consumption log file includes a power consumption trace log file and a first power consumption database log file.

[0046] Step S220: Merge the power consumption trace log file with the first power consumption database log file to obtain a second power consumption database log file.

[0047] In the embodiment of the present application, steps S210 and S220 may refer to the content of the foregoing embodiments, and will not be elaborated herein.

[0048] Step S230: Create a target data table in the columnar database according to the target configuration information.

[0049] In the embodiment of the present application, the target configuration information may include information about at least one data table into which the data in the power consumption DB log file needs to be written when the power consumption DB log file is loaded into the columnar database. The information of the data table may include the table name of the data table, etc. When the server loads the obtained second power consumption DB log file into the columnar database, it may create a corresponding data table (as the target data table) according to the information of the data table in the target configuration information.

[0050] In some embodiments, when the server creates a target data table according to the target configuration information, it may determine a corresponding table creation statement according to the information of the data table in the target configuration information; then create the above target data table based on the table creation statement. The table creation statement may indicate the name of the data table to be created, the table fields, the data types corresponding to the table fields, and may also indicate the partition fields and sorting fields when data is written.

[0051] Step S240: Write the second power consumption database log file into the target data table.

[0052] In the embodiment of the present application, after the server creates the above target data table, it may write the second power consumption DB log file into the created target data table, thereby completing the loading of the merged second power consumption DB log file into the columnar database.

[0053] In some embodiments, when analyzing power consumption detail data, it may be necessary to analyze different devices or different functional modules. Therefore, there are usually multiple data tables. Thus, the number of the above target data tables may be multiple. When writing the second power consumption DB log file into the created target data tables, the database log file corresponding to each target data table may be obtained from the second power consumption database log file according to the target configuration information; and the database log file corresponding to each target data table is written into each target data table. The target configuration information may further include the name of the power consumption DB log file corresponding to each data table. Therefore, according to this name, the corresponding file can be obtained from the above second power consumption DB log file for each target data table as the DB log file corresponding to the target data table.

[0054] In a possible implementation, the server can create corresponding target data tables through the SQLite engine according to the information of the data tables in the target configuration information and the names of the power consumption DB log files corresponding to each data table, and obtain the corresponding files from the above-mentioned second power consumption DB log files and write them into the target data tables.

[0055] Optionally, the above columnar database is a ClickHouse database. Since different database engines involve different field types and there will be compatibility problems, such as the int length problem, the time granularity problem, etc., in the ClickHouse database, the field type can be of String type.

[0056] The data processing method provided by the embodiments of the present application can collect and merge the power consumption trace log files and power consumption DB log files generated by the device. For the power consumption DB log files obtained after merging, corresponding data tables are created in the columnar database, and then the obtained power consumption DB log files are written into the created data tables, so that the power consumption detail data required can be directly queried and analyzed from these data tables in the columnar database, thus facilitating the user to analyze the power consumption data generated by the device.

[0057] Please refer to Figure 4 , Figure 4 which shows a schematic flowchart of the data processing method provided by another embodiment of the present application. This data processing method is applied to the above-mentioned server. The following will elaborate in detail on Figure 4 the process shown. The data processing method may specifically include the following steps:

[0058] Step S310: Obtain the power consumption log files uploaded by the target device, where the power consumption log files include power consumption trace log files and the first power consumption database log files.

[0059] Step S320: Merge the power consumption trace log files and the first power consumption database log files to obtain the second power consumption database log files.

[0060] Step S330: Load the second power consumption database log files into the columnar database according to the target configuration information.

[0061] In the embodiments of the present application, steps S310 to S330 may refer to the content of the foregoing embodiments and will not be elaborated herein.

[0062] Step S340: Obtain the first analysis request sent by the client.

[0063] In the embodiments of the present application, after merging the power consumption trace log file and the power consumption DB log file in the collected power consumption log file and loading the obtained power consumption DB log file into the columnar database, a user who needs to query and analyze the power consumption detail data of the electronic device can initiate a first analysis request to the server through the client, so as to implement the query and analysis required by the user for the power consumption detail data of the electronic device; correspondingly, the server can receive the first analysis request sent by the client. The above client can be any device with communication and storage functions, including but not limited to a PC (Personal Computer), a PDA (tablet computer), a smart TV, a smart phone, a smart wearable device or other intelligent communication devices with network connection functions. Optionally, the client can be an application client or a web client, which is not limited herein.

[0064] In some embodiments, the client can be a web front end for data query and analysis or an application software on a terminal device. The visual presentation on the client interface can directly allow the user to experience all functions and set corresponding operations according to needs, and then generate relevant data requests to be responded to and executed by the server. Optionally, the client can display a query and analysis interface, and the client can initiate a first analysis request to the server according to the operations in the query and analysis interface.

[0065] In a possible embodiment, the above first analysis request may carry an analysis rule for querying and analyzing power consumption detail data. The analysis rule can be a selection condition set for the power consumption detail data to be queried and analyzed. According to these selection conditions, the user can customize the data processing rule, and thus more diverse analysis rules can also be obtained. For example, the analysis rule may include a data screening condition for the power consumption detail data, and this data screening condition can be used to query the corresponding power consumption detail data from the data stored in the server.

[0066] Step S350: In response to the first analysis request, obtain target power consumption detail data from the columnar database.

[0067] In the embodiments of the present application, after the server obtains the above first analysis request, it can, in response to the first analysis request, obtain the target power consumption detail data to be analyzed this time from the columnar database. Among them, the server can query the corresponding power consumption detail data from the columnar database according to the information carried in the first analysis request as the target power consumption detail data to be analyzed this time.

[0068] In some embodiments, the above first analysis request may carry data screening conditions, which can be used to screen corresponding power consumption detail data from the columnar database as the target power consumption detail data to be analyzed this time. For example, the data screening conditions may include device identification information of the device (such as device number, etc.), date and other information. When the server obtains the target power consumption detail data from the columnar database in response to the first analysis request, it can query the corresponding power consumption detail data from the columnar database according to the above data screening conditions as the above target power consumption detail data. Among them, the server can determine the target query statement corresponding to the above data screening conditions in response to the first analysis request, and then based on the target query statement, query the corresponding power consumption detail data from the columnar database as the above target power consumption detail data.

[0069] Among them, the query statement can be an SQL query statement or other languages that can achieve efficient query, which is not limited here. A series of query interfaces can be predefined in the columnar database. Each interface implements a specific query mode, and the mode definition contains all the information of a data query, including data screening conditions. According to the data screening conditions, the query analysis engine can determine the target query interface from the predefined multiple query interfaces, and then use the query analysis engine to parse the target query interface to generate the corresponding query statement. Optionally, the front-end page displayed by the client can automatically render the front-end menu component according to the meta-information of the query mode, realizing the automation of the data query analysis system. Users can assemble different query interfaces into a query view. Each time this view is opened, all the query interfaces included in the view will be automatically submitted to the query analysis engine to generate specific query statements. By defining the query interface, the scalability and operability of data analysis are guaranteed. The server can pass the query statement to the data access layer service, and the data access layer service queries the corresponding power consumption detail data from the columnar database according to the query statement.

[0070] Step S360: Analyze and process the target power consumption detail data to obtain the corresponding analysis and processing result.

[0071] In the embodiments of the present application, since the data stored in the columnar database is the data obtained by loading the power consumption DB log file into the columnar database, the above-obtained target power consumption detail data is also structured data. Therefore, after querying the above target power consumption detail data, the pre-configured data processing logic can be executed to analyze and process the structured data to obtain the analysis and processing result.

[0072] Step S370: Return the analysis and processing result to the client.

[0073] In an embodiment of the present application, after obtaining the above analysis and processing results, the analysis and processing results can be returned to the client; correspondingly, the client can receive the analysis and processing results returned by the server. After the client obtains the above analysis and processing results, the analysis and processing results can be displayed to help the user perform relevant analysis.

[0074] The data processing method provided by the embodiment of the present application is to collect and merge the power consumption tracking logs and power consumption database logs generated by the device, and load the merged file into the columnar database. The server can obtain the corresponding power consumption detail data from the columnar database according to the analysis request initiated by the client, perform analysis and processing on the obtained power consumption detail data, obtain the analysis and processing results, and feedback the analysis and processing results to the client, so as to realize the online query and analysis of the power consumption detail data by the user, and facilitate the user to analyze the power consumption data generated by the device.

[0075] Please refer to Figure 5 , Figure 5 which shows a schematic flowchart of the data processing method provided by another embodiment of the present application. This data processing method is applied to the above-mentioned server. The following will elaborate in detail on Figure 5 the process shown. The data processing method may specifically include the following steps:

[0076] Step S410: Obtain the power consumption log file uploaded by the target device, where the power consumption log file includes a power consumption tracking log file and a first power consumption database log file.

[0077] Step S420: Merge the power consumption tracking log file and the first power consumption database log file to obtain a second power consumption database log file.

[0078] Step S430: Load the second power consumption database log file into the columnar database according to the target configuration information.

[0079] In an embodiment of the present application, steps S410 to S430 may refer to the content of the foregoing embodiment and will not be elaborated herein.

[0080] Step S440: Obtain a first analysis request sent by the client, where the first analysis request carries a target chart type.

[0081] Step S450: In response to the first analysis request, obtain target power consumption detail data from the columnar database.

[0082] In an embodiment of the present application, the above first analysis request may carry a target chart type. That is to say, when analyzing and querying power consumption detail data, in addition to setting data filtering conditions, the way the processing result wants to be presented can also be set to implement the analysis and processing of target data. For example, the target power consumption detail data can be displayed on the analysis interface through data such as line charts, pie charts, and bar charts.

[0083] Step S460: Use a graphics conversion adapter to analyze and process the target power consumption detail data to obtain chart data corresponding to the target chart type as the analysis and processing result.

[0084] In an embodiment of the present application, the server can use a graphics conversion adapter to analyze and process the target power consumption detail data to obtain chart data corresponding to the target chart type as the analysis and processing result. It can be understood that the original expression of the target power consumption detail data obtained by the above query is in tabular form, but this form alone is not sufficient to support data analysis requirements. Therefore, specific analysis and processing of the target power consumption detail data are required to generate corresponding processing results. For example, the power consumption detail data of each device is statistically processed to generate a corresponding statistical chart; the target chart type corresponds to statistically processed chart data, and the chart data can be used as the final processing result to help users perform relevant analysis.

[0085] Step S470: Return the analysis and processing result to the client.

[0086] In an embodiment of the present application, after obtaining the above analysis and processing result, the analysis and processing result can be returned to the client. Correspondingly, the client can receive the analysis and processing result returned by the server. After the client obtains the above analysis and processing result, the chart data in the analysis and processing result can be displayed according to the above target chart type to help users perform relevant analysis. Thus, a visual interface can be provided for users to view the processing result of the power consumption detail data.

[0087] The data processing method provided by the embodiment of the present application is to collect and merge the power consumption tracking logs and power consumption database logs generated by the device, and load the merged file into the columnar database. The server can obtain corresponding power consumption detail data from the columnar database according to the analysis request initiated by the client, analyze and process the obtained power consumption detail data to obtain the analysis and processing result, and feedback the analysis and processing result to the client, thereby realizing the online query and analysis of the power consumption detail data by the user, which facilitates the user to analyze the power consumption data generated by the device. In addition, when generating the analysis and processing result, the server can generate the analysis and processing result according to the chart type carried in the analysis request, so as to provide a visual interface for users to view the processing result of the power consumption detail data.

[0088] Please refer to Figure 6 which shows a structural block diagram of a data processing device 500 provided by an embodiment of the present application. The data processing device 500 applies the above-mentioned server. The data processing device 500 includes: a file acquisition module 510, a file merging module 520, and a file loading module 530. Among them, the file acquisition module 510 is used to acquire a power consumption log file uploaded by a target device, and the power consumption log file includes a power consumption tracking log file and a first power consumption database log file; the file merging module 520 is used to merge the power consumption tracking log file and the first power consumption database log file to obtain a second power consumption database log file; the file loading module 530 is used to load the second power consumption database log file into a columnar database according to target configuration information.

[0089] In some embodiments, the file loading module 530 can be specifically used to create a target data table in the columnar database according to the target configuration information; write the second power consumption database log file into the target data table.

[0090] In a possible embodiment, the file loading module 530 can also be used to obtain a database log file corresponding to each target data table from the second power consumption database log file according to the target configuration information; write the database log file corresponding to each target data table into each target data table.

[0091] In some embodiments, the data processing device 500 may further include a request acquisition module, a request response module, a data analysis module, and a result return module. The request acquisition module is used to acquire a first analysis request sent by a client after the second power consumption database log file is loaded into the columnar database; the request response module is used to respond to the first analysis request and acquire target power consumption detail data from the columnar database; the data analysis module is used to analyze and process the target power consumption detail data to obtain a corresponding analysis and processing result; the result return module is used to return the analysis and processing result to the client.

[0092] In a possible embodiment, the first analysis request carries a data filtering condition, and the request response module can be specifically used to respond to the first analysis request, determine a target query statement corresponding to the data filtering condition; query corresponding power consumption detail data from the columnar database based on the target query statement as the target power consumption detail data.

[0093] In a possible implementation manner, the target chart type is carried in the first analysis request. The data analysis module may specifically be used to analyze and process the target power consumption detail data by using a graphic conversion adapter to obtain chart data corresponding to the target chart type as the analysis processing result.

[0094] In some implementation manners, the columnar database is a ClickHouse database.

[0095] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described devices and modules may refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0096] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical, or other forms of coupling.

[0097] In addition, in each embodiment of the present application, each functional module may be integrated in a processing module, or each module may exist physically alone, or two or more modules may be integrated in one module. The above-integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0098] In summary, for the solution provided in the present application, by obtaining a power consumption log file uploaded by a target device, where the power consumption log file includes a power consumption tracking log file and a first power consumption database log file, merging the power consumption tracking log file and the first power consumption database log file to obtain a second power consumption database log file, and loading the second power consumption database log file into a columnar database according to target configuration information. Thus, it is possible to collect and merge the power consumption tracking log and the power consumption database log generated by the device, and load the merged file into the columnar database, so that the power consumption detail data required can be directly queried from the columnar database for analysis, thereby facilitating the user to analyze the power consumption data generated by the device.

[0099] Please refer to Figure 7, which shows a structural block diagram of a server provided by an embodiment of the present application. The server 100 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server 100 in the present application may include one or more of the following components: a processor 110, a memory 120, and one or more application programs, where one or more application programs may be stored in the memory 120 and configured to be executed by one or more processors 110, and one or more application programs are configured to execute the methods described in the foregoing method embodiments.

[0100] The processor 110 may include one or more processing cores. The processor 110 uses various interfaces and lines to connect various parts within the entire server 100, and by running or executing instructions, programs, code sets, or instruction sets stored in the memory 120, and by calling data stored in the memory 120, it executes various functions of the server 100 and processes data. Optionally, the processor 110 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 110 may integrate a combination of one or several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, and application programs, etc.; the GPU is responsible for rendering and drawing display content; the modem is used to process wireless communications. It can be understood that the above modem may not be integrated into the processor 110 and may be implemented separately through a communication chip.

[0101] The memory 120 may include a Random Access Memory (RAM), or may also include a Read-Only Memory. The memory 120 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 120 may include a program storage area and a data storage area. Among them, the program storage area can store instructions for implementing the operating system, instructions for implementing at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area can also store data created by the server 100 during use (such as phone book, audio and video data, chat record data, etc.).

[0102] Please refer to Figure 8 , which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. Program code is stored in the computer-readable medium 800, and the program code can be called by a processor to execute the method described in the above method embodiments.

[0103] The computer-readable storage medium 800 can be an electronic memory such as a flash memory, an Electrically Erasable Programmable Read-Only Memory (EEPROM), an EPROM, a hard disk or a ROM. Optionally, the computer-readable storage medium 800 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 800 has a storage space for the program code 810 for executing any method step in the above method. These program codes can be read out from or written into one or more computer program products. The program code 810 can be compressed in an appropriate form, for example.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

1. A data processing method, characterized in that, The method includes: Obtaining a power consumption log file uploaded by a target device, where the power consumption log file includes a power consumption tracking log file and a first power consumption database log file; Merging the power consumption tracking log file and the first power consumption database log file to obtain a second power consumption database log file; Loading the second power consumption database log file into a columnar database according to target configuration information.

2. The method according to claim 1, wherein The loading of the second power consumption database log file into the columnar database includes: Creating a target data table in the columnar database according to target configuration information; Writing the second power consumption database log file into the target data table.

3. The method according to claim 2, wherein There are multiple target data tables, and the writing of the second power consumption database log file into the target data tables includes: Obtaining the database log file corresponding to each target data table from the second power consumption database log file according to the target configuration information; Writing the database log file corresponding to each target data table into each target data table.

4. The method according to claim 1, wherein After the second power consumption database log file is loaded into the columnar database, the method further includes: Obtaining a first analysis request sent by a client; In response to the first analysis request, obtaining target power consumption detail data from the columnar database; Performing analysis processing on the target power consumption detail data to obtain a corresponding analysis processing result; Returning the analysis processing result to the client.

5. The method according to claim 4, wherein The first analysis request carries a data filtering condition, and the obtaining of the target power consumption detail data from the columnar database in response to the first analysis request includes: In response to the first analysis request, determining a target query statement corresponding to the data filtering condition; Based on the target query statement, querying the corresponding power consumption detail data from the columnar database as the target power consumption detail data.

6. The method according to claim 4, wherein The first analysis request carries a target chart type, and the performing of analysis processing on the target power consumption detail data to obtain a corresponding analysis processing result includes: Using a graphic conversion adapter to perform analysis processing on the target power consumption detail data to obtain chart data corresponding to the target chart type as the analysis processing result.

7. The method according to any one of claims 1-6, characterized in that, The columnar database is a ClickHouse database.

8. A data processing device, characterized in that, The device includes: a file acquisition module, a file merging module, and a file loading module, where The file acquisition module is used to obtain a power consumption log file uploaded by a target device, where the power consumption log file includes a power consumption tracking log file and a first power consumption database log file; The file merging module is used to merge the power consumption tracking log file and the first power consumption database log file to obtain a second power consumption database log file; The file loading module is used to load the second power consumption database log file into a columnar database according to target configuration information.

9. A server, characterized in that, Includes: One or more processors; A memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to perform the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Program code is stored in the computer-readable storage medium, and the program code can be called by a processor to execute the method according to any one of claims 1-7.

Images