Data processing method of data processing system

Abstract

According to an embodiment disclosed in the present document, a data processing method comprises the steps of: generating one or more analyzed results by analyzing battery data by means of a first analysis engine among a plurality of analysis engines driven independently of each other, and generating a first compressed file by compressing the one or more analyzed results; storing the first compressed file in a data lake; extracting the first compressed file stored in the data lake and transmitting the extracted first compressed file to an analysis device; acquiring the one or more analyzed results by decompressing the first compressed file by means of the analysis device; generating a plurality of data frames by processing the one or more analyzed results; and storing the plurality of data frames in a database.

Description

Data processing method of a data processing system

[0001] This application claims the benefit of priority based on Korean Patent Application No. 10-2024-0124023 filed on September 11, 2024, and all contents disclosed in the document of said Korean Patent Application are incorporated herein as part of this specification.

[0002] The embodiments disclosed in this document relate to a data processing method of a data processing system.

[0003] Recently, active research and development on secondary batteries has been underway. Here, secondary batteries refer to rechargeable batteries, encompassing conventional Ni / Cd and Ni / MH batteries as well as the more recent lithium-ion batteries. Among secondary batteries, lithium-ion batteries have the advantage of significantly higher energy density compared to conventional Ni / Cd and Ni / MH batteries. Furthermore, lithium-ion batteries can be manufactured in a compact and lightweight manner, making them suitable for use as power sources for mobile devices. Recently, their scope of application has expanded to include electric vehicles, drawing attention as a next-generation energy storage medium.

[0004] As batteries are increasingly used as the power source for electric vehicles, the management and analysis of data from these installed batteries are becoming crucial. In particular, driven by technological advancements, the types of battery data are becoming increasingly sophisticated and diverse. Consequently, efficient data processing is essential for analyzing and managing battery data.

[0005] One objective of the embodiments disclosed in this document is to provide a data processing method of a data processing system capable of efficiently performing battery data processing.

[0006] In addition, one objective of the embodiments disclosed in this document is to provide a data processing method of a data processing system capable of reducing data storage capacity and processing load and reducing database connection load during the processing of battery data.

[0007] The technical problems of the embodiments disclosed in this document are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by those skilled in the art from the description below.

[0008] According to an embodiment disclosed in this document, a data processing method may include the steps of: analyzing battery data by a first analysis engine among a plurality of analysis engines that operate independently of each other to generate one or more analysis results, and compressing the one or more analysis results to generate a first compressed file; storing the first compressed file in a data lake; extracting the first compressed file stored in the data lake and transmitting it to an analysis device; obtaining the one or more analysis results by decompressing the first compressed file by the analysis device; processing the one or more analysis results to generate a plurality of data frames; and storing the plurality of data frames in a database.

[0009] According to an embodiment, the plurality of analysis engines may be characterized by being configured with an MSA structure.

[0010] According to an embodiment, the plurality of analysis engines include different analysis algorithms, and the step of generating the first compressed file may include the step of generating one or more analysis results corresponding to the analysis algorithm of the first analysis engine.

[0011] According to an embodiment, the step of transmitting to the analysis device may be characterized in that the first compressed file stored in the data lake is transmitted sequentially to the analysis device by a message queue.

[0012] According to an embodiment, the step of generating the plurality of data frames may include: a step of converting data included in one or more analysis results to correspond to a plurality of schema structures; and a step of generating the plurality of data frames by grouping the data corresponding to each of the plurality of schema structures together.

[0013] According to an embodiment, each of the plurality of schema structures may be characterized as corresponding to the storage structure of each of the plurality of tables included in the database.

[0014] According to an embodiment, the step of storing in the database may be characterized by loading each of the plurality of data frames into a table having a corresponding storage structure.

[0015] According to an embodiment, the step of storing in the database may include: borrowing the connection object from a connection pool in which a connection object is stored and in which a connection object has been established with the database; and loading each of the plurality of data frames into the database using the connection object.

[0016] According to an embodiment, the step of storing in the database may further include the step of returning the connection object to the connection pool.

[0017] The data processing method of a data processing system according to the embodiments disclosed in this document can reduce storage capacity associated with data processing, reduce data processing load, and increase the efficiency of data processing by compressing and processing analysis results generated from analysis engines.

[0018] In addition, various effects that can be identified directly or indirectly through this document may be provided.

[0019] FIG. 1 is a flowchart illustrating a data processing process according to one embodiment disclosed in this document.

[0020] FIG. 2 is a diagram showing the structure of a data processing system according to one embodiment disclosed in this document.

[0021] FIG. 3 is a drawing showing the overall structure of a data processing system according to one embodiment disclosed in this document.

[0022] FIG. 4 is a flowchart illustrating a data processing method according to one embodiment disclosed in this document.

[0023] Hereinafter, various embodiments of the present invention are described with reference to the accompanying drawings. However, this is not intended to limit the present invention to specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of the present invention.

[0024] In this document, the singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B or C,” “at least one of A, B and C,” and “at least one of A, B, or C” may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as “first,” “second,” or “first” or “second” may be used simply to distinguish a component from another corresponding component and do not limit the components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0025] Each component (e.g., module or program) of the components described in this document may include a singular or multiple entities. According to various embodiments, one or more of the components or operations may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as they were performed by the corresponding component among the multiple components prior to the integration. According to various embodiments, operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0026] As used in this document, the terms "module" or "...part" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0027] Various embodiments of this document may be implemented as software (e.g., a program or application) comprising one or more instructions stored in a storage medium (e.g., memory) readable by a machine. For example, the processor of the machine may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by a machine may be provided in the form of a non-transitory storage medium. Here, "non-transitory" simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium.

[0028] FIG. 1 is a flowchart illustrating a data processing process according to an embodiment disclosed in this document. FIG. 2 is a diagram showing the structure of a data processing system according to an embodiment disclosed in this document.

[0029] Referring to FIGS. 1 and 2, the data processing system (10) can perform a series of processes to analyze battery data, process the analysis results generated, and store the processed data in a database. For example, the data processing system (10) can store the processed data in a database through data processing processes of extracting, transforming, and loading the analysis results generated by analyzing battery data.

[0030] The data processing system (10) can process the analysis results generated by analyzing battery data, and as the importance of battery data analysis increases, the types and amounts of battery data used for analysis are increasing. As the amount of battery data increases, the increase in the analysis result data of the data processing system (10) can rapidly increase the storage capacity required for data storage, the increase in data processing load, and the increase in database connection load.

[0031] Accordingly, the data processing system (10) can reduce the required storage capacity and data processing load and increase the efficiency of data processing by compressing and processing the analysis results of the battery data. In addition, the data processing system (10) can reduce the database connection load by utilizing connection objects of the connection pool during the process of loading the processed data into the database.

[0032] Hereinafter, with reference to FIGS. 1 and FIGS. 2, a data processing method of a data processing system (10) will be described. The data processing method of the data processing system (10) may include a series of processes for analyzing battery data, extracting, transforming, and loading the analysis results.

[0033] First, referring to FIG. 1, in step S110, the data processing system (10) can generate a first compressed file. More specifically, the data processing system (10) can generate a first compressed file by the first analysis engine (110).

[0034] The first analysis engine (110) may be any one of a plurality of analysis engines that operate independently of each other. The plurality of analysis engines may include analysis engines that operate independently of each other, and any one of the plurality of analysis engines may be referred to as the first analysis engine.

[0035] According to one embodiment, a plurality of analysis engines may be configured in a Microservices Architecture (MSA) structure. The plurality of analysis engines may operate independently of each other and may have an MSA structure in which mutual dependencies are minimized. An MSA structure refers to an architecture that enables the modification and combination of applications by implementing a single large application by dividing it into multiple small applications, and each analysis engine may operate as a single application.

[0036] Each of the plurality of analysis engines can analyze battery data and generate an analysis result. To this end, each of the plurality of analysis engines may include an analysis algorithm for generating an analysis result, and according to one embodiment, each of the plurality of analysis engines may include different analysis algorithms.

[0037] The analysis algorithm may include various types of algorithms, such as a RUL algorithm for analyzing the battery's RUL (Remaining Useful Lifecycle), a SOH algorithm for analyzing the battery's SOH (State of Health), a statistical algorithm for performing statistical analysis on battery data, and a failure prediction algorithm for predicting battery failure. However, the types of analysis algorithms described above are merely examples and are not limited thereto.

[0038] For example, the SOH algorithm can use battery data such as battery voltage, battery current, and battery SOC, and can calculate battery SOH data as an analysis result.

[0039] Each analysis engine may have different types of battery data used for analysis, types of analysis result data, and analysis cycles depending on the analysis algorithm included. Additionally, the data processing system (10) may be configured by selectively combining analysis engines according to the purpose of battery data analysis. For example, different analysis algorithms and analysis engines may be combined depending on whether the battery data analysis is real-time analysis or batch analysis.

[0040] Each analysis engine may have different analysis characteristics and may derive different analysis results. For example, the first analysis engine may derive analysis results related to battery performance, and the second analysis engine may derive analysis results related to battery usage. For instance, the first analysis engine may derive analysis results related to battery performance, such as SOH and RUL, while the second analysis engine may derive analysis results related to battery usage, such as battery charging guides and optimal energy efficiency guides.

[0041] The first analysis engine (110) can analyze battery data and generate one or more analysis results. Battery data can be collected from various configurations, such as the battery's battery management system and the vehicle's OBD device. Battery data may include various types of data, such as the battery's charge state, temperature, current, and voltage.

[0042] For example, the first analysis engine (110) may receive battery data from an external device (e.g., a cloud server) and may generate one or more analysis results by analyzing the provided battery data. The analysis results may include one or more analysis data, and depending on the type of analysis engine and / or analysis algorithm, at least some of the analysis data may differ.

[0043] For example, the first analysis engine (110) can generate one or more analysis results corresponding to an analysis algorithm included in the first analysis engine (110). For example, if the analysis algorithm of the first analysis engine (110) is a RUL algorithm, the first analysis engine (110) can generate an analysis result including RUL data of the battery.

[0044] The first analysis engine (110) can compress one or more analysis results to create a first compressed file. The first compressed file may include one or more analysis results used for compression. For example, analysis data derived from the first analysis engine (110) may have a .CSV extension, and the first analysis engine (110) can compress one or more analysis results to create a first compressed file having a tar.gz extension.

[0045] The first compressed file may be at least one, and may be composed of multiple files depending on the characteristics of the analysis result, the capacity of the analysis result, etc.

[0046] In step S120, the first analysis engine (110) can transmit the first compressed file to the data lake (200). Multiple analysis engines and the data lake (200) are connected so that multiple compressed files can be transmitted from the multiple analysis engines to the data lake (200) through various communication methods.

[0047] In step S130, the first compressed file can be stored in the data lake (200). The data processing system (10) can store each compressed file generated from each of the plurality of analysis engines in the data lake (200), just like the first compressed file of the first analysis engine (110).

[0048] The data lake (200) can store raw data in an unprocessed form before the data is processed. For example, the data lake (200) can store the raw data of the first compressed file (113). The data lake (200) can perform the role of storing the raw data of the compressed files for a long time.

[0049] In step S140, compressed files stored in the data lake (200) can be extracted sequentially. The data processing system (10) can extract the first compressed file (113) stored in the data lake (200) and transmit it to the analysis device (300). For example, the data processing system (10) can sequentially transmit the compressed files stored in the data lake (200) to the analysis device (300).

[0050] According to one embodiment, a first compressed file stored in a data lake (200) can be sequentially transmitted to an analysis device (300) by a message queue (500).

[0051] A message queue (500) is a type of communication method used when exchanging data between configurations, and can mediate data transmission between different configurations by using a queue that can act as a buffer during the data transmission process. The message queue (500) can mediate data transmission between the data lake (200) and the analysis device (300).

[0052] The message queue (500) can sequentially transmit data stored in the data lake (200), namely compressed files, to the analysis device (300). The compressed files stored in the data lake (200) can be transmitted to the message queue (500) and temporarily stored, and the analysis device (300) can sequentially pool and obtain the compressed files stored in the message queue (500). In this way, the compressed files can be sequentially transmitted from the data lake (200) to the analysis device (300) by the message queue (500).

[0053] In step S150, the analysis device (300) can generate multiple data frames.

[0054] First, the analysis device (300) can obtain one or more analysis results included in the first compressed file by decompressing the first compressed file extracted from the data lake (200). The analysis device (300) can obtain one or more analysis results included in the first compressed file by decompressing the first compressed file transmitted from the message queue (500). The analysis device (300) can obtain analysis results included in the compressed file by decompressing the compressed file for the transformation of analysis data included in the analysis results.

[0055] The analysis device (300) can process one or more analysis results to generate multiple data frames. The analysis device (300) can process one or more analysis results to transform the data and reflect it in the database (400). That is, the analysis device (300) can generate a mapping between the analysis result data and the database (400) in order to load the analysis result data into the database (400) that serves as the storage target.

[0056] According to one embodiment, the analysis device (300) can convert data included in one or more analysis results to correspond to a plurality of schema structures. A schema may refer to a concept that defines the data structure of the data. For example, the analysis result data may have a respective data structure, i.e., a schema. The analysis device (300) can convert analysis result data having different schemas to correspond to a plurality of schema structures. For example, the analysis device (300) can convert the analysis result data included in the analysis results to correspond to a plurality of schema structures by extracting or merging some of the data items of the analysis result data.

[0057] Multiple schema structures may be pre-configured. According to one embodiment, each of the multiple schema structures may correspond to a storage structure for each of the multiple tables included in the database (400). Here, the storage structure for each of the multiple tables may refer to the schema structure of the data stored in each table.

[0058] For example, each storage structure may define multiple columns regarding data types. The data types in each column may include various types such as data collection time, data collection vehicle ID, ID of the analysis engine that generated the analysis result, battery voltage, battery current, battery SOC, and battery temperature.

[0059] The analysis device (300) can convert the analysis result data to have a plurality of schema structures corresponding to the storage structure of each of the plurality of tables in order to convert the analysis result data and load it into the database (400). This can be understood as creating a mapping between the analysis result data and the database (400).

[0060] According to one embodiment, the analysis device (300) can generate a plurality of data frames by grouping data corresponding to each of a plurality of schema structures. The analysis device (300) can generate data frames by grouping and arranging data having the same schema structure. For example, a data frame may be a structure in which data having the same schema structure are arranged in parallel.

[0061] In step S160, each of the multiple data frames can be loaded into a corresponding table. The multiple data frames generated by the analysis device (300) can be loaded into a database (400). The database (400) may include multiple tables, and each of the multiple data frames can be loaded into a corresponding table. For example, each data frame can be loaded into a table having a storage structure corresponding to the schema structure of the data frame. That is, a mapping between the schema structure of the data frame and the database (400) table can be generated by the analysis device (300), and each data frame can be loaded into a corresponding table according to the mapping result.

[0062] According to one embodiment, the analysis device (300) may rent a connection object from a connection pool. Here, the connection pool may refer to a type of library in which connection objects that have established a connection with the database (400) are stored. For example, the analysis device (300) may rent a connection object whenever a new data frame is created.

[0063] The analysis device (300) can load each of a plurality of data frames into the database (400) using a connection object borrowed from the connection pool. As described above, the connection object can establish a connection with the database (400) and load the data frames of the analysis device (300) into the database (400). By temporarily borrowing a connection object to connect to the database (400), the analysis device (300) can reduce the database connection load during the data processing process.

[0064] When the analysis device (300) completes loading of multiple data frames into the database (400), it can return the rented connection object to the connection pool.

[0065] In step S170, the database (400) can store multiple data frames. Each data frame can be stored in a table having a corresponding storage structure.

[0066] Referring to FIG. 2, the first analysis engine (110) can analyze battery data to generate one or more analysis results (111). Additionally, the first analysis engine (110) can compress one or more analysis results (111) to generate a first compressed file (113). For example, each of the analysis result data included in the one or more analysis results (111) may be data having a first format (e.g., .CSV), and the first analysis engine (110) can compress it to generate a first compressed file having a second format (e.g., tar.gz).

[0067] The first compressed file (113) generated by the first analysis engine (110) can be stored in the data lake (200). The data lake (200) can store the original data of the first compressed file (113) for a long period. Additionally, the data lake (200) can store compressed files generated from each of a plurality of analysis engines, including the first analysis engine (110).

[0068] Compressed files stored in the data lake (200) can be provided sequentially to the analysis device (300) via a message queue (500). The message queue (500) may have a first-in, first-out structure, and data that is input first may be output first. The message queue (500) may temporarily store input data and sequentially output data input into the message queue (500). For example, a first compressed file stored in the data lake (200) may be input into the message queue (500) and provided sequentially to the analysis device (300) according to the order in which it was input.

[0069] The analysis device (300) can process a first compressed file (113) provided from a message queue (500) to generate a plurality of data frames (311). First, the analysis device (300) can decompress the first compressed file (113) to obtain one or more analysis results (111) included in the first compressed file (113).

[0070] Subsequently, the analysis device (300) can process one or more analysis results (111) to generate multiple data frames (311). For example, the analysis device (300) can convert each of the one or more analysis results (111) to have a predetermined schema structure and combine data having the same schema structure to generate multiple data frames (311).

[0071] The analysis device (300) can load a plurality of data frames (311) into a database (400). In one embodiment, a connection object (313) can be borrowed from a connection pool where a connection object that has established a connection with the database (400) is stored, and each of the plurality of data frames (311) can be loaded into the database (400) using the connection object (313).

[0072] The analysis device (300) can rent a connection object (313) when a plurality of data frames (311) are generated, perform transaction processing on each data frame to load it into the database (400), and return the connection object (313) when loading is completed. In this way, the data processing system (10) can reduce the database (400) connection load by attempting to connect to the database (400) only when loading and storage into the database (400) are required.

[0073] The database (400) may have multiple tables (410, 420, 430), and each of the multiple data frames may be loaded into a table having a corresponding storage structure. For example, the analysis device (300) may generate a mapping between multiple data frames and tables, and load each data frame into a corresponding table according to the mapping.

[0074] For example, a data frame having a first schema structure can be stored in a first table (410), a data frame having a second schema structure can be stored in a second table (420), and a data frame having a third schema structure can be stored in a third table (430).

[0075] FIG. 3 is a diagram showing the overall structure of a data processing system according to one embodiment disclosed in this document. Hereinafter, content that overlaps with the content described in FIG. 1 and FIG. 2 will be omitted.

[0076] Referring to FIG. 3, the data processing system (10) may include a plurality of analysis engines (100). The plurality of analysis engines (100) may have an MSA structure, and each analysis engine (110, 120, 130) may be operated independently of each other.

[0077] Each analysis engine (110, 120, 130) included in the plurality of analysis engines (100) can analyze battery data and generate each analysis result and compressed file. For example, the first analysis engine (110) can generate a first analysis result (111) and a first compressed file (113), the second analysis engine (120) can generate a second analysis result (121) and a second compressed file (123), and the third analysis engine (130) can generate a third analysis result (131) and a third compressed file (133).

[0078] Each compressed file (113, 123, 133) generated by each analysis engine (110, 120, 130) can be stored in a data lake (200) and provided sequentially to an analysis device (300) by a message queue (500).

[0079] The analysis device (300) can generate multiple data frames by processing compressed files provided sequentially from the message queue (500). For example, the analysis device (300) can decompress the second compressed file (123) to obtain a second analysis result (121), and process the second analysis result (121) to generate a second data frame (321).

[0080] In FIG. 3, the process of the analysis device (300) generating data frames from each compressed file (113, 123, 133) is illustrated in a single diagram, but each process may not be identical in time and may be performed sequentially.

[0081] When multiple data frames corresponding to a single compressed file are generated, the analysis device (300) can borrow a connection object from the connection pool and load it into the database (400). For example, when a third data frame (331) corresponding to a third compressed file (133) is generated, the analysis device (300) can borrow a connection object (333) and load it into the database (400). Similarly, when a first data frame (311) is generated, the analysis device (300) can borrow a connection object (313) and load it into the database (400), and when a second data frame (321) is generated, it can borrow a connection object (323) and load it into the database (400).

[0082] FIG. 4 is a flowchart illustrating a data processing method according to one embodiment disclosed in this document.

[0083] Referring to FIG. 4, in step S410, a first analysis engine (110) can analyze battery data to generate a first compressed file. The first analysis engine (110) may be any one of a plurality of analysis engines, and each analysis engine may generate a respective compressed file from the battery data. The first analysis engine (110) may analyze the battery data to generate one or more analysis results and compress one or more analysis results to generate a first compressed file.

[0084] In step S420, the first compressed file can be stored in the data lake (200). The data lake (200) can store each compressed file generated from each of the multiple analysis engines, including the first compressed file. The data lake (200) can store the original data of the compressed file.

[0085] In step S430, the first compressed file stored in the data lake (200) can be extracted and transmitted to the analysis device. For example, the compressed file stored in the data lake (200) can be transmitted sequentially to the analysis device (300) via the message queue (500).

[0086] In step S440, one or more analysis results can be obtained by decompressing the first compressed file by the analysis device (300). The analysis device (300) can obtain one or more analysis results included in the first compressed file by decompressing the file compressed for data processing.

[0087] In step S450, one or more analysis results can be processed by the analysis device (300) to generate multiple data frames. The analysis device (300) can convert one or more analysis results to correspond to multiple schema structures and group data having the same schema structure to generate multiple data frames.

[0088] In step S460, multiple data frames can be stored in the database (400). The analysis device (300) can load the generated multiple data frames into the database (400). For example, the analysis device (300) can load multiple data frames into the database (400) by borrowing connection objects from a connection pool.

[0089] Meanwhile, the present specification and drawings disclose preferred embodiments of the present disclosure. Although specific terms have been used, they are used merely in a general sense to facilitate the explanation of the technical content of the present disclosure and to aid in understanding the embodiments, and are not intended to limit the scope of the present disclosure. It is obvious to those skilled in the art that, in addition to the embodiments disclosed herein, other variations based on the technical concept of the present disclosure are possible.

[0090] A device or terminal according to the embodiments described above may include a processor, memory for storing and executing program data, permanent storage such as a disk drive, a communication port for communicating with an external device, and user object devices such as a touch panel, a key, a button, etc. Methods implemented as software modules or algorithms may be stored on a computer-readable recording medium as computer-readable code or program instructions executable on the processor. Here, computer-readable recording media include magnetic storage media (e.g., ROM (read-only memory), RAM (random-access memory), floppy disks, hard disks, etc.) and optical reading media (e.g., CD-ROM, DVD (Digital Versatile Disc)). Computer-readable recording media may be distributed across networked computer systems, allowing computer-readable code to be stored and executed in a distributed manner. The medium may be readable by a computer, stored in memory, and executed by a processor.

[0091] The present embodiment may be represented by functional block configurations and various processing steps. These functional blocks may be implemented by various numbers of hardware and / or software configurations that execute specific functions. For example, the embodiment may employ integrated circuit configurations such as memory, processing, logic, look-up tables, etc., capable of executing various functions by the control of one or more microprocessors or other control devices. Similar to how components may be implemented as software programming or software elements, the present embodiment may be implemented in programming or scripting languages ​​such as C, C++, Java, assembler, Python, etc., including various algorithms implemented as combinations of data structures, processes, routines, or other programming configurations. Functional aspects may be implemented as algorithms executed on one or more processors. Additionally, the present embodiment may employ prior art for electronic configuration, signal processing, and / or data processing. Terms such as “mechanism,” “element,” “means,” and “configuration” may be used broadly and are not limited to mechanical and physical configurations. The above terms may include the meaning of a series of software processes (routines) in conjunction with processors, etc.

Claims

1. In a data processing method performed by a data processing system, A step of analyzing battery data by a first analysis engine among a plurality of analysis engines that operate independently of each other to generate one or more analysis results, and compressing the one or more analysis results to generate a first compressed file; A step of storing the above-mentioned first compressed file in a data lake; A step of extracting the first compressed file stored in the data lake and transmitting it to an analysis device; A step of obtaining one or more analysis results by decompressing the first compressed file by the analysis device; A step of generating a plurality of data frames by processing one or more of the above analysis results; and A data processing method comprising the step of storing the above plurality of data frames in a database.

2. In Paragraph 1, The above plurality of analysis engines are characterized by being configured in an MSA structure, Data processing method.

3. In Paragraph 1, The above plurality of analysis engines include different analysis algorithms, and The step of generating the first compressed file above is, A step comprising generating one or more analysis results corresponding to the analysis algorithm of the first analysis engine, Data processing method.

4. In Paragraph 1, The step of transmitting to the analysis device above is, Characterized by the first compressed file stored in the data lake being sequentially transmitted to the analysis device by a message queue. Data processing method.

5. In Paragraph 1, The step of generating the above plurality of data frames is, A step of converting data included in one or more of the above analysis results to correspond to a plurality of schema structures; and A method comprising the step of generating the plurality of data frames by grouping data corresponding to each of the plurality of schema structures. Data processing method.

6. In Paragraph 5, Each of the above plurality of schema structures is characterized by corresponding to the storage structure of each of the plurality of tables included in the database. Data processing method.

7. In Paragraph 6, The step of storing in the above database is, Characterized by loading each of the above plurality of data frames into a table having a corresponding storage structure, Data processing method.

8. In Paragraph 1, The step of storing in the above database is, A step of borrowing a connection object from a connection pool where a connection object with which a connection to the database has been established is stored; and A method comprising the step of loading each of the plurality of data frames into the database using the above-mentioned connection object. Data processing method.

9. In Paragraph 8, The step of storing in the above database is, A method further comprising the step of returning the above connection object to the above connection pool. Data processing method.

10. A computer-readable, non-transient recording medium having a program for executing the method of paragraph 1 on a computer.

Images