Data storage method, apparatus and electronic device
By matching pre-configured partitioned storage rules, call detail record (CDR) data is stored in different distributed storage directories according to data dimensions, which solves the problems of low operation and maintenance and low query efficiency when the data volume is large, and achieves efficient data storage and query.
Patent Information
- Application Number
- CN202311130101.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-04
- Publication Date
- 2026-08-25
- Estimated Expiration
- 2043-09-04
AI Technical Summary
In existing technologies, when the amount of data is too large, multiple data tables need to be designed for data partitioning and storage, which leads to reduced data operation and maintenance efficiency and low data query efficiency.
By acquiring call detail record (CDR) data and preset storage rule information, and based on the matching of CDR fields and filter fields, the target storage rule information and storage directory are determined. The CDR data is stored in the corresponding distributed storage directory, and the data storage and query process is optimized by using hash values and directory suffix fields during querying and exporting.
It reduced data operation and maintenance costs, improved data query efficiency, and enhanced data export efficiency by merging exported files.
Smart Images

Figure CN117112567B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a data storage method, apparatus and electronic device. Background Technology
[0002] A large amount of call detail records (CDRs) are generated during the transmission and application of network data using cellular networks. To handle this large amount of CDR data, it is necessary to perform partitioned storage and multi-dimensional filtering and export.
[0003] In existing technologies, traditional relational databases are used for database sharding and table partitioning, storing data in multiple tables across multiple databases. During export, data is then filtered out from each table in each database for export processing.
[0004] However, in the above methods, when the data volume is too large, multiple data tables need to be designed for data partitioning and storage, which leads to a decrease in data operation and maintenance efficiency; and when filtering data, each data table needs to be queried, which takes a long time and further reduces data query efficiency. Summary of the Invention
[0005] This application provides a data storage method, apparatus, and electronic device to solve the problem that when the data volume is too large, it is necessary to design multiple data tables for data partitioning and storage, which leads to reduced data operation and maintenance efficiency and reduced data query efficiency.
[0006] In a first aspect, this application provides a data storage method, the method comprising:
[0007] Acquire call detail record (CDR) data and at least one preset storage rule information; wherein, the CDR data is CDR data generated by the user during network use, the CDR data includes at least one CDR field, the CDR field representing the information of the service to which the CDR data belongs; the preset storage rule information represents the storage processing method of the CDR data, the preset storage rule information includes a filter field, the filter field representing the data dimension to which the CDR data belongs when it is stored;
[0008] Based on at least one call detail record (CDR) field in the CDR data and a filter field in the at least one preset storage rule information, target storage rule information is determined from the at least one preset storage rule information; wherein, the target storage rule information represents a storage processing method that is effective for processing the CDR data;
[0009] Based on the call detail record (CDR) data and the target storage rule information, the storage directory for the CDR data is determined; wherein, the storage directory is a storage directory used to store the CDR data.
[0010] The call detail records (CDRs) are stored in the CDR storage directory.
[0011] In one example, the call detail record (CDR) field has a first field value, which represents the field attribute of the CDR field; the filter field includes at least one enumeration value, which represents the field type of the filter field;
[0012] Based on at least one call detail record (CDR) field in the CDR data and a filter field in the at least one preset storage rule information, the target storage rule information is determined from the at least one preset storage rule information, including:
[0013] The call detail record (CDR) field that is identical to the filter field in the preset storage rule information among the at least one CDR field is identified as the target field of the CDR data; wherein, the target field represents the data dimension to which the CDR data belongs when it is stored;
[0014] If it is determined that there is an enumerated value in the filter field that is the same as the first field value of the target field, then the preset storage rule information to which the filter field belongs in the at least one preset storage rule information is determined to be the target storage rule information.
[0015] In one example, the call detail record (CDR) field has a first field value, which represents the field attribute of the CDR field; the storage directory of the CDR data is determined based on the CDR data and the target storage rule information, including:
[0016] Extract the initial storage directory from the target storage rule information; wherein the initial storage directory includes at least one storage field, the storage field representing the data dimension to which the call detail record data stored in the initial storage directory belongs; the storage field has a second field value, the second field value representing the field attribute of the storage field;
[0017] The second field value of the stored field in the initial storage directory is replaced with the first field value of the call detail record (CDR) field in the call detail record (CDR) data that is consistent with the stored field in the initial storage directory, to obtain the updated storage directory;
[0018] Based on the updated storage directory, determine the key value of the call detail record (CDR) data; wherein, the key value represents the data identifier of the CDR data;
[0019] A preset database is retrieved, and the storage directory of the call detail record (CDR) data is determined based on the updated storage directory, the key value of the CDR data, and the preset database. The preset database includes at least one hash value and a key value corresponding to the hash value, wherein the hash value represents the storage attribute of the CDR data stored in the preset database.
[0020] In one example, determining the storage directory for the call detail record (CDR) data based on the updated storage directory, the key values of the CDR data, and the preset database includes:
[0021] If it is determined that there is a key value in the preset database that is the same as the key value of the call detail record (CDR) data, then add 1 to the hash value corresponding to the key value in the preset database that is the same as the key value of the CDR data to obtain the hash value of the CDR data.
[0022] Based on the hash value of the call detail record (CDR) data and the preset threshold in the target storage rule information, a first directory suffix field of the CDR data is determined; wherein, the first directory suffix field represents the storage sequence number of the CDR data in the storage directory; the preset threshold represents the maximum amount of CDR data that the target storage rule information can process;
[0023] The storage directory of the call detail record (CDR) data is obtained based on the updated storage directory and the first directory suffix field of the CDR data.
[0024] In one example, the first directory suffix field k = m / n; where m is the hash value of the call detail record data and n is the preset threshold in the target storage rule information.
[0025] In one example, determining the storage directory for the call detail record (CDR) data based on the updated storage directory, the key values of the CDR data, and the preset database further includes:
[0026] If it is determined that there is no key value in the preset database that is the same as the key value of the call detail record (CDR) data, then the key value of the CDR data is added to the preset database, and the hash value of the CDR data is determined to be 1.
[0027] The second directory suffix field of the call detail record (CDR) data is set to 0; wherein, the second directory suffix field represents the storage sequence number of the CDR data in the storage directory;
[0028] The storage directory of the call detail record (CDR) data is obtained based on the updated storage directory and the second directory suffix field of the CDR data.
[0029] In one example, the method further includes:
[0030] Read configuration information; wherein, the configuration information is used to export target data, the target data being the call detail records (CDRs) to be exported; the configuration information includes at least one filtering field for the target data, and the storage directory for the target data;
[0031] The storage directory in the configuration information is determined to be the target storage directory; wherein, the target storage directory is the storage directory where the target data is located;
[0032] If it is determined that there are no subdirectories in the target storage directory, then according to the filter fields in the configuration information, the target data stored in the target storage directory is exported to the Secure File Information Transfer Subsystem (SFTP) server.
[0033] In one example, after determining the storage directory in the configuration information, the process includes:
[0034] If it is determined that at least one subdirectory exists in the target storage directory, the target data stored in the subdirectory is exported to obtain an exported file; wherein, the exported file includes the target data; the exported file corresponds one-to-one with the subdirectory;
[0035] The exported files are merged to obtain a merged exported file, which is then uploaded to the SFTP server.
[0036] Secondly, this application provides a data storage device, the device comprising:
[0037] An acquisition unit is used to acquire call detail record (CDR) data and at least one preset storage rule information; wherein, the CDR data is CDR data generated by a user during network usage, and the CDR data includes at least one CDR field, the CDR field representing the information of the service to which the CDR data belongs; the preset storage rule information represents the storage processing method of the CDR data, and the preset storage rule information includes a filter field, the filter field representing the data dimension to which the CDR data belongs when it is stored;
[0038] The first determining unit is configured to determine target storage rule information from the at least one preset storage rule information based on at least one call detail record field in the call detail record data and a filter field in the at least one preset storage rule information; wherein, the target storage rule information represents a storage processing method that is effective for processing the call detail record data;
[0039] The second determining unit is configured to determine the storage directory of the call detail record (CDR) data based on the CDR data and the target storage rule information; wherein the storage directory is a storage directory used to store the CDR data.
[0040] A storage unit is used to store the call detail record (CDR) data in the CDR data storage directory.
[0041] In one example, the call detail record (CDR) field has a first field value, which represents the field attribute of the CDR field; the filter field includes at least one enumeration value, which represents the field type of the filter field;
[0042] The first determining unit includes:
[0043] The first determining module is used to determine the call detail record (CDR) field that is the same as the filter field of the preset storage rule information among the at least one CDR field, and that is the target field of the CDR data; wherein, the target field represents the data dimension to which the CDR data belongs when it is stored;
[0044] The second determining module is used to determine, if it is determined that there is an enumeration value in the filtering field that is the same as the first field value of the target field, the preset storage rule information to which the filtering field belongs in the at least one preset storage rule information is the target storage rule information.
[0045] In one example, the call detail record (CDR) field has a first field value, which represents the field attribute of the CDR field; the second determining unit includes:
[0046] An extraction module is used to extract the initial storage directory from the target storage rule information; wherein the initial storage directory includes at least one storage field, the storage field representing the data dimension to which the call detail record data stored in the initial storage directory belongs; the storage field has a second field value, the second field value representing the field attribute of the storage field;
[0047] The replacement module is used to replace the second field value of the stored field in the initial storage directory with the first field value of the call detail record (CDR) field in the call detail record (CDR) data that is consistent with the stored field in the initial storage directory, so as to obtain the updated storage directory.
[0048] The third determining module is used to determine the key value of the call detail record (CDR) data based on the updated storage directory; wherein the key value represents the data identifier of the CDR data;
[0049] The fourth determining module is used to retrieve a preset database and determine the storage directory of the call detail record (CDR) data based on the updated storage directory, the key value of the CDR data, and the preset database. The preset database includes at least one hash value and a key value corresponding to the hash value, wherein the hash value represents the storage attribute of the CDR data stored in the preset database.
[0050] In one example, the fourth determining module is specifically used for:
[0051] If it is determined that there is a key value in the preset database that is the same as the key value of the call detail record (CDR) data, then add 1 to the hash value corresponding to the key value in the preset database that is the same as the key value of the CDR data to obtain the hash value of the CDR data.
[0052] Based on the hash value of the call detail record (CDR) data and the preset threshold in the target storage rule information, a first directory suffix field of the CDR data is determined; wherein, the first directory suffix field represents the storage sequence number of the CDR data in the storage directory; the preset threshold represents the maximum amount of CDR data that the target storage rule information can process;
[0053] The storage directory of the call detail record (CDR) data is obtained based on the updated storage directory and the first directory suffix field of the CDR data.
[0054] In one example, the first directory suffix field k = m / n; where m is the hash value of the call detail record data and n is the preset threshold in the target storage rule information.
[0055] In one example, the fourth determining module is further specifically used for:
[0056] If it is determined that there is no key value in the preset database that is the same as the key value of the call detail record (CDR) data, then the key value of the CDR data is added to the preset database, and the hash value of the CDR data is determined to be 1.
[0057] The second directory suffix field of the call detail record (CDR) data is set to 0; wherein, the second directory suffix field represents the storage sequence number of the CDR data in the storage directory;
[0058] The storage directory of the call detail record (CDR) data is obtained based on the updated storage directory and the second directory suffix field of the CDR data.
[0059] In one example, the device further includes:
[0060] A reading unit is used to read configuration information; wherein, the configuration information is used to export target data, the target data being call detail records (CDRs) to be exported; the configuration information includes at least one filtering field for the target data and the storage directory for the target data;
[0061] The third determining unit is used to determine the storage directory in the configuration information as the target storage directory; wherein, the target storage directory is the storage directory where the target data is located;
[0062] The first export unit is used to export the target data stored in the target storage directory to the Secure File Information Transfer Subsystem (SFTP) server, based on the filter field in the configuration information, if it is determined that no subdirectory exists in the target storage directory.
[0063] In one example, after the third determining unit determines the storage directory in the configuration information, the process includes:
[0064] The second export unit is used to export the target data stored in the subdirectory if it is determined that at least one subdirectory exists in the target storage directory, and obtain an export file; wherein the export file includes the target data; and the export file corresponds one-to-one with the subdirectory.
[0065] The upload unit is used to merge the exported files to obtain a merged exported file, and upload the merged exported file to the SFTP server.
[0066] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;
[0067] The memory stores computer-executed instructions;
[0068] The processor executes computer execution instructions stored in the memory to implement the method described in the first aspect.
[0069] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in the first aspect.
[0070] Fifthly, this application provides a computer program product comprising: a computer program stored in a readable storage medium, wherein at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the method described in the first aspect.
[0071] The data storage method, apparatus, and electronic equipment provided in this application, by matching pre-configured partitioned storage rules, store call detail record (CDR) data into different distributed storage directories according to different data dimensions; thereby, it can reduce data operation and maintenance costs and improve data query efficiency. Attached Figure Description
[0072] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0073] Figure 1 A flowchart illustrating a data storage method provided in an embodiment of this application;
[0074] Figure 2 A flowchart illustrating another data storage method provided in an embodiment of this application;
[0075] Figure 3This is a schematic diagram of the structure of a data storage device provided in an embodiment of this application;
[0076] Figure 4 A schematic diagram of another data storage device provided in an embodiment of this application;
[0077] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;
[0078] Figure 6 This is a block diagram illustrating an electronic device according to an exemplary embodiment.
[0079] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0080] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0081] A large amount of call detail records (CDRs) are generated during the transmission and application of network data using cellular networks. To handle this large amount of CDR data, it is necessary to perform partitioned storage and multi-dimensional filtering and export.
[0082] In one example, a traditional relational database is used for database sharding and table partitioning. Data is stored in multiple tables across multiple databases. During export, data is filtered out from each table in each database and then exported.
[0083] However, in the above methods, when the data volume is too large, multiple data tables need to be designed for data partitioning and storage, which leads to a decrease in data operation and maintenance efficiency; and when filtering data, each data table needs to be queried, which takes a long time and further reduces data query efficiency.
[0084] The data storage method, apparatus, and electronic device provided in this application are intended to solve the above-mentioned technical problems of the prior art.
[0085] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and storage of related data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.
[0086] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.
[0087] Figure 1 This is a flowchart illustrating a data storage method provided in an embodiment of this application, such as... Figure 1 As shown, the method includes:
[0088] S101. Obtain call detail record (CDR) data and at least one preset storage rule information; wherein, the CDR data is the CDR data generated by the user during network use, and the CDR data includes at least one CDR field, which represents the information of the service to which the CDR data belongs; the preset storage rule information represents the storage processing method of the CDR data, and the preset storage rule information includes a filter field, which represents the data dimension to which the CDR data belongs when it is stored.
[0089] For example, the execution subject of this embodiment may be an electronic device. First, during network usage, users generate call detail records (CDRs). For example, CDRs generated by IoT cellular networks contain many fields such as: serial number, user identifier, calling number, called number, session start time, session end time, session duration, internet traffic, network standard, and other information. The business systems deployed on electronic devices obtain this CDR data through the network, including information about the business to which one or more CDRs belong, i.e., CDR fields. For example, the session start time (BEGIN_DATE), in the format yyyy-MM-dd HH:mm:ss, identifies the day to which the CDR belongs; the enterprise account identifier (ACCT_ID), a unique identifier for B-end enterprises within the business system, identifies which B-end enterprise the CDR belongs to; the province code (PROVINCE_CODE) identifies the province; and the network standard (NETWORK_TYPE) identifies the network standard used, such as 4G (4th Generation Mobile Communication Technology) or 5G (5th Generation Mobile Communication Technology). (Mobile Communication Technology). Meanwhile, the business systems deployed on electronic devices obtain one or more pre-set storage processing methods for call detail record (CDR) data from a configuration center, database, or remote dictionary service Redis (Remote Dictionary Server), i.e., preset storage rule information. Each preset storage rule information includes filter fields to characterize the data dimension to which the CDR data belongs when it is stored, for subsequent processing. Preset storage rule information can be dynamically added or deleted. Filter fields can be a single field or a combination of multiple fields. When there are multiple fields, the fields are related by & (&) for AND and by || for OR. Only one of the logic, & or ||, can appear in multiple filter fields within the same rule.
[0090] S102. Based on at least one call detail record (CDR) field in the CDR data and a filter field in at least one preset storage rule information, determine target storage rule information from at least one preset storage rule information; wherein, the target storage rule information represents the storage processing method that is effective for CDR data processing.
[0091] For example, based on multiple call detail record (CDR) fields in the acquired CDR data and filter fields in multiple preset storage rule information, the multiple CDR fields in the CDR data are analyzed and matched sequentially with each filter field in the preset storage rule information. From the multiple preset storage rule information, the storage processing method that is effective for processing the acquired CDR data is determined, that is, the target storage rule information that matches the acquired CDR data.
[0092] S103. Determine the storage directory for the call detail record (CDR) data based on the CDR data and the target storage rule information; wherein, the storage directory is the storage directory used to store the CDR data.
[0093] For example, based on the target storage rule information that matches the acquired call detail record (CDR) data, the target storage rule information is analyzed to obtain the corresponding storage directory for the CDR data; wherein, the storage directory is the storage directory used to store the CDR data, and if multiple target storage rule information can be matched, the storage directory corresponding to each target storage rule information is determined.
[0094] S104. Store the call detail record (CDR) data in the CDR data storage directory.
[0095] For example, based on the target storage rule information matched by the call detail record (CDR) data and the storage directory matched by the CDR data, the CDR data is stored in the storage directory corresponding to the target storage rule information. If multiple target storage rule information can be matched, the CDR data will also be saved in multiple storage directories.
[0096] This embodiment provides a data storage method that, by matching pre-configured partitioned storage rules, stores call detail records (CDRs) data into different distributed storage directories according to different data dimensions; thereby, it can reduce data operation and maintenance costs and improve data query efficiency.
[0097] Figure 2 A flowchart illustrating another data storage method provided in this application embodiment is shown below. Figure 2 As shown, the method includes:
[0098] S201. Obtain call detail record (CDR) data and at least one preset storage rule information; wherein, the CDR data is CDR data generated by the user during network use, and the CDR data includes at least one CDR field, which represents the information of the service to which the CDR data belongs; the preset storage rule information represents the storage processing method of the CDR data, and the preset storage rule information includes a filter field, which represents the data dimension to which the CDR data belongs when it is stored.
[0099] For example, this step can be referred to step 101, and will not be repeated here.
[0100] S202. Determine at least one call detail record (CDR) field that is identical to the filter field of the preset storage rule information, and designate it as the target field of the CDR data; wherein, the target field represents the data dimension to which the CDR data belongs when it is stored.
[0101] In one example, the call detail record (CDR) field has a first field value, which represents the field attribute of the CDR field; the filter field includes at least one enumeration value, which represents the field type of the filter field.
[0102] For example, the business system deployed on the electronic device, after acquiring user call detail record (CDR) data and multiple preset storage rule information via the network, parses each CDR field in the CDR data to obtain the field value of each CDR field, i.e., the first field value, which represents the field attribute of each CDR field, such as the data structure type of each CDR field. Simultaneously, it parses the filter fields of each preset storage rule information to obtain one or more enumerated values for the filter fields of each preset storage rule information, each enumerated value representing the field type of the filter field. For each preset storage rule information, the filter fields of each preset storage rule information are compared with the CDR fields in the acquired CDR fields. Then, the CDR field that is identical to the filter field of each preset storage rule information is determined from the CDR data; this is the target field of the CDR data under each preset storage rule information, representing the data dimension to which the CDR data belongs when it is stored.
[0103] S203. If it is determined that there is an enumeration value in the filter field that is the same as the first field value of the target field, then the preset storage rule information to which the filter field belongs in at least one preset storage rule information is determined as the target storage rule information.
[0104] For example, the first field value of the target field is compared with the enumeration values included in the filter fields of each preset storage rule information to determine whether the first field value of the target field is consistent with the enumeration values included in the filter fields of each preset storage rule information. For each preset storage rule information, if it is determined that there is an enumeration value in the filter fields of the preset storage rule information that is the same as the first field value of the target field, the match is successful, and the preset storage rule information is determined to be the target storage rule information matched by the call detail record (CDR) data; if it does not exist, the match fails, the CDR data is not saved in the database, and the matching of the next preset storage rule information continues. It is worth noting that if the filter fields configured in the preset storage rule information are a combination of multiple fields, such as: ACCT_ID&NETWORK_TYPE&APN, then the first field value of each CDR field needs to be retrieved from the CDR data, and then judged with the enumeration value in the preset storage rule information according to the logical relationship of & and || to determine whether the preset storage rule information can be matched.
[0105] S204. Extract the initial storage directory from the target storage rule information; wherein, the initial storage directory includes at least one storage field, the storage field representing the data dimension to which the call detail record data stored in the initial storage directory belongs; the storage field has a second field value, the second field value representing the field attribute of the storage field.
[0106] In one example, the call detail record (CDR) field has a first field value, which represents the field attribute of the CDR field.
[0107] For example, for the target storage rule information matched by the call detail record (CDR) data, an initial storage directory is extracted from the target storage rule information. The initial storage directory includes at least one storage field to characterize the data dimension to which the CDR data stored in the initial storage directory belongs. The storage field has a second field value to characterize the field attribute of the storage field. The CDR field has a first field value to characterize the field attribute of the CDR field.
[0108] S205. Replace the second field value of the storage field in the initial storage directory with the first field value of the call detail record (CDR) field in the call detail record (CDR) data that is consistent with the storage field in the initial storage directory, and obtain the updated storage directory.
[0109] For example, based on the initial storage directory extracted from the target storage rule information, the storage fields in the initial storage directory are compared and analyzed with the call detail record (CDR) fields in the CDR data. The CDR fields in the CDR data that match the storage fields in the initial storage directory are then identified. Next, the second field value of the storage field in the initial storage directory is replaced with the first field value of the CDR field in the CDR data that matches the storage field in the initial storage directory. This yields the updated storage directory corresponding to the target storage rule information.
[0110] For example, the initial storage directory is basedir / {BEGIN_DATE} / {ACCT_ID}. The two call detail record (CDR) fields corresponding to BEGIN_DATE and ACCT_ID are determined from the CDR data. The second field value corresponding to BEGIN_DATE and ACCT_ID in the storage directory is replaced with the first field value of the two CDR fields BEGIN_DATE and ACCT_ID in the CDR data. The updated storage directory obtained after the replacement is as follows: basedir / 20230501 / A01.
[0111] S206. Determine the key value of the call detail record (CDR) data based on the updated storage directory; wherein the key value represents the data identifier of the CDR data.
[0112] For example, based on the updated storage directory, the contents of the updated storage directory are determined as the key value of the call detail record (CDR) data to represent the data identifier of the CDR data. For example, if the updated storage directory is basedir / 20230501 / A01, then 20230501 / A01 is the key value of the CDR data.
[0113] S207. Retrieve the preset database and determine the storage directory of the call detail record (CDR) data based on the updated storage directory, the key value of the CDR data, and the preset database. The preset database includes at least one hash value and a key value corresponding to the hash value. The hash value represents the storage attribute of the CDR data stored in the preset database.
[0114] For example, the business system of an electronic device retrieves a preset database based on a remote service, such as the remote dictionary service Redis (Remote Dictionary Server). The preset database includes one or more hash values and a key value corresponding to each hash value. Each hash value represents the storage attribute of the call detail record (CDR) data stored in the preset database. Based on the content of the updated storage directory and the key value of the CDR data, the hash value of the CDR data is determined in the preset database. The updated storage directory, the key value of the CDR data, and the hash value of the CDR data are then concatenated, for example, by directly combining these three elements to determine the storage directory of the CDR data.
[0115] In one example, step S207 includes the following steps:
[0116] The first step of step S207 is: if it is determined that there is a key value in the preset database that is the same as the key value of the call detail record (CDR) data, then add 1 to the hash value corresponding to the key value in the preset database that is the same as the key value of the CDR data to obtain the hash value of the CDR data.
[0117] The second step of step S207 is to determine the first directory suffix field of the call detail record (CDR) data based on the hash value of the CDR data and the preset threshold in the target storage rule information; wherein, the first directory suffix field represents the storage sequence number of the CDR data in the storage directory; the preset threshold represents the maximum amount of CDR data that the target storage rule information can process.
[0118] The third step of step S207 is to obtain the storage directory of the call detail record (CDR) data based on the updated storage directory and the first directory suffix field of the CDR data.
[0119] In one example, the first directory suffix field k = m / n; where m is the hash value of the call detail record data and n is the preset threshold in the target storage rule information.
[0120] For example, after retrieving the preset database, the system first queries the preset database to see if a key with the same key value as the call detail record (CDR) data exists. If a key with the same key value as the CDR data exists in the preset database, the system extracts the hash value corresponding to the key with the same key value as the CDR data from the preset database, increments the hash value by 1 to obtain the hash value of the CDR data, and saves the hash value of the CDR data to the preset database. Considering the data volume, the target storage rule information pre-sets a preset threshold. This preset threshold is the maximum amount of CDR data that the target storage rule information can process. When the number of data entries in the storage directory, such as the storage directory of the Hadoop Distributed File System, reaches the controlled preset threshold, a new subdirectory will be automatically opened for storage, as shown in Table 1, which includes multiple filter fields, enumeration values, storage directories, and preset thresholds in the preset storage rule information.
[0121] Table 1 Example of Partition Storage Rule Definition
[0122]
[0123] The first directory suffix field of the call detail record (CDR) data is obtained by analyzing the hash value of the CDR data and the preset threshold in the target storage rule information. This suffix field represents the storage sequence number of the CDR data in the storage directory. It can be calculated using the formula k = m / n, where m is the hash value of the CDR data and n is the preset threshold in the target storage rule information. The updated storage directory and the first directory suffix field of the CDR data are then processed to obtain the CDR data's storage directory. For example, concatenating the updated storage directory basedir / 20230501 / A01 and the first directory suffix field 3 of the CDR data results in the storage directory basedir / 20230501 / A01 / 3.
[0124] In one example, step S207 further includes the following steps:
[0125] In the fourth step of step S207, if it is determined that there is no key value in the preset database that is the same as the key value of the call detail record (CDR) data, then the key value of the CDR data is added to the preset database, and the hash value of the CDR data is determined to be 1.
[0126] The fifth step of step S207 is to determine that the second directory suffix field of the call detail record (CDR) data is 0; wherein, the second directory suffix field represents the storage sequence number of the CDR data in the storage directory.
[0127] The sixth step of step S207 is to obtain the storage directory of the call detail record (CDR) data based on the updated storage directory and the second directory suffix field of the CDR data.
[0128] For example, after retrieving the preset database, the system first checks if a key with the same key value as the call detail record (CDR) data exists in the preset database. If no key with the same key value exists, it indicates that the CDR data is the first CDR data in the storage directory. The key value of the CDR data is then added to the preset database, set as the key value of the CDR data in the preset database, and its hash value is set to 1. Then, the second directory suffix field of the CDR data is automatically generated as 0 to represent the storage sequence number of the CDR data in the storage directory, i.e., the first stored CDR data. The updated storage directory and the second directory suffix field of the CDR data are processed to obtain the storage directory of the CDR data. For example, the updated storage directory basedir / 20230501 / A01 and the first directory suffix field 0 of the CDR data are concatenated to obtain the storage directory basedir / 20230501 / A01 / 0.
[0129] S208. Store the call detail record (CDR) data in the CDR data storage directory.
[0130] For example, this step can be referred to as step 104, and will not be repeated here.
[0131] S209. Read configuration information; wherein, the configuration information is used to export target data, the target data is the call detail record data to be exported; the configuration information includes at least one filter field for the target data, and the storage directory of the target data.
[0132] For example, after an electronic device completes the storage of call detail records (CDRs), it needs to perform a data export operation, i.e., export the target data, which is the CDR data to be exported. For instance, the operator business system deployed on the electronic device needs to export and transmit the previous day's CDR data by dimension to various sub-business units after the CDR data is entered into the database each day. This includes transmitting CDR data from each province to each provincial region, transmitting CDR data from some key B-end enterprise customers to each B-end enterprise customer, and transmitting CDR data for 5G network standards to the 5G business department. When the electronic device enters the CDR data export process, it will automatically read the configuration information used to export the target data from the configuration center, database, or Redis cache, as shown in Table 2.
[0133] Table 2 Example of Configuration Information
[0134]
[0135] Each configuration information includes at least one filtering field for the target data and the storage directory for the target data. The filtering field in each configuration information also includes enumerated values.
[0136] S210. Determine the storage directory in the configuration information as the target storage directory; where the target storage directory is the storage directory where the target data is located.
[0137] For example, the storage directory in the configuration information is extracted and determined as the target storage directory, i.e., the storage directory where the target data is located. Because there are multiple export dimensions, i.e., filter fields, and each filter field is configured with multiple enumeration values, multiple storage directories will be combined, such as basedir / 20230501 / A01, basedir / 20230501 / A02, basedir / 20230501 / P01, basedir / 20230501 / P02, basedir / 20230501 / 5G, basedir / 20230501 / A01&5G&101, etc.
[0138] S211. If it is determined that there are no subdirectories in the target storage directory, then according to the filter field in the configuration information, the target data stored in the target storage directory is exported to the Secure File Information Transfer Subsystem (SFTP) server.
[0139] For example, each target storage directory is iterated through, and it is determined whether each target storage directory has subdirectories. For each target storage directory, if it is determined that there are no subdirectories, the target data stored in that target storage directory is exported and uploaded to the Secure File Information Transfer Subsystem (SFTP) server according to the data dimension represented by the filter field in the configuration information. It is worth noting that when exporting files and uploading them to the SFTP server, permissions and storage addresses are isolated according to each exported data dimension, i.e., each enumerated value of the filter field, to prevent data cross-leakage.
[0140] S212. If it is determined that there is at least one subdirectory in the target storage directory, the target data stored in the subdirectory is exported to obtain an exported file; wherein the exported file includes the target data; the exported file corresponds one-to-one with the subdirectory.
[0141] For example, each target storage directory is traversed in a loop to determine whether there are subdirectories in each target storage directory. For each target storage directory, if it is determined that there is at least one subdirectory, the target data stored in each subdirectory is exported to obtain an export file containing the target data that corresponds to each subdirectory.
[0142] S213. Merge the exported files to obtain a merged exported file, and upload the merged exported file to the SFTP server.
[0143] For example, all exported files are merged to obtain a merged exported file. Each merged exported file includes the target data stored in each target storage directory. Then, the merged exported file is uploaded to an SFTP server.
[0144] In this embodiment, based on the above embodiments, on the one hand, by matching the pre-configured partition storage rules, call detail records (CDRs) are stored in different distributed storage directories according to different data dimensions; thereby, data operation and maintenance costs can be reduced and data query efficiency can be improved; on the other hand, when exporting CDRs according to different dimensions, the corresponding storage directory can be found directly to obtain the target data, thereby improving the efficiency of data query and data export.
[0145] Figure 3 This is a schematic diagram of the structure of a data storage device provided in an embodiment of this application, such as... Figure 3 As shown, the device 300 includes:
[0146] The acquisition unit 301 is used to acquire call detail record (CDR) data and at least one preset storage rule information. The CDR data is generated by the user during network usage and includes at least one CDR field, which represents the information of the service to which the CDR data belongs. The preset storage rule information represents the storage processing method of the CDR data and includes a filter field, which represents the data dimension to which the CDR data belongs when it is stored.
[0147] The first determining unit 302 is used to determine target storage rule information from at least one preset storage rule information based on at least one call detail record field in the call detail record data and a filter field in at least one preset storage rule information; wherein, the target storage rule information represents the storage processing method that is effective for call detail record data processing.
[0148] The second determining unit 303 is used to determine the storage directory of the call detail record (CDR) data based on the CDR data and the target storage rule information; wherein, the storage directory is the storage directory used to store the CDR data.
[0149] Storage unit 304 is used to store call detail record (CDR) data in the CDR data storage directory.
[0150] The apparatus in this embodiment can execute the technical solutions in the above method. Its specific implementation process and technical principles are the same, and will not be repeated here.
[0151] Figure 4 This is a schematic diagram of another data storage device provided in an embodiment of this application, as shown below. Figure 4 As shown, the device 400 includes:
[0152] The acquisition unit 401 is used to acquire call detail record (CDR) data and at least one preset storage rule information. The CDR data is generated by the user during network usage and includes at least one CDR field, which represents the information of the service to which the CDR data belongs. The preset storage rule information represents the storage processing method of the CDR data and includes a filter field, which represents the data dimension to which the CDR data belongs when it is stored.
[0153] The first determining unit 402 is used to determine target storage rule information from at least one preset storage rule information based on at least one call detail record field in the call detail record data and a filter field in at least one preset storage rule information; wherein, the target storage rule information represents the storage processing method that is effective for call detail record data processing.
[0154] The second determining unit 403 is used to determine the storage directory of the call detail record (CDR) data based on the CDR data and the target storage rule information; wherein, the storage directory is the storage directory used to store the CDR data.
[0155] Storage unit 404 is used to store call detail record (CDR) data in the CDR data storage directory.
[0156] In one example, the call detail record (CDR) field has a first field value, which represents the field attribute of the CDR field; the filter field includes at least one enumeration value, which represents the field type of the filter field.
[0157] The first determining unit 402 includes:
[0158] The first determining module 4021 is used to determine at least one call detail record (CDR) field that is the same as the filter field of the preset storage rule information, and to determine the target field of the CDR data; wherein the target field represents the data dimension to which the CDR data belongs when it is stored.
[0159] The second determining module 4022 is used to determine, if it is determined that there is an enumerated value in the filter field that is the same as the first field value of the target field, the preset storage rule information to which the filter field belongs in at least one preset storage rule information is the target storage rule information.
[0160] In one example, the call detail record (CDR) field has a first field value, which represents the field attribute of the CDR field; the second determining unit 403 includes:
[0161] Extraction module 4031 is used to extract the initial storage directory from the target storage rule information; wherein, the initial storage directory includes at least one storage field, the storage field representing the data dimension to which the call detail record data stored in the initial storage directory belongs; the storage field has a second field value, the second field value representing the field attribute of the storage field.
[0162] The replacement module 4032 is used to replace the second field value of the stored field in the initial storage directory with the first field value of the call detail record (CDR) field in the call detail record (CDR) data that is consistent with the stored field in the initial storage directory, so as to obtain the updated storage directory.
[0163] The third determining module 4033 is used to determine the key value of the call detail record (CDR) data based on the updated storage directory; wherein the key value represents the data identifier of the CDR data.
[0164] The fourth determining module 4034 is used to retrieve the preset database and determine the storage directory of the call detail record (CDR) data based on the updated storage directory, the key value of the CDR data, and the preset database. The preset database includes at least one hash value and a key value corresponding to the hash value. The hash value represents the storage attribute of the CDR data stored in the preset database.
[0165] In one example, the fourth determining module 4034 is specifically used for:
[0166] If it is determined that there is a key value in the preset database that is the same as the key value of the call detail record (CDR) data, then add 1 to the hash value corresponding to the key value in the preset database that is the same as the key value of the CDR data to obtain the hash value of the CDR data.
[0167] Based on the hash value of the call detail record (CDR) data and the preset threshold in the target storage rule information, the first directory suffix field of the CDR data is determined; wherein, the first directory suffix field represents the storage sequence number of the CDR data in the storage directory; the preset threshold represents the maximum amount of CDR data that the target storage rule information can process.
[0168] The storage directory of the call detail record (CDR) data is obtained based on the updated storage directory and the first directory suffix field of the CDR data.
[0169] In one example, the first directory suffix field k = m / n; where m is the hash value of the call detail record data and n is the preset threshold in the target storage rule information.
[0170] In one example, the fourth determining module 4034 is also specifically used for:
[0171] If it is determined that there is no key value in the preset database that is the same as the key value of the call detail record (CDR) data, then the key value of the CDR data is added to the preset database, and the hash value of the CDR data is determined to be 1.
[0172] The second directory suffix field of the call detail record (CDR) data is set to 0; where the second directory suffix field represents the storage sequence number of the CDR data in the storage directory.
[0173] The storage directory of the call detail record (CDR) data is obtained based on the updated storage directory and the second directory suffix field of the CDR data.
[0174] In one example, device 400 also includes:
[0175] The reading unit 405 is used to read configuration information; wherein, the configuration information is used to export target data, the target data is the call detail record data to be exported; the configuration information includes at least one filter field of the target data, and the storage directory of the target data.
[0176] The third determining unit 406 is used to determine the storage directory in the configuration information as the target storage directory; wherein, the target storage directory is the storage directory where the target data is located.
[0177] The first export unit 407 is used to export the target data stored in the target storage directory to the Secure File Information Transfer Subsystem (SFTP) server, based on the filter field in the configuration information, if it is determined that no subdirectory exists in the target storage directory.
[0178] In one example, after the third determining unit 403 determines the storage directory in the configuration information, it includes:
[0179] The second export unit 408 is used to export the target data stored in the subdirectory if it is determined that there is at least one subdirectory in the target storage directory, and obtain an export file; wherein the export file includes the target data; the export file corresponds one-to-one with the subdirectory.
[0180] Upload unit 409 is used to merge the exported files to obtain a merged exported file, and then upload the merged exported file to the SFTP server.
[0181] The apparatus in this embodiment can execute the technical solutions in the above method. Its specific implementation process and technical principles are the same, and will not be repeated here.
[0182] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 5 As shown, the electronic device 500 includes: a memory 501 and a processor 502; the memory 501 is a memory for storing executable instructions of the processor 502.
[0183] The processor 502 is configured to perform the method provided in the above embodiments.
[0184] The electronic device 500 also includes a receiver 503 and a transmitter 504. The receiver 503 is used to receive instructions and data sent by other devices, and the transmitter 504 is used to send instructions and data to external devices.
[0185] Figure 6This is a block diagram illustrating an electronic device 800 according to an exemplary embodiment. The electronic device 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.
[0186] Electronic device 800 may include one or more of the following components: processing component 802, memory 804, power supply component 806, multimedia component 808, audio component 810, input / output interface 812, sensor component 814, and communication component 816.
[0187] Processing component 802 typically controls the overall operation of electronic device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.
[0188] Memory 804 is configured to store various types of data to support the operation of electronic device 800. Examples of this data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic storage, flash memory, magnetic disk, or optical disk.
[0189] Power supply component 806 provides power to various components of electronic device 800. Power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800.
[0190] Multimedia component 808 includes a screen that provides an output interface between electronic device 800 and user. In some embodiments, the screen may include a liquid crystal display and a touch panel. If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When electronic device 800 is in an operating mode, such as a shooting mode or video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
[0191] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone configured to receive external audio signals when electronic device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.
[0192] Input / output interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.
[0193] Sensor assembly 814 includes one or more sensors for providing status information for various aspects of electronic device 800. For example, sensor assembly 814 can detect the on / off state of electronic device 800, the relative positioning of components such as the display and keypad of electronic device 800, changes in position of electronic device 800 or a component of electronic device 800, the presence or absence of user contact with electronic device 800, orientation or acceleration / deceleration of electronic device 800, and temperature changes of electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.
[0194] Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. Electronic device 800 can access wireless networks based on communication standards. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication module to facilitate short-range communication. For example, the near-field communication module may be implemented based on radio frequency identification (RFID), infrared data association (IRA) technology, ultra-wideband (UWB) technology, Bluetooth technology, and other technologies.
[0195] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits, digital signal processors, digital signal processing devices, programmable logic devices, field-programmable gate arrays, controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.
[0196] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of an electronic device 800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a random access memory, magnetic tape, floppy disk, or optical data storage device, etc.
[0197] This application also provides a non-transitory computer-readable storage medium, which, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform the above-described method.
[0198] According to an embodiment of this application, this application also provides a computer program product, which includes: a computer program stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the solution provided in any of the above embodiments.
[0199] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.
[0200] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A data storage method, characterized in that, The method includes: Acquire call detail record (CDR) data and at least one preset storage rule information; wherein, the CDR data is generated by the user during network usage, and the CDR data includes at least one CDR field, the CDR field representing the service information to which the CDR data belongs; the preset storage rule information represents the storage processing method of the CDR data, and the preset storage rule information includes a filter field, an initial storage directory, and a preset threshold, the filter field representing the data dimension to which the CDR data belongs when it is stored, the initial storage directory being a directory path template containing at least one storage field, the storage field being used to be replaced with the actual value of the CDR field when generating the actual storage directory, and the preset threshold representing the maximum amount of CDR data that the preset storage rule information can process; Based on at least one call detail record (CDR) field in the CDR data and a filter field in the at least one preset storage rule information, target storage rule information is determined from the at least one preset storage rule information; wherein, the target storage rule information represents a storage processing method that is effective for processing the CDR data; Based on the call detail record (CDR) data and the target storage rule information, the storage directory of the CDR data is determined; wherein, determining the storage directory of the CDR data includes: generating an updated storage directory based on the initial storage directory in the target storage rule information and the CDR data; determining the storage directory suffix of the CDR data based on the updated storage directory and a preset threshold in the target storage rule information; and obtaining the storage directory of the CDR data by combining the updated storage directory and the storage directory suffix. The call detail records (CDRs) are stored in the CDR storage directory.
2. The method according to claim 1, characterized in that, The call detail record (CDR) field has a first field value, which represents the field attribute of the CDR field; the filter field includes at least one enumeration value, which represents the field type of the filter field; Based on at least one call detail record (CDR) field in the CDR data and a filter field in the at least one preset storage rule information, the target storage rule information is determined from the at least one preset storage rule information, including: The call detail record (CDR) field that is identical to the filter field in the preset storage rule information among the at least one CDR field is identified as the target field of the CDR data; wherein, the target field represents the data dimension to which the CDR data belongs when it is stored; If it is determined that there is an enumerated value in the filter field that is the same as the first field value of the target field, then the preset storage rule information to which the filter field belongs in the at least one preset storage rule information is determined to be the target storage rule information.
3. The method according to claim 1, characterized in that, The call detail record (CDR) field has a first field value, which represents the field attribute of the CDR field; the updated storage directory is generated based on the initial storage directory in the target storage rule information and the CDR data; The storage directory suffix of the call detail record (CDR) data is determined based on the updated storage directory and the preset threshold in the target storage rule information. By combining the updated storage directory and the storage directory suffix, the storage directory for the call detail record (CDR) data is obtained, including: Extract the initial storage directory from the target storage rule information; wherein, the storage field represents the data dimension to which the call detail record (CDR) data stored in the initial storage directory belongs; the storage field has a second field value, which represents the field attribute of the storage field; The second field value of the stored field in the initial storage directory is replaced with the first field value of the call detail record (CDR) field in the call detail record (CDR) data that is consistent with the stored field in the initial storage directory, to obtain the updated storage directory; Based on the updated storage directory, determine the key value of the call detail record (CDR) data; wherein, the key value represents the data identifier of the CDR data; A preset database is retrieved, and the storage directory of the call detail record (CDR) data is determined based on the updated storage directory, the key value of the CDR data, and the preset database. The preset database includes at least one hash value and a key value corresponding to the hash value, wherein the hash value represents the storage attribute of the CDR data stored in the preset database.
4. The method according to claim 3, characterized in that, The storage directory for the call detail record (CDR) data is determined based on the updated storage directory, the key values of the CDR data, and the preset database, including: If it is determined that there is a key value in the preset database that is the same as the key value of the call detail record (CDR) data, then add 1 to the hash value corresponding to the key value in the preset database that is the same as the key value of the CDR data to obtain the hash value of the CDR data. Based on the hash value of the call detail record (CDR) data and the preset threshold in the target storage rule information, the first directory suffix field of the CDR data is determined; wherein, the first directory suffix field represents the storage sequence number of the CDR data in the storage directory. The storage directory of the call detail record (CDR) data is obtained based on the updated storage directory and the first directory suffix field of the CDR data.
5. The method according to claim 4, characterized in that, First directory suffix field ;in, The hash value of the call detail record (CDR) data. The preset threshold is stored in the target rule information.
6. The method according to claim 3, characterized in that, Determining the storage directory of the call detail record (CDR) data based on the updated storage directory, the key values of the CDR data, and the preset database further includes: If it is determined that there is no key value in the preset database that is the same as the key value of the call detail record (CDR) data, then the key value of the CDR data is added to the preset database, and the hash value of the CDR data is determined to be 1. The second directory suffix field of the call detail record (CDR) data is set to 0; wherein, the second directory suffix field represents the storage sequence number of the CDR data in the storage directory; The storage directory of the call detail record (CDR) data is obtained based on the updated storage directory and the second directory suffix field of the CDR data.
7. The method according to any one of claims 1-6, characterized in that, The method further includes: Read configuration information; wherein, the configuration information is used to export target data, the target data being the call detail records (CDRs) to be exported; the configuration information includes at least one filtering field for the target data, and the storage directory for the target data; The storage directory in the configuration information is determined to be the target storage directory; wherein, the target storage directory is the storage directory where the target data is located; If it is determined that there are no subdirectories in the target storage directory, the target data stored in the target storage directory will be exported to the Secure File Information Transfer Subsystem (SFTP) server according to the filter fields in the configuration information.
8. The method according to claim 7, characterized in that, After determining the storage directory in the configuration information, the following is included: If it is determined that at least one subdirectory exists in the target storage directory, the target data stored in the subdirectory is exported to obtain an exported file; wherein, the exported file includes the target data; the exported file corresponds one-to-one with the subdirectory; The exported files are merged to obtain a merged exported file, which is then uploaded to the SFTP server.
9. A data storage device, characterized in that, The device includes: An acquisition unit is used to acquire call detail record (CDR) data and at least one preset storage rule information. The CDR data is generated by a user during network usage and includes at least one CDR field, which represents the service information to which the CDR data belongs. The preset storage rule information represents the storage processing method of the CDR data and includes a filter field, an initial storage directory, and a preset threshold. The filter field represents the data dimension to which the CDR data belongs when it is stored. The initial storage directory is a directory path template containing at least one storage field, which is used to replace the actual value of the CDR field when generating the actual storage directory. The preset threshold represents the maximum amount of CDR data that the preset storage rule information can process. The first determining unit is configured to determine target storage rule information from the at least one preset storage rule information based on at least one call detail record field in the call detail record data and a filter field in the at least one preset storage rule information; wherein, the target storage rule information represents a storage processing method that is effective for processing the call detail record data; The second determining unit is used to determine the storage directory of the call detail record (CDR) data based on the CDR data and the target storage rule information. Storage unit, used to store the call detail record (CDR) data in the storage directory of the CDR data; The second determining unit is specifically used for: generating an updated storage directory based on the initial storage directory in the target storage rule information and the call detail record (CDR) data; determining the storage directory suffix of the CDR data according to the updated storage directory and a preset threshold in the target storage rule information; and obtaining the storage directory of the CDR data by combining the updated storage directory and the storage directory suffix.
10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 8.
Citation Information
Patent Citations
Method and system for storing sorted call bills by catalog
CN101827175A