Data storage method, apparatus, medium, and device
By configuring an independent MongoDB database for each business unit and utilizing batch consumption and tag classification, the problem of a sudden increase in pressure on the bank's database was solved, thereby improving data processing throughput and data isolation, and reducing database storage pressure.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- PING AN BANK CO LTD
- Filing Date
- 2023-03-28
- Publication Date
- 2026-07-21
AI Technical Summary
When banks use Elasticsearch as a storage tool, the pressure on the database increases sharply as business volume grows, leading to higher data maintenance costs and impacting data processing throughput and data management efficiency.
Configure an independent MongoDB database for each business unit. Through batch consumption and business unit tag classification, write data to the corresponding MongoDB database and then further write it to the data table. Use RocketMQ for batch writing to achieve data isolation and reduce pressure.
It increased data processing throughput, ensured data isolation for each business unit, prevented a single MongoDB database failure from affecting other business units, and reduced database storage pressure.
Smart Images
Figure CN116383224B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of electronic communication technology, and more particularly to a data update technology field, especially to a data storage method, apparatus, medium and device. Background Technology
[0002] Banks currently have a business scenario where they configure business hit rules to evaluate real-time messages. If a customer matches a rule, the user is considered to have matched the event, and the event code, customer ID, and hit time are stored in the database for query services. Currently, banks typically choose Elasticsearch (a distributed search and analytics engine) as their storage tool. However, as business volume increases, the number of configured events, customer hit data, and data update frequency also increase, leading to a surge in database pressure and rising database maintenance costs. Summary of the Invention
[0003] This application provides a data storage method, apparatus, medium, and device. The data storage method provided pre-configures an independent MongoDB database for each business unit, processes multiple data entries simultaneously using batch processing, and then categorizes each data entry according to its associated business unit tag before writing it into the corresponding MongoDB database, and further writes it into the corresponding data table. This not only increases data processing throughput but also ensures data isolation for each business unit. If one business unit's MongoDB database experiences a problem, it will not affect the normal operation of other business units' MongoDB databases, while also reducing database storage pressure.
[0004] One embodiment of this application provides a data storage method, the data storage method comprising:
[0005] Listen for target business data that matches preset rules;
[0006] Determine whether the target business data is generated for the first time within a preset time period;
[0007] The target business data generated for the first time or the target business data that is not generated for the first time but is the most recent time is used as the data to be written;
[0008] Obtain the business party tag carried in the data to be written;
[0009] Find the corresponding database address based on the business party's tag;
[0010] Based on the database address, the data to be written is written to the corresponding target database.
[0011] In the data storage method described in this application embodiment, the step of writing the data to be written to the corresponding target database according to the database address includes:
[0012] The data to be written is pushed to RocketMQ, which then writes the data to the corresponding target database in batches according to the database address.
[0013] In the data storage method described in this application embodiment, after writing the data to be written to the corresponding target database, the method further includes:
[0014] Obtain the data table address corresponding to the business party tag carried in the data to be written;
[0015] Based on the data table address, the data to be written is written to the corresponding target data table in the target database.
[0016] In the data storage method described in the embodiments of this application, each database includes at least two data tables, which are used to store business data for different writing time periods.
[0017] The step of writing the data to be written into the target data table corresponding to the target database includes:
[0018] The data to be written is written to the corresponding target data table according to the time period to which the data to be written belongs.
[0019] In the data storage method described in the embodiments of this application, after obtaining the calculation result, the method further includes:
[0020] Receive a removal request for the target data table;
[0021] Obtain the data table number contained in the removal request;
[0022] Remove the target data table corresponding to the data table number from the database.
[0023] In the data storage method described in the embodiments of this application, the database is a MongoDB database.
[0024] Accordingly, another aspect of this application embodiment also provides a data storage device, the data storage device comprising:
[0025] The data monitoring module is used to monitor target business data that matches preset rules;
[0026] The data judgment module is used to determine whether the target business data is generated for the first time within a preset time period;
[0027] The data determination module is used to identify the target business data that is generated for the first time or the target business data that is generated for the first time but is closest to the current time as the data to be written.
[0028] The data acquisition module is used to acquire the business party tag carried in the data to be written;
[0029] The data lookup module is used to find the corresponding database address based on the business party's tag;
[0030] The data writing module is used to write the data to be written into the corresponding target database according to the database address.
[0031] In the data storage device described in this application embodiment, the data writing module is used to push the data to be written to RocketMQ, and RocketMQ writes the data to be written in batches to the corresponding target database according to the database address.
[0032] Accordingly, another aspect of this application embodiment also provides a storage medium storing a plurality of instructions adapted for loading by a processor to execute the data storage method described above.
[0033] Accordingly, another aspect of this application embodiment also provides a terminal device, including a processor and a memory, wherein the memory stores a plurality of instructions, and the processor loads the instructions to execute the data storage method described above.
[0034] This application provides a data storage method, apparatus, medium, and device. The method involves: monitoring target business data that matches preset rules; determining whether the target business data is being generated for the first time within a preset time period; using the newly generated target business data or the target business data that is not being generated for the first time but is closest to the current time as data to be written; obtaining the business party tag carried by the data to be written; finding the corresponding database address based on the business party tag; and writing the data to be written to the corresponding target database based on the database address. Using the data storage method provided in this application, by pre-configuring an independent MongoDB database for each business party, batch processing of multiple data entries is achieved. Then, data is categorized according to the business party tag attached to each data entry and written to the corresponding MongoDB database, and further written to the corresponding data table. This not only increases the throughput of data processing but also ensures that the data stored by each business party is isolated. When the MongoDB database corresponding to one business party encounters a problem, it will not affect the normal use of the MongoDB databases of other business parties, while also reducing the storage pressure on the database. Attached Figure Description
[0035] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0036] Figure 1 This is a flowchart illustrating the data storage method provided in an embodiment of this application.
[0037] Figure 2 A schematic diagram of the data storage device provided in the embodiments of this application.
[0038] Figure 3 Another schematic diagram of the data storage device provided in the embodiments of this application.
[0039] Figure 4 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation
[0040] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.
[0041] It should be noted that the following is a brief introduction to the background of this solution:
[0042] This solution addresses the problem that "currently, most data is stored by dividing a single database into multiple data storage areas to store different data types. However, this approach not only increases database storage pressure but also hinders data management." Understandably, banks often have a business scenario where they configure business hit rules to evaluate real-time messages. If a customer matches a rule, the user is considered to have matched the event, and the event code, customer ID, and hit time are stored in the database for query services. Currently, banks typically choose Elasticsearch as their storage tool. However, as business volume increases, the number of configured events, customer hit data, and data update frequency also increase, leading to a surge in database pressure and rising database maintenance costs.
[0043] It's important to explain that Elasticsearch is a distributed, highly scalable, and real-time search and data analytics engine. It easily enables the searching, analysis, and exploration of large amounts of data. Leveraging Elasticsearch's horizontal scalability can make data more valuable in production environments. The implementation principle of Elasticsearch mainly involves the following steps: First, the user submits data to the Elasticsearch database. Then, the tokenizer segments the corresponding text into tokens, storing the token weights and results in the database. When a user searches for data, the results are ranked and scored according to their weights, and then the results are presented to the user.
[0044] To address the aforementioned technical problems, this application provides a data storage method. Using this method, each business entity is pre-configured with an independent MongoDB database. Multiple data entries are processed simultaneously using batch processing, and then categorized according to the business entity tags attached to each entry before being written to the corresponding MongoDB database, and further written to the corresponding data table. This not only increases data processing throughput but also ensures data isolation for each business entity. If one business entity's MongoDB database experiences a problem, it will not affect the normal operation of other business entities' MongoDB databases, while also reducing database storage pressure.
[0045] Please see Figure 1 , Figure 1 This is a flowchart illustrating the data storage method provided in an embodiment of this application. The data storage method is applied to a terminal device. Optionally, the terminal device can be a terminal or a server. Optionally, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, the terminal can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, smart voice interaction device, smart home appliance, or in-vehicle terminal, but is not limited to these.
[0046] In one embodiment, the method may include the following steps:
[0047] Step 101: Monitor the target business data that matches the preset rules.
[0048] Among them, preset rules refer to the hit rules that are set in advance by humans. For example, "a customer browses a page of a specific product" is a preset rule. When the operation of a customer browsing a page of a specific product is detected, the data generated by this rule is used as the target business data.
[0049] Step 102: Determine whether the target business data is generated for the first time within a preset time period.
[0050] In this embodiment, if a customer hits an event for the first time, the data is added to the database. Subsequent hits of the same event only update the hit time. By using the first-time generated target business data or the target business data that is not the first-time generated data but is the most recent to the current time as the data to be written, the number of database accesses can be reduced, thereby reducing the pressure on the database.
[0051] Therefore, it is necessary to first determine whether the target business data is generated for the first time within the preset time period.
[0052] Step 103: Select the target business data that is generated for the first time or the target business data that is generated for the first time but is closest to the current time as the data to be written.
[0053] In this embodiment, the key target business data focused on is real-time updated data. Only data that was last updated within a specified time period qualifies as target business data, and only target business data obtained under this condition can match real-time activities in banking operations. For example, suppose the bank determines a customer's level of interest in promoted products on a designated page based on the customer's activity level when logging into the page (the closer the login time is to the current time, the higher the activity level). If customer 1 and customer 2 log into the designated page in the morning and afternoon of the same day, respectively, customer 2, logging in in the afternoon (closer to the current time), is determined to have a higher level of interest in the promoted products on the page, and marketing activity messages can be targeted to customer 2. Meanwhile, for customer 1, activity reminder messages can be sent to increase customer 1's activity level.
[0054] Based on the above introduction, by using the newly generated target business data or the target business data that is not newly generated but most recent to the current time as the data to be written, the number of database accesses can be reduced, thereby reducing the pressure on the database. Specifically, the primary key can be determined using the customer number + combined event (browsing the page for 3 seconds within a certain time period). If it exists, the data is updated; otherwise, it is inserted. This reduces the number of database accesses, thus reducing the pressure on the database.
[0055] Step 104: Obtain the business party tag carried in the data to be written.
[0056] The business party tag carried in the data to be written can be represented by an ID number, which is used to indicate the business party information corresponding to different data to be written.
[0057] Step 105: Locate the corresponding database address based on the business party label.
[0058] In this process, different business party tags are pre-mapped to their corresponding database addresses to establish a mapping table between business party tags and database addresses. This facilitates quick lookup of the database address corresponding to a business party tag and then writes the data to be written into the corresponding database.
[0059] Step 106: Write the data to be written to the corresponding target database according to the database address.
[0060] When a business unit connects, a separate MongoDB database is applied for for each business unit. This ensures that the data stored by each business unit is isolated. If the MongoDB database of one business unit has a problem, it will not affect the use of the MongoDB database of other business units.
[0061] It's important to explain that MongoDB is a database based on distributed file storage. It is written in C++ and aims to provide scalable, high-performance data storage solutions for web applications.
[0062] MongoDB is a product that falls between relational and non-relational databases. Among non-relational databases, it is the most feature-rich and most relational-like. It supports a very flexible data structure, using a JSON-like BSON format, allowing it to store relatively complex data types. MongoDB's greatest strength lies in its powerful query language, whose syntax is somewhat similar to object-oriented query languages. It can perform almost all the functions of single-table queries in relational databases and also supports indexing.
[0063] In some embodiments, data to be written can be pushed to RocketMQ, which then writes the data in batches to the corresponding target database based on the database address. Using RocketMQ to implement batch consumption, such as consuming 10-100 data entries at a time, can increase the throughput of data processing.
[0064] In some embodiments, after writing the data to be written to the corresponding target database, the method further includes the following steps:
[0065] Obtain the data table address corresponding to the business party tag carried in the data to be written; based on the data table address, write the data to be written into the corresponding target data table in the target database.
[0066] In a business-side MongoDB database, a table is created quarterly. For example, the first quarter of 2022 would have a table named `business_202201`, resulting in four tables per year. How is the data stored in these tables? The creation date of events is determined by the business's configuration. If an event was created in the first quarter, it's written to `business_202201`. This allows for data lifecycle management. Events created earlier are typically in a separate quarterly table, and data in these quarterly tables can be deleted to prevent the data from growing uncontrollably.
[0067] Specifically, a removal request for a target data table can be sent to a terminal device running the data storage method provided in this application; the terminal device obtains the data table number contained in the removal request; and removes the target data table from the database according to the data table number. Multiple data tables are configured in each database to store data for different time periods, and any other data tables can be deleted as needed. Compared to a solution with only one data table, this solution offers greater operational flexibility for data deletion.
[0068] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.
[0069] In practice, this application is not limited by the execution order of the described steps. Without causing conflicts, some steps may be performed in other orders or simultaneously.
[0070] As can be seen from the above, the data storage method provided in this application embodiment monitors target business data that hits preset rules; determines whether the target business data is generated for the first time within a preset time period; takes the newly generated target business data or the target business data that is not newly generated but is closest to the current time as the data to be written; obtains the business party tag carried by the data to be written; finds the corresponding database address according to the business party tag; and writes the data to be written to the corresponding target database according to the database address. Using the data storage method provided in this application embodiment, by pre-configuring an independent MongoDB database for each business party, batch consumption is used to process multiple data entries simultaneously, and then the data is categorized according to the business party tag carried by each data entry and written to the corresponding MongoDB database, and further written to the corresponding data table. This not only increases the throughput of data processing but also ensures that the data stored by each business party is isolated. When the MongoDB database corresponding to one business party has a problem, it will not affect the normal use of the MongoDB databases of other business parties, while also reducing the storage pressure on the database.
[0071] This application also provides a data storage device, which can be integrated into a terminal device.
[0072] Please see Figure 2 , Figure 2 A schematic diagram of the structure of a data storage device provided in an embodiment of this application. The data storage device 30 may include:
[0073] Data monitoring module 31 is used to monitor target business data that hits preset rules;
[0074] Data judgment module 32 is used to determine whether the target business data is generated for the first time within a preset time period;
[0075] Data determination module 33 is used to select the target business data generated for the first time or the target business data that is not generated for the first time and is closest to the current time as the data to be written;
[0076] Data acquisition module 34 is used to acquire the business party tag carried in the data to be written;
[0077] Data lookup module 35 is used to find the corresponding database address based on the business party tag;
[0078] The data writing module 36 is used to write the data to be written into the corresponding target database according to the database address.
[0079] In some embodiments, the data writing module 36 is used to push the data to be written to RocketMQ, and RocketMQ writes the data to be written in batches to the corresponding target database according to the database address.
[0080] In some embodiments, the apparatus further includes a secondary data writing module, configured to obtain the data table address corresponding to the business party tag carried by the data to be written; and write the data to be written into the target data table corresponding to the target database according to the data table address.
[0081] In some embodiments, each database includes at least two data tables, which are used to store business data for different writing time periods; the secondary data writing module is used to write the corresponding target data table according to the time period to which the data to be written belongs.
[0082] In some embodiments, the apparatus further includes a removal module, configured to receive a removal request for a target data table; obtain a data table number contained in the removal request; and remove the target data table corresponding to the data table number from the database.
[0083] In some embodiments, the database is a MongoDB database.
[0084] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities.
[0085] As can be seen from the above, the data storage device 30 provided in this application embodiment includes a data monitoring module 31 for monitoring target business data that hits a preset rule; a data judgment module 32 for determining whether the target business data is generated for the first time within a preset time period; a data determination module 33 for using the target business data generated for the first time or the target business data that is not generated for the first time but is closest to the current time as the data to be written; a data acquisition module 34 for acquiring the business party tag carried by the data to be written; a data search module 35 for searching the corresponding database address according to the business party tag; and a data writing module 36 for writing the data to be written into the corresponding target database according to the database address.
[0086] Please see Figure 3 , Figure 3 This is another schematic diagram of the data storage device provided in the embodiments of this application. The data storage device 30 includes a memory 120, one or more processors 180, and one or more application programs, wherein the one or more application programs are stored in the memory 120 and configured to be executed by the processor 180; the processor 180 may include a data monitoring module 31, a data judgment module 32, a data determination module 33, a data acquisition module 34, a data search module 35, and a data writing module 36.
[0087] For example, the structure and connection relationship of the above components can be as follows:
[0088] Memory 120 can be used to store applications and data. The applications stored in memory 120 contain executable code. Applications can be composed of various functional modules. Processor 180 executes various functional applications and data storage by running the applications stored in memory 120. Furthermore, memory 120 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory 120 may also include a memory controller to provide processor 180 with access to memory 120.
[0089] The processor 180 is the control center of the device, connecting various parts of the terminal through various interfaces and lines. It performs various functions and processes data by running or executing applications stored in the memory 120 and calling data stored in the memory 120, thereby providing overall monitoring of the device. Optionally, the processor 180 may include one or more processing cores; preferably, the processor 180 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications.
[0090] Specifically, in this embodiment, the processor 180 loads the executable code corresponding to the processes of one or more applications into the memory 120 according to the following instructions, and the processor 180 runs the applications stored in the memory 120 to achieve various functions:
[0091] Data monitoring commands are used to monitor target business data that matches preset rules;
[0092] The data judgment instruction is used to determine whether the target business data is generated for the first time within a preset time period;
[0093] The data determination instruction is used to select the target business data that is generated for the first time or the target business data that is not generated for the first time and is most recent to the current time as the data to be written.
[0094] Data acquisition instructions are used to acquire the business party tag carried in the data to be written;
[0095] The data lookup command is used to find the corresponding database address based on the business party's tag;
[0096] A data write instruction is used to write the data to be written into the corresponding target database according to the database address.
[0097] In some embodiments, the data write instruction is used to push the data to be written to RocketMQ, and RocketMQ writes the data to be written in batches to the corresponding target database according to the database address.
[0098] In some embodiments, the program further includes a secondary write instruction for obtaining the data table address corresponding to the business party tag carried by the data to be written; and writing the data to be written into the target data table corresponding to the target database according to the data table address.
[0099] In some embodiments, each database includes at least two data tables, which are used to store business data for different writing time periods; the secondary write instruction is used to write to the corresponding target data table according to the time period to which the data to be written belongs.
[0100] In some embodiments, the program further includes a removal instruction for receiving a removal request for a target data table; obtaining a data table number contained in the removal request; and removing the target data table corresponding to the data table number from the database.
[0101] In some embodiments, the database is a MongoDB database.
[0102] This application also provides a terminal device. The terminal device may be a server, smartphone, computer, tablet computer, or other similar device.
[0103] Please see Figure 4 , Figure 4 A schematic diagram of a terminal device provided in an embodiment of this application is shown. This terminal device can be used to implement the data storage method provided in the above embodiments. The terminal device 1200 can be a smartphone or a tablet computer.
[0104] like Figure 4 As shown, the terminal device 1200 may include an RF (Radio Frequency) circuit 110, a memory 120 including one or more (only one is shown in the figure) computer-readable storage media, an input unit 130, a display unit 140, a sensor 150, an audio circuit 160, a transmission module 170, a processor 180 including one or more (only one is shown in the figure) processing cores, and a power supply 190, etc. Those skilled in the art will understand that... Figure 4 The structure of the terminal device 1200 shown does not constitute a limitation on the terminal device 1200, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:
[0105] RF circuit 110 is used to receive and transmit electromagnetic waves, realizing the mutual conversion between electromagnetic waves and electrical signals, thereby enabling communication with communication networks or other devices. RF circuit 110 may include various existing circuit elements used to perform these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, Subscriber Identity Module (SIM) cards, memory, etc. RF circuit 110 can communicate with various networks such as the Internet, corporate intranets, and wireless networks, or communicate with other devices via wireless networks.
[0106] The memory 120 can be used to store software programs and modules, such as the program instructions / modules corresponding to the data storage method in the above embodiments. The processor 180 executes various functional applications and data storage by running the software programs and modules stored in the memory 120. It can automatically select a vibration reminder mode to update data according to the current scenario of the terminal device, ensuring that scenarios such as meetings are not disturbed, while also ensuring that users can sense incoming calls, thus improving the intelligence of the terminal device. The memory 120 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 120 may further include memory remotely located relative to the processor 180, and these remote memories can be connected to the terminal device 1200 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0107] Input unit 130 can be used to receive input numerical or character information, and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. Specifically, input unit 130 may include touch-sensitive surface 131 and other input devices 132. Touch-sensitive surface 131, also known as a touch display screen or touchpad, can collect user touch operations on or near it (such as user operations using fingers, styluses, or any suitable object or accessory on or near touch-sensitive surface 131), and drive corresponding connection devices according to a pre-set program. Optionally, touch-sensitive surface 131 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to processor 180, and can receive and execute commands from processor 180. In addition, the touch-sensitive surface 131 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. Besides the touch-sensitive surface 131, the input unit 130 may also include other input devices 132. Specifically, other input devices 132 may include, but are not limited to, one or more of the following: a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick.
[0108] Display unit 140 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of terminal device 1200. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Display unit 140 may include display panel 141, optionally configured as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. Further, touch-sensitive surface 131 may cover display panel 141. When touch-sensitive surface 131 detects a touch operation on or near it, it transmits the information to processor 180 to determine the type of touch event. Subsequently, processor 180 provides corresponding visual output on display panel 141 according to the type of touch event. Although in Figure 4 In this embodiment, the touch-sensitive surface 131 and the display panel 141 are implemented as two separate components to realize input and output functions. However, in some embodiments, the touch-sensitive surface 131 and the display panel 141 can be integrated to realize input and output functions.
[0109] The terminal device 1200 may also include at least one sensor 150, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 141 according to the ambient light level, and the proximity sensor can turn off the display panel 141 and / or backlight when the terminal device 1200 is moved to the ear. As a type of motion sensor, a gravity acceleration sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, tapping), etc. Other sensors that the terminal device 1200 may also be configured with, such as a gyroscope, barometer, hygrometer, thermometer, and infrared sensor, will not be described in detail here.
[0110] Audio circuitry 160, speaker 161, and microphone 162 provide an audio interface between the user and terminal device 1200. Audio circuitry 160 converts received audio data into electrical signals, which are then transmitted to speaker 161, where they are converted into sound signals for output. Conversely, microphone 162 converts collected sound signals into electrical signals, which are received by audio circuitry 160, converted back into audio data, and then processed by processor 180 before being transmitted via RF circuitry 110 to, for example, another terminal, or output to memory 120 for further processing. Audio circuitry 160 may also include an earphone jack to facilitate communication between peripheral headphones and terminal device 1200.
[0111] Terminal device 1200, through transmission module 170 (e.g., Wi-Fi module), can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 4 The transmission module 170 is shown, but it is understood that it is not a necessary component of the terminal device 1200 and can be omitted as needed without changing the nature of the invention.
[0112] The processor 180 is the control center of the terminal device 1200. It connects to various parts of the mobile phone via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 120, and by calling data stored in the memory 120, it performs various functions of the terminal device 1200 and processes data, thereby providing overall monitoring of the mobile phone. Optionally, the processor 180 may include one or more processing cores; in some embodiments, the processor 180 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 180.
[0113] The terminal device 1200 also includes a power supply 190 that supplies power to the various components. In some embodiments, the power supply can be logically connected to the processor 180 through a power management system, thereby enabling functions such as discharge management and power consumption management through the power management system. The power supply 190 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.
[0114] Although not shown, the terminal device 1200 may also include a camera (such as a front-facing camera and a rear-facing camera), a Bluetooth module, etc., which will not be described in detail here. Specifically, in this embodiment, the display unit 140 of the terminal device 1200 is a touch screen display, and the terminal device 1200 also includes a memory 120 and one or more programs, one or more of which are stored in the memory 120 and configured to be executed by one or more processors 180. One or more programs contain instructions for performing the following operations:
[0115] Data monitoring commands are used to monitor target business data that matches preset rules;
[0116] The data judgment instruction is used to determine whether the target business data is generated for the first time within a preset time period;
[0117] The data determination instruction is used to select the target business data that is generated for the first time or the target business data that is not generated for the first time and is most recent to the current time as the data to be written.
[0118] Data acquisition instructions are used to acquire the business party tag carried in the data to be written;
[0119] The data lookup command is used to find the corresponding database address based on the business party's tag;
[0120] A data write instruction is used to write the data to be written into the corresponding target database according to the database address.
[0121] In some embodiments, the data write instruction is used to push the data to be written to RocketMQ, and RocketMQ writes the data to be written in batches to the corresponding target database according to the database address.
[0122] In some embodiments, the program further includes a secondary write instruction for obtaining the data table address corresponding to the business party tag carried by the data to be written; and writing the data to be written into the target data table corresponding to the target database according to the data table address.
[0123] In some embodiments, each database includes at least two data tables, which are used to store business data for different writing time periods; the secondary write instruction is used to write to the corresponding target data table according to the time period to which the data to be written belongs.
[0124] In some embodiments, the program further includes a removal instruction for receiving a removal request for a target data table; obtaining a data table number contained in the removal request; and removing the target data table corresponding to the data table number from the database.
[0125] In some embodiments, the database is a MongoDB database.
[0126] This application also provides a storage medium storing a computer program. When the computer program is run on a computer, the computer executes the data storage method described in any of the above embodiments.
[0127] Listen for target business data that matches preset rules;
[0128] Determine whether the target business data is generated for the first time within a preset time period;
[0129] The target business data generated for the first time or the target business data that is not generated for the first time but is the most recent time is used as the data to be written;
[0130] Obtain the business party tag carried in the data to be written;
[0131] Find the corresponding database address based on the business party's tag;
[0132] Based on the database address, the data to be written is written to the corresponding target database.
[0133] It should be noted that, for the data storage method described in this application, those skilled in the art will understand that all or part of the process of the data storage method described in the embodiments of this application can be implemented by a computer program controlling related hardware. The computer program can be stored in a computer-readable storage medium, such as in the memory of a terminal device, and executed by at least one processor within the terminal device. During execution, it can include the process of the embodiments of the data storage method described. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), etc.
[0134] For the data storage device described in the embodiments of this application, its functional modules can be integrated into a single processing chip, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.
[0135] The data storage methods, apparatus, media, and devices provided in the embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this application; at the same time, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A data storage method, characterized in that, include: Listen for target business data that matches preset rules; Determine whether the target business data is generated for the first time within a preset time period; The target business data generated for the first time or the target business data that is not generated for the first time but is the most recent time is used as the data to be written; If a customer is hit for the first time in an event, the data is newly inserted into the database. If the same event is hit again in a subsequent event, only the hit time is updated. Obtain the business party tag carried in the data to be written; Find the corresponding database address based on the business party's tag; Based on the database address, the data to be written is written to the corresponding target database, which is a MongoDB database. When a business unit connects, a separate set of MongoDB databases is applied for for each business unit, so that the data stored by the business unit is isolated.
2. The data storage method as described in claim 1, characterized in that, The step of writing the data to be written to the corresponding target database according to the database address includes: The data to be written is pushed to RocketMQ, which then writes the data to the corresponding target database in batches according to the database address.
3. The data storage method as described in claim 1, characterized in that, After writing the data to be written to the corresponding target database, the method further includes: Obtain the data table address corresponding to the business party tag carried in the data to be written; Based on the data table address, the data to be written is written to the corresponding target data table in the target database.
4. The data storage method as described in claim 3, characterized in that, Each of the databases includes at least two data tables, which are used to store business data for different writing time periods. The step of writing the data to be written into the target data table corresponding to the target database includes: The data to be written is written to the corresponding target data table according to the time period to which the data to be written belongs.
5. The data storage method as described in claim 4, characterized in that, The method further includes: Receive a removal request for the target data table; Obtain the data table number contained in the removal request; Remove the target data table corresponding to the data table number from the database.
6. A data storage device, characterized in that, The data storage device includes: The data monitoring module is used to monitor target business data that matches preset rules; The data judgment module is used to determine whether the target business data is generated for the first time within a preset time period. If the customer hits the event for the first time, the data is added to the database. If the event is hit again in a subsequent period, only the hit time is updated. The data determination module is used to identify the target business data that is generated for the first time or the target business data that is generated for the first time but is closest to the current time as the data to be written. The data acquisition module is used to acquire the business party tag carried in the data to be written; The data lookup module is used to find the corresponding database address based on the business party's tag; The data writing module is used to write the data to be written to the corresponding target database according to the database address. The target database is a MongoDB database. When a business party connects, a separate MongoDB database is applied for for each business party, so that the data stored by the business party is isolated.
7. The data storage device as described in claim 6, characterized in that, The data writing module is used to push the data to be written to RocketMQ, and RocketMQ writes the data to be written in batches to the corresponding target database according to the database address.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to execute the data storage method according to any one of claims 1 to 5.
9. A terminal device, characterized in that, The device includes a processor and a memory, the memory storing a plurality of instructions, the processor loading the instructions to execute the data storage method according to any one of claims 1 to 5.