Data processing method and apparatus, electronic device, and medium
By setting weight coefficients during data processing to manage the connection between the database and the cache, the resource consumption and data inconsistency problems caused by frequent database reads and writes in traditional methods are solved, achieving efficient data filtering and stable business system operation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHERY AUTOMOBILE CO LTD
- Filing Date
- 2025-07-03
- Publication Date
- 2026-08-04
AI Technical Summary
Traditional data processing methods, when dealing with massive amounts of data, frequently read and write to the database, leading to excessive consumption of network communication resources, affecting data processing latency and business system stability. Furthermore, the cache refresh mechanism cannot guarantee the consistency between cached data and database data.
By setting a preset configuration file, the weight coefficients corresponding to the updated data filtering rules in the database are determined, and the connection with the database is re-established based on the weight coefficients. The data filtering rules in the cache are then updated, reducing the number of database read and write operations and maintaining the consistency between cached data and database data.
It significantly reduces the number of database read/write operations, avoids data processing delays and message blocking, and improves the stability and data processing efficiency of business systems.
Smart Images
Figure CN120892417B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a data processing method, a data processing apparatus, a computer-readable storage medium, and an electronic device. Background Technology
[0002] With the development of vehicle-to-everything (V2X) technology, the types of data reported by vehicle terminals are becoming increasingly diverse. Filtering and selecting data from this vast amount of data to identify specific vehicles and indicators that meet specific criteria is crucial for the efficient and stable operation of enterprise data application systems. Traditional data processing methods involve starting a data processing program to read source data, establishing a database connection, constructing SQL statements based on filtering criteria, querying the database line by line to obtain target data, pushing data that meets the criteria to downstream business systems via message channels, releasing the database connection, and repeating this process until all source data has been processed. However, with the increasing number of connected vehicle terminals and the exponential growth in data volume, this method consumes excessive network communication resources during frequent database reads and writes. Under conditions of limited server capacity and bandwidth, message backlog can occur, leading to data processing delays. For indicators with high timeliness requirements, such as remaining battery power and remaining mileage, prolonged backlog and delays can result in severely distorted data by the time it reaches downstream business systems, impacting their operation.
[0003] Based on this, related technologies load the database into the cache all at once and start a listening process to monitor whether the database is updated in real time. When a database update is detected, the cache is refreshed synchronously, and a timed mechanism is set to refresh the cache periodically. In this way, when processing source data, data can be directly interacted with the system memory according to filtering rules. However, when the database updates data frequently, the cache will also be refreshed frequently, which cannot significantly reduce the number of database connections. When using a timed mechanism to refresh the cache, it cannot effectively guarantee the consistency between cached data and database data. For example, if the cache is set to be updated once an hour, when the database data changes, theoretically the cache can only be updated at the latest after an hour. When processing massive amounts of data, related technologies either cannot significantly reduce the number of database read and write operations or cannot effectively maintain the consistency between cached data and database data. When dealing with some business systems with high real-time requirements, it is easy to cause data processing delays and message blocking, affecting the stability of the business system. Summary of the Invention
[0004] This application aims to at least partially address one of the technical problems in related technologies. To this end, the first objective of this application is to propose a data processing method, which includes: acquiring vehicle data and loading data filtering rules from a database into a cache; determining the weight coefficient corresponding to each updated data filtering rule based on a preset configuration file when the data filtering rules in the database are determined to be updated, wherein the preset configuration file includes multiple data filtering rules and the weight coefficient corresponding to each data filtering rule; determining a target weight coefficient based on the weight coefficient corresponding to each updated data filtering rule; updating the data filtering rules in the cache based on the target weight coefficient; and performing data filtering operations on the vehicle data according to the data filtering rules in the cache. This application, by setting a preset configuration file to determine the weight coefficient corresponding to the updated data rules in the database and determining to re-establish a connection with the database based on the weight coefficient, can significantly reduce the number of database read / write operations when processing massive amounts of data, effectively maintain the consistency between cached data and database data, and to a certain extent avoid data processing delays and message blocking, thereby improving the stability of the business system.
[0005] The second objective of this application is to provide a data processing apparatus.
[0006] The third objective of this application is to provide a computer-readable storage medium.
[0007] The fourth objective of this application is to propose an electronic device.
[0008] To achieve the above objectives, a first aspect of this application proposes a data processing method, comprising: acquiring vehicle data and loading data filtering rules from a database into a cache; determining a weight coefficient corresponding to each updated data filtering rule based on a preset configuration file when the data filtering rules in the database are determined to be updated, wherein the preset configuration file includes multiple data filtering rules and a weight coefficient corresponding to each data filtering rule; determining a target weight coefficient based on the weight coefficient corresponding to each updated data filtering rule; updating the data filtering rules in the cache based on the target weight coefficient; and performing data filtering operations on the vehicle data according to the data filtering rules in the cache.
[0009] According to one embodiment of this application, updating the data filtering rules in the cache based on the target weight coefficient includes: loading the updated data filtering rules in the database into the cache when the target weight coefficient is greater than a preset weight coefficient threshold.
[0010] According to one embodiment of this application, the above method further includes: when the target weight coefficient is less than or equal to a preset weight coefficient threshold, performing data filtering operations on the vehicle data according to the data filtering rules in the cache to determine the target data, and pushing the target data to downstream services.
[0011] According to one embodiment of this application, determining a target weight coefficient based on the weight coefficients corresponding to the updated data filtering rules includes: determining the target weight coefficient based on the sum of the weight coefficients corresponding to the updated data filtering rules.
[0012] According to one embodiment of this application, the method further includes: obtaining the current business of the vehicle; and determining a preset weight coefficient threshold based on the current business of the vehicle.
[0013] According to one embodiment of this application, obtaining vehicle data includes: obtaining vehicle source data; and parsing the vehicle source data to determine vehicle data.
[0014] According to one embodiment of this application, after pushing the target data to downstream services, the above method further includes: if there is unprocessed vehicle source data in the vehicle source data, parsing the unprocessed vehicle source data to determine the vehicle data.
[0015] To achieve the above objectives, a second aspect of this application provides a data processing apparatus, comprising: an acquisition module for acquiring vehicle data and loading data filtering rules from a database into a cache; a first determination module for determining a weight coefficient corresponding to each updated data filtering rule based on a preset configuration file when it is determined that the data filtering rules in the database have been updated, wherein the preset configuration file includes multiple data filtering rules and a weight coefficient corresponding to each data filtering rule; a second determination module for determining a target weight coefficient based on the weight coefficient corresponding to each updated data filtering rule; and an update module for updating the data filtering rule operation module in the cache based on the target weight coefficient, for performing data filtering operations on the vehicle data according to the data filtering rules in the cache.
[0016] To achieve the above objectives, a third aspect of this application provides a computer-readable storage medium having a data processing program stored thereon, which, when executed by a processor, implements the aforementioned data processing method.
[0017] To achieve the above objectives, a fourth aspect of this application provides an electronic device, including a memory, a processor, and a data processing program stored in the memory and executable on the processor. When the processor executes the data processing program, it implements the aforementioned data processing method.
[0018] According to the data processing method, apparatus, electronic device, and medium of this application, vehicle data is acquired, and data filtering rules from the database are loaded into a cache. When it is determined that the data filtering rules in the database are updated, a weight coefficient corresponding to each updated data filtering rule is determined based on a preset configuration file. The preset configuration file includes multiple data filtering rules and a weight coefficient corresponding to each data filtering rule. A target weight coefficient is determined based on the weight coefficient corresponding to each updated data filtering rule. The data filtering rules in the cache are updated based on the target weight coefficient, and data filtering operations are performed on the vehicle data according to the data filtering rules in the cache. This application, by setting a preset configuration file to determine the weight coefficient corresponding to the updated data rules in the database and determining to re-establish a connection with the database based on the weight coefficient, can significantly reduce the number of database read / write operations when processing massive amounts of data, and can effectively maintain the consistency between cached data and database data, thereby avoiding data processing delays and message blocking to a certain extent and improving the stability of the business system. Attached Figure Description
[0019] Figure 1 This is a flowchart of a data processing method according to some embodiments of this application;
[0020] Figure 2 This is a flowchart of a data processing method according to other embodiments of this application;
[0021] Figure 3 This is a block diagram of a data processing apparatus according to some embodiments of this application;
[0022] Figure 4 This is a block diagram of an electronic device according to some embodiments of this application. Detailed Implementation
[0023] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.
[0024] The data processing methods, apparatus, electronic devices, and media of embodiments of this application will now be described in detail with reference to the accompanying drawings.
[0025] Figure 1 This is a flowchart of a data processing method according to some embodiments of this application. (Refer to...) Figure 1 The data processing method in this application embodiment may include the following steps:
[0026] S110 retrieves vehicle data and loads the data filtering rules from the database into the cache.
[0027] Specifically, vehicle data includes the Vehicle Identification Number (VIN) of multiple vehicles, the remaining battery power of the vehicle, the number of times the vehicle's intelligent driving system has been used, the duration of intelligent driving use, the mileage driven by intelligent driving, and the number of braking actions. After the application obtains the vehicle data, it establishes a connection with the database, loads the data filtering rules in the database into the cache, and then releases the connection with the database.
[0028] S120, when it is determined that the data filtering rules in the database are updated, the weight coefficient corresponding to each updated data filtering rule is determined based on the preset configuration file, wherein the preset configuration file includes multiple data filtering rules and the weight coefficient corresponding to each data filtering rule.
[0029] S130, determine the target weight coefficient based on the weight coefficient corresponding to each updated data filtering rule.
[0030] Specifically, before using the cached data filtering rules to filter vehicle data, the data filtering rules in the database may change. Therefore, a listening process needs to be started to monitor whether the data filtering rules in the database have been updated. If the data filtering rules in the database have been updated, the application re-establishes a connection with the database, loads the updated data filtering rules from the database into the cache, then releases the connection with the database, and uses the updated data filtering rules in the cache to filter the vehicle data. If the data filtering rules in the database have not been updated, the application does not need to re-establish a connection with the database and uses the original data filtering rules in the cache to filter the vehicle data.
[0031] However, when the database frequently updates its data filtering rules, the cache also refreshes frequently, failing to significantly reduce the number of database connections. If a timed mechanism is used to refresh the cache, the consistency between the data filtering rules in the cache and those in the database cannot be effectively guaranteed. Therefore, when a data filtering rule update in the database is determined, the weight coefficient corresponding to each updated data filtering rule is found in a preset configuration file. A target weight coefficient is then determined based on this weight coefficient. For example, the weight coefficient of each updated data filtering rule is input into a preset formula to output the target weight coefficient. Further, based on the target weight coefficient, it is determined whether to re-establish a connection with the database to update the data filtering rules in the cache. The preset configuration file includes multiple data filtering rules and their corresponding weight coefficients. A higher weight coefficient for a data filtering rule indicates a higher real-time requirement for the data being filtered by that rule.
[0032] S140 updates the data filtering rules in the cache based on the target weight coefficient, and performs data filtering operations on the vehicle data according to the data filtering rules in the cache.
[0033] Specifically, the decision to update the data filtering rules in the cache can be determined based on the magnitude of the target weight coefficient. For example, if the target weight coefficient is relatively large, a new connection to the database is established, the updated data filtering rules from the database are loaded into the cache, and the vehicle data is filtered according to the updated data filtering rules in the cache. If the target weight coefficient is relatively small, there is no need to re-establish a connection to the database; the vehicle data is filtered directly according to the original data filtering rules in the cache.
[0034] It should be noted that the above method can be applied to both the vehicle and the cloud. When applied to the vehicle, the database is the vehicle database and the cache is the vehicle cache. When applied to the cloud, the database is the cloud database and the cache is the cloud cache.
[0035] This application determines the weight coefficients corresponding to the updated data rules in the database by setting a preset configuration file, and then re-establishes a connection with the database based on the weight coefficients. This can significantly reduce the number of database read / write operations when processing massive amounts of data, and effectively maintain the consistency between cached data and database data. To a certain extent, it avoids data processing delays and message blocking, and improves the stability of the business system.
[0036] In some embodiments, updating the data filtering rules in the cache based on the target weight coefficient includes: loading the updated data filtering rules in the database into the cache when the target weight coefficient is greater than a preset weight coefficient threshold.
[0037] In some embodiments, the method further includes: when the target weight coefficient is less than or equal to a preset weight coefficient threshold, performing data filtering operations on the vehicle data according to the data filtering rules in the cache to determine the target data, and pushing the target data to downstream services. The preset weight coefficient threshold can be determined according to actual conditions, and no specific restrictions are imposed here.
[0038] Specifically, the decision to update the cached data filtering rules can be made by comparing the target weight coefficient with a preset weight coefficient threshold. For example, if the target weight coefficient is greater than the preset weight coefficient threshold, it indicates that the data to be filtered by the updated data filtering rules in the database has high real-time requirements, and a new connection to the database needs to be established to update the cached data filtering rules. If the target weight coefficient is less than the preset weight coefficient threshold, it indicates that the data to be filtered by the updated data filtering rules in the database does not have high real-time requirements, and a new connection to the database is not needed. The original data filtering rules in the cache can be used to filter the vehicle data to determine the target data, and then the target data can be pushed to downstream services.
[0039] In this way, when dealing with massive amounts of data, the number of database read and write operations can be significantly reduced, and the consistency between cached data and database data can be effectively maintained.
[0040] In some embodiments, determining the target weight coefficient based on the weight coefficients corresponding to the updated data filtering rules includes: determining the target weight coefficient based on the sum of the weight coefficients corresponding to the updated data filtering rules.
[0041] Specifically, the target weight coefficient can be determined by calculating the sum of the weight coefficients corresponding to each updated data filtering rule.
[0042] For example, suppose the data filtering rules and their corresponding weight coefficients in the preset configuration file include: Vehicle Identification VIN code - weight coefficient 1; Remaining vehicle battery power - weight coefficient 1; Remaining vehicle mileage - weight coefficient 1; Number of times intelligent driving was used - weight coefficient 0.2; Intelligent driving usage time - weight coefficient 0.2; Mileage driven by intelligent driving - weight coefficient 0.2; Number of braking actions - weight coefficient 0.1. If the updated data filtering rules in the database are Vehicle Identification VIN code and Number of braking actions, the target weight coefficient is 1.1.
[0043] In some embodiments, the above method further includes: obtaining the current business of the vehicle; and determining a preset weight coefficient threshold based on the current business of the vehicle.
[0044] Specifically, when the business data required for the vehicle's current business needs to be cached and updated frequently, a smaller preset weight coefficient threshold can be selected; when the business data required for the vehicle's current business does not need to be cached and updated frequently, a larger preset weight coefficient threshold can be selected. Therefore, the preset weight coefficient threshold can be determined based on the vehicle's current business. For example, the preset weight coefficient threshold corresponding to the vehicle's current business can be determined by looking up a two-dimensional relationship mapping table between vehicle businesses and preset weight coefficient thresholds. This two-dimensional relationship mapping table includes multiple vehicle businesses and the preset weight coefficient threshold corresponding to each vehicle business.
[0045] In some embodiments, acquiring vehicle data includes: acquiring vehicle source data; and parsing the vehicle source data to determine the vehicle data.
[0046] Specifically, vehicle source data refers to raw data obtained by the application from data sources (such as sensors, log files, external systems, etc.). For example, the vehicle's remaining battery power can be collected by the battery management system; the number of times intelligent driving was used, the duration of intelligent driving use, and the mileage achieved through intelligent driving can be collected by the on-board diagnostic system; and the number of braking actions can be checked by a position sensor located at the brake pedal. Because vehicle source data is not uniformly formatted, it needs to be parsed after acquisition, including data cleaning, format conversion, and type conversion, to convert the vehicle source data into a format that the application can understand and process, thus determining the vehicle data.
[0047] In some embodiments, after pushing the target data to downstream services, the above method further includes: if there is unprocessed vehicle source data in the vehicle source data, parsing the unprocessed vehicle source data to determine the vehicle data.
[0048] Specifically, due to the large volume of vehicle source data, it is necessary to process the vehicle source data in batches to improve data processing efficiency. Therefore, after performing data filtering on the current batch of vehicle source data and pushing the filtered target data to downstream businesses, it is also necessary to determine whether there is any unprocessed vehicle source data. If it is determined that there is still unprocessed vehicle source data, the unprocessed vehicle source data is parsed to identify the vehicle data. Then, the data filtering rules in the database are loaded into the cache, and the database is monitored for updates. If the database is updated, the target weight coefficient is determined based on the weight coefficient corresponding to each updated data filtering rule, and the cached data filtering rules are updated based on the target weight coefficient.
[0049] As a concrete example, refer to Figure 2 The data processing method in this application embodiment may further include the following steps:
[0050] S201, Begin.
[0051] S202, the application reads and parses the source data.
[0052] S203, the application establishes a connection with the database.
[0053] S204, the application loads the data filtering rules from the database into the cache.
[0054] S205, the application releases its connection to the database.
[0055] S206, monitor updates to data filtering rules in the database.
[0056] S207, determine the weight coefficient corresponding to each updated data filtering rule based on the preset configuration file.
[0057] S208, determine the target weight coefficient based on the weight coefficient corresponding to each updated data filtering rule.
[0058] S209, determine whether the target weight coefficient is greater than the preset weight coefficient threshold. If yes, execute S203; otherwise, execute S210.
[0059] S210 performs data filtering operations on vehicle data according to the data filtering rules in the cache to determine the target data.
[0060] S211 pushes the target data to downstream businesses.
[0061] S212, determine if there is any unprocessed data in the source data. If so, execute S202; otherwise, execute S213.
[0062] S213, End.
[0063] In summary, this application determines the weight coefficients corresponding to the updated data rules in the database by setting a preset configuration file, and then determines to re-establish a connection with the database based on the weight coefficients. This can significantly reduce the number of database read / write operations when processing massive amounts of data, and effectively maintain the consistency between cached data and database data. To a certain extent, it avoids data processing delays and message blocking, and improves the stability of the business system.
[0064] Corresponding to the above embodiments, this application also proposes a data processing apparatus.
[0065] Reference Figure 3 The data processing device 300 includes: an acquisition module 310, a first determination module 320, a second determination module 330, an update module 340, and an operation module 350.
[0066] The acquisition module 310 acquires vehicle data and loads the data filtering rules from the database into a cache. The first determination module 320, upon determining that the data filtering rules in the database have been updated, determines the weight coefficient corresponding to each updated data filtering rule based on a preset configuration file. The preset configuration file includes multiple data filtering rules and their corresponding weight coefficients. The second determination module 330 determines a target weight coefficient based on the weight coefficient corresponding to each updated data filtering rule. The update module 340 updates the data filtering rules in the cache based on the target weight coefficient. The operation module 350 performs data filtering operations on the vehicle data according to the data filtering rules in the cache.
[0067] According to one embodiment of this application, the update module 340 is specifically used to load the updated data filtering rules in the database into the cache when the target weight coefficient is greater than a preset weight coefficient threshold.
[0068] According to one embodiment of this application, the update module 340 is further configured to, when the target weight coefficient is less than or equal to a preset weight coefficient threshold, perform data filtering operations on the vehicle data according to the data filtering rules in the cache to determine the target data and push the target data to the downstream business.
[0069] According to one embodiment of this application, the second determining module 330 is specifically used to determine the target weight coefficient based on the sum of the weight coefficients corresponding to the updated data filtering rules.
[0070] According to one embodiment of this application, the current service of the vehicle is obtained; a preset weight coefficient threshold is determined based on the current service of the vehicle.
[0071] According to one embodiment of this application, the acquisition module 310 is specifically used to acquire vehicle source data and parse the vehicle source data to determine vehicle data.
[0072] According to one embodiment of this application, after the target data is pushed to the downstream business, if there is unprocessed vehicle source data in the vehicle source data, the unprocessed vehicle source data is parsed to determine the vehicle data.
[0073] It should be noted that the above explanations of the embodiments and beneficial effects of the data processing method also apply to the data processing apparatus of the embodiments of this application. To avoid redundancy, they will not be elaborated in detail here.
[0074] Corresponding to the above embodiments, this application also proposes a computer-readable storage medium.
[0075] The computer-readable storage medium of this application stores a data processing program thereon, which, when executed by a processor, implements the aforementioned data processing method.
[0076] It should be noted that the above explanation of the embodiments and beneficial effects of the data processing method also applies to the computer-readable storage medium of the embodiments of this application. To avoid redundancy, it will not be elaborated in detail here.
[0077] Corresponding to the above embodiments, this application also proposes an electronic device.
[0078] See Figure 4 As shown, the electronic device 400 of this application includes a memory 410, a processor 420, and a data processing program stored in the memory 410 and executable on the processor 420. When the processor executes the data processing program, it implements the aforementioned data processing method.
[0079] It should be noted that the above explanation of the embodiments and beneficial effects of the data processing method also applies to the electronic devices in the embodiments of this application. To avoid redundancy, they will not be elaborated in detail here.
[0080] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.
[0081] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0082] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0083] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0084] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.
[0085] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.
Claims
1. A data processing method, characterized by, The method includes: Acquire vehicle data and load the data filtering rules from the database into the cache; When it is determined that the data filtering rules in the database are updated, the weight coefficient corresponding to each updated data filtering rule is determined based on a preset configuration file, wherein the preset configuration file includes multiple data filtering rules and the weight coefficient corresponding to each data filtering rule; The target weight coefficient is determined based on the weight coefficient corresponding to each of the updated data filtering rules; The data filtering rules in the cache are updated based on the target weight coefficient, and the vehicle data is filtered according to the data filtering rules in the cache. The method of updating the data filtering rules in the cache based on the target weight coefficient includes: when the target weight coefficient is greater than a preset weight coefficient threshold, loading the updated data filtering rules in the database into the cache; If the target weight coefficient is less than or equal to the preset weight coefficient threshold, the vehicle data is filtered according to the data filtering rules in the cache to determine the target data, and the target data is pushed to the downstream business.
2. The data processing method according to claim 1, characterized in that, Determining the target weight coefficient based on the weight coefficients corresponding to the updated data filtering rules includes: The target weight coefficient is determined based on the sum of the weight coefficients corresponding to the updated data filtering rules.
3. The data processing method of claim 1, wherein, The method further includes: Obtain the vehicle's current business; The preset weighting coefficient threshold is determined based on the vehicle's current business.
4. The data processing method of claim 1, wherein, Obtain vehicle data, including: Obtain vehicle source data; The vehicle source data is parsed to determine the vehicle data.
5. The data processing method according to claim 4, characterized in that, After pushing the target data to downstream services, the method further includes: If unprocessed vehicle source data exists in the vehicle source data, the unprocessed vehicle source data is parsed to determine the vehicle data.
6. A data processing apparatus, characterized by The device includes: The acquisition module is used to acquire vehicle data and load the data filtering rules from the database into the cache. The first determining module is used to determine the weight coefficient corresponding to each updated data filtering rule based on a preset configuration file when it is determined that the data filtering rules in the database have been updated. The preset configuration file includes multiple data filtering rules and the weight coefficient corresponding to each data filtering rule. The second determining module is used to determine the target weight coefficient based on the weight coefficient corresponding to each of the updated data filtering rules; The update module is used to update the data filtering rules in the cache based on the target weight coefficient. Updating the data filtering rules in the cache based on the target weight coefficient includes: loading the updated data filtering rules from the database into the cache when the target weight coefficient is greater than a preset weight coefficient threshold; and performing data filtering operations on the vehicle data according to the data filtering rules in the cache when the target weight coefficient is less than or equal to the preset weight coefficient threshold to determine target data, and pushing the target data to downstream services. The operation module is used to perform data filtering operations on the vehicle data according to the data filtering rules in the cache.
7. A computer readable storage medium characterized in that, It stores a data processing program, which, when executed by a processor, implements the data processing method according to any one of claims 1-5.
8. An electronic device, comprising: The device includes a memory, a processor, and a data processing program stored in the memory and executable on the processor. When the processor executes the data processing program, it implements the data processing method according to any one of claims 1-5.