Data processing method and device, storage medium and electronic equipment

Abstract

The invention discloses a data processing method and device, a storage medium and electronic equipment. The method comprises the following steps: monitoring update data in a first database of a network controller through a first process; under the condition that the update data is monitored, a second database is updated through the first process based on the update data, and the second database is a database arranged outside the network service component; and obtaining target network data associated with a network resource updating request from the second database under the condition that the second process receives the network resource updating request, and updating the first database according to the target network data and the network resource updating request. Through application of the method and the device, the problem of low component processing efficiency due to the fact that a single process of a network service component in a cloud computing platform performs data real-time caching and request processing in the prior art and the process processes many transactions is solved.

Description

Technical Field

[0001] The present application relates to the field of cloud computing, and more specifically, to a data processing method, device, storage medium, and electronic device. Background Art

[0002] In cloud computing environments, network services components play a crucial role, managing the dynamic configuration and scheduling of virtual network resources. These components interact with network controllers to control virtual networks. Traditionally, network services architectures rely on a single process responsible for both real-time data caching and request processing. This design presents challenges in large-scale, high-density cloud computing scenarios.

[0003] In large-scale deployments, cloud platforms may need to handle tens of thousands of virtual machines and network resources. This means that a single process in the network service component must cache a large amount of data in the network controller database for high-speed access and updates. Furthermore, the network service component must also receive numerous network resource update requests, resulting in a high volume of process processing and low component processing efficiency.

[0004] Currently, no effective solution has been proposed to the above-mentioned problems in related technologies. Summary of the Invention

[0005] The main purpose of this application is to provide a data processing method, device, storage medium and electronic device to solve the problem in the related art that a single process of a network service component in a cloud computing platform performs real-time data caching and request processing, resulting in the process processing many transactions and thus low component processing efficiency.

[0006] To achieve the above-mentioned objectives, according to one aspect of the present application, a data processing method is provided. The method is applied to a network service component in a cloud computing platform, the network service component including at least a first process and multiple second processes, and includes: monitoring, by the first process, updated data in a first database of a network controller, wherein the network controller is used to manage a virtual network created by the cloud computing platform based on the network data in the first database; updating, by the first process, a second database based on the updated data when the updated data is monitored, the second database being a database provided outside the network service component; and obtaining, by the second process, target network data associated with the network resource update request from the second database when a network resource update request is received, and updating the first database based on the target network data and the network resource update request.

[0007] Furthermore, the data processing method also includes: determining the table name of the data table to which the updated data belongs in the first database; combining the data identifier and the table name of the updated data to obtain a first identifier, and determining a first key-value pair based on the first identifier and the updated data; determining a second key-value pair based on the table name and the data identifier of the updated data; updating the metadata table in the second database according to the first key-value pair, and updating the index table in the second database according to the second key-value pair.

[0008] Furthermore, the data processing method also includes: determining whether the data content of the updated data includes a first attribute, wherein the first attribute refers to an attribute whose attribute value is the same as a preset value; if the data content of the updated data includes the first attribute, deleting the attribute value of the first attribute in the updated data to obtain updated update data; and determining a first key-value pair based on the first identifier and the updated update data.

[0009] Furthermore, the data processing method also includes: determining the target update data based on the network resource update request, and judging whether there is network data with the same table name and data identifier as the target update data in the second database; if there is network data with the same table name and data identifier as the target update data in the second database, determining the network data as the target network data; and storing the target network data in the second database to the first storage area in the second process.

[0010] Furthermore, the data processing method also includes: determining whether the data content of the target update data is the same as that of the target network data; if the data content of the target update data is different from that of the network data, storing the target update data in a second storage area in the second process; and updating the first database based on the data in the second storage area.

[0011] Furthermore, the data processing method also includes: determining whether a second attribute exists in the target network data, wherein the second attribute refers to an attribute whose attribute value is a null value; if the second attribute exists in the target network data, updating the attribute value of the second attribute in the target network data according to a preset value to obtain updated target network data; and determining whether the data content of the target update data is the same as that of the updated target network data.

[0012] Furthermore, the data processing method also includes: when a third process monitors an update of the second database, updating target data in the third database based on the updated data in the second database, wherein the target data has a dependency relationship with the updated data in the second database.

[0013] To achieve the above-mentioned objectives, according to another aspect of the present application, a data processing device is provided. The device includes: a processing module configured to monitor, through a first process, updated data in a first database of a network controller, wherein the network controller is configured to manage a virtual network based on the network data in the first database, the virtual network being created by a cloud computing platform; a first update module configured to update, through the first process, a second database based on the updated data upon monitoring the updated data, the second database being a database provided outside the network service component; and a second update module configured to obtain, through the second process, target network data associated with the network resource update request from the second database upon receiving a network resource update request, and to update the first database based on the target network data and the network resource update request.

[0014] Furthermore, the processing module also includes: a first determination submodule, used to determine the table name of the data table to which the updated data belongs in the first database; a second determination submodule, used to combine the data identifier and table name of the updated data to obtain a first identifier, and determine a first key-value pair based on the first identifier and the updated data; a third determination submodule, used to determine a second key-value pair based on the table name and the data identifier of the updated data; a first update submodule, used to update the metadata table in the second database according to the first key-value pair, and update the index table in the second database according to the second key-value pair.

[0015] Furthermore, the second determination submodule also includes: a first judgment submodule, used to judge whether the data content of the updated data includes a first attribute, wherein the first attribute refers to an attribute whose attribute value is the same as a preset value; a deletion submodule, used to delete the attribute value of the first attribute in the updated data when the data content of the updated data includes the first attribute, and obtain the updated update data; a fourth determination submodule, used to determine the first key-value pair based on the first identifier and the updated update data.

[0016] Furthermore, the second update module also includes: a second judgment submodule, used to determine the target update data according to the network resource update request, and to determine whether there is network data with the same table name and data identifier as the target update data in the second database; a fifth determination submodule, used to determine the network data as the target network data if there is network data with the same table name and data identifier as the target update data in the second database; and a first storage submodule, used to store the target network data in the second database to the first storage area in the second process.

[0017] Furthermore, the second update module also includes: a third judgment submodule, used to judge whether the data content of the target update data is the same as that of the target network data; a second storage submodule, used to store the target update data in the second storage area in the second process when the data content of the target update data is different from that of the network data; and a second update submodule, used to update the first database based on the data in the second storage area.

[0018] Furthermore, the third judgment submodule also includes: a first judgment unit, used to judge whether a second attribute exists in the target network data, wherein the second attribute refers to an attribute whose attribute value is a null value; an update unit, used to update the attribute value of the second attribute in the target network data according to a preset value when the second attribute exists in the target network data, to obtain updated target network data; a second judgment unit, used to judge whether the data content of the target update data is the same as that of the updated target network data.

[0019] Furthermore, the data processing device also includes: a third update module, which is used to update the target data in the third database based on the updated data in the second database when monitoring the update of the second database through a third process, wherein the target data has a dependency relationship with the updated data in the second database.

[0020] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a computer-readable storage medium is provided, which includes a stored executable program, wherein when the executable program runs, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned data processing method.

[0021] In order to achieve the above-mentioned purpose, according to another aspect of the present application, an electronic device is provided, which includes a memory storing an executable program; and a processor for running the program, wherein the above-mentioned data processing method is executed when the program is running.

[0022] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a computer program product is provided, comprising computer instructions, which implement the steps of the above-mentioned data processing method when executed by a processor.

[0023] In an embodiment of the present application, by setting a first process to monitor updated data in the first database of the network controller and updating the second database when the updated data is monitored, and setting a second process to update the first database based on the content in the second database and the network resource update request when receiving a network resource update request, compared with related technologies, the interaction mode between the network service component in the cloud computing platform and the first database of the network controller is changed, and the independent processing of data monitoring and caching and request transaction processing is achieved. This can reduce the workload of a single process and avoid the memory usage when each second process caches data separately, thereby effectively improving the processing efficiency of the network service component.

[0024] It can be seen that the method provided by the present application achieves the purpose of data caching and request processing through different processes, realizes the technical effect of improving the processing efficiency of network service components, and solves the technical problem in the related technology that a single process of the network service component in the cloud computing platform performs real-time data caching and request processing, resulting in the process processing many transactions and thus low component processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0026] Figure 1 This is a hardware structure block diagram of a computer terminal provided according to an embodiment of the present application;

[0027] Figure 2 is a flow chart of a data processing method provided according to an embodiment of the present application;

[0028] Figure 3 is a schematic diagram of the operation of the network service component provided according to an embodiment of the present application;

[0029] Figure 4 is a schematic diagram of a data processing device provided according to an embodiment of the present application;

[0030] Figure 5 This is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] It should be noted that the collected information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation portals for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or institutions to provide users with corresponding operation portals for users to choose to agree or refuse the automated decision-making results; if the user chooses to refuse, the expert decision-making process will be entered.

[0034] Example 1

[0035] According to an embodiment of the present application, an embodiment of a data processing method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0036] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a data processing method. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more (illustrated as 102a, 102b, ..., 102n in the figure) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0037] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0038] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the data processing method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implementing the above-mentioned data processing method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0039] The transmission device 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 10. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0040] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0041] Under the above operating environment, this application provides Figure 2 The data processing method shown. Figure 2 is a flow chart of a data processing method according to embodiment 1 of the present application, which is applied to a network service component in a cloud computing platform, wherein the network service component includes at least a first process and multiple second processes, such as Figure 2 As shown, the method includes the following steps:

[0042] Step S201 : monitoring updated data in a first database of a network controller through a first process, wherein the network controller is used to manage a virtual network according to the network data in the first database, and the virtual network is created by a cloud computing platform.

[0043] Alternatively, cloud computing platforms can provide Infrastructure as a Service (IaaS) for building private and public clouds. They manage large amounts of computing, storage, and network resources, abstracting them through virtualization technology so that users can acquire and release resources on demand. For example, users can quickly create virtual machines (computing resources), allocate storage space (storage resources), and configure network connections (network resources) through the cloud computing platform's interface.

[0044] Optionally, a cloud computing platform includes multiple components, such as a component responsible for computing services and managing the lifecycle of virtual machines; a component that provides block storage services for persistent storage of virtual machine data; and a network service component. The network service component is responsible for network services and can create and manage virtual networks to enable communication between virtual machines and connections between virtual machines and external networks.

[0045] Optionally, the network controller is designed to provide a high-performance, distributed network control plane for virtualized / containerized environments. The network controller provides a distributed approach to managing and controlling virtual networks. The network controller's first database can be a database used to store long-term, unchanging network configurations, or a database used to store short-term network status information. The network controller relies on the network data in the first database to manage virtual networks created by the cloud computing platform. The network controller is independent of the network service components.

[0046] In the related art, the network service component includes multiple API (Application Programming Interface, API) processes, each API process is responsible for processing API requests (i.e., network resource update requests) and updating the first database of the network controller according to the request. In order to achieve differentiated updates of the data in the first database, the API process adopts the method of monitoring changes in the first database and caching the data of the first database in the memory, so as to achieve rapid acquisition and comparison of relevant data in the process of processing network resource update requests. In large-scale and high-density scenarios, the network service component needs to configure many API processes to improve the concurrent processing capabilities of the service, but each API process needs to cache the data in the first database, resulting in a large amount of memory usage. When the data in the first database changes, it will be notified to each API process for processing, resulting in a lot of CPU usage. And each API process is equivalent to a client of the first database. As the number of API processes increases, the processing pressure of the first database will increase, causing some cluster message processing delays, affecting the stability of the network controller. And the API process is also responsible for monitoring the data processing after the data in the first database is updated, affecting the processing of the service of the first database, resulting in slow response of the API process.

[0047] Therefore, in this application, multiple processes are set up in the network service component, and the cache function is externalized to improve the data processing efficiency of the network service component.

[0048] Optionally, the first process in this embodiment is configured to monitor data updates in the first database. The first process continuously monitors updated data in the first database, which may originate from events such as the creation or deletion of a virtual machine or changes to network configuration. When data in the database changes, the first process immediately captures the updated data to ensure that the network service component can respond promptly.

[0049] Step S202: When updated data is monitored, the second database is updated based on the updated data through the first process, where the second database is a database set outside the network service component.

[0050] Optionally, once the first process monitors updated data, it synchronizes the data to a second database. The second database can be an in-memory database, which is a database located outside the network service component. In fact, the aforementioned first database belongs to the network controller, and the aforementioned first database is also a database located outside the network service component.

[0051] Optionally, the second database and the network service component may be set on the same physical node (ie, physical server).

[0052] Step S203: upon receiving the network resource update request, the second process obtains target network data associated with the network resource update request from the second database, and updates the first database according to the target network data and the network resource update request.

[0053] Optionally, a network resource update request can be sent by a user or a system outside the cloud computing platform. In network virtualization and cloud computing environments, a network resource update request is a request to modify, add, or delete network resources. Network resources include, but are not limited to, virtual networks, virtual routers, virtual switches, ports, firewall rules, load balancers, network address translation rules, network services, and the like. For example, a network resource update request might be to add or delete a virtual port or modify an IP address.

[0054] When the second process receives the network resource update request, it retrieves the target network data associated with the network resource update request from the second database. For example, it retrieves the current value of the data requested to be updated in the network resource update request and uses it as the target network data. The second process then updates the first database based on the target network data and the target value of the data requested to be updated in the network resource update request.

[0055] Optionally, after the first database is updated, the network controller can manage the virtual network according to the updated data in the first database.

[0056] Optionally, the above method can not only improve the processing efficiency of the network service component, but also avoid the problem that when each second process caches data separately, each second process needs to interact with the first database, resulting in high processing pressure on the first database, thereby improving the stability of the first database.

[0057] In an embodiment of the present application, by setting a first process to monitor updated data in the first database of the network controller and updating the second database when the updated data is monitored, and setting a second process to update the first database based on the content in the second database and the network resource update request when receiving a network resource update request, compared with related technologies, the interaction mode between the network service component in the cloud computing platform and the first database of the network controller is changed, and the independent processing of data monitoring and caching and request transaction processing is achieved. This can reduce the workload of a single process and avoid the memory usage when each second process caches data separately, thereby effectively improving the processing efficiency of the network service component.

[0058] It can be seen that the method provided by the present application achieves the purpose of data caching and request processing through different processes, realizes the technical effect of improving the processing efficiency of network service components, and solves the technical problem in the related technology that a single process of the network service component in the cloud computing platform performs real-time data caching and request processing, resulting in the process processing many transactions and thus low component processing efficiency.

[0059] Optionally, in the data processing method provided in an embodiment of the present application, the second database is updated based on the updated data through a first process, including: determining the table name of the data table to which the updated data belongs in the first database; combining the data identifier and the table name of the updated data to obtain a first identifier, and determining a first key-value pair based on the first identifier and the updated data; determining a second key-value pair based on the table name and the data identifier of the updated data; updating the metadata table in the second database based on the first key-value pair, and updating the index table in the second database based on the second key-value pair.

[0060] Alternatively, a first identifier can be formed by combining the table name and the data identifier of the updated data, thereby using the first identifier as the key and the updated data as the value to form a first key-value pair for locating a specific metadata record. For example, the first key-value pair can be: "Logical_Switch_757e906f…:{"acls":"","external_ids":""…}", where "Logical_Switch_757e906f…" is the key and "{"acls":"","external_ids":""…}" is the value.

[0061] Optionally, a second key-value pair is formed using the table name as the key and the data identifier of the updated data as the value for fast index lookup. For example, the second key-value pair can be "Chassis_Private:(99f9...,15e377...)", where "Chassis_Private" is the key and "(99f9...,15e377...)" is the value.

[0062] Optionally, a metadata table in the second database is updated based on the first key-value pair. For example, if data matching the key of the first key-value pair exists in the metadata table, the data is updated; if data matching the key of the first key-value pair does not exist in the metadata table, the first key-value pair is added to the metadata table as new data.

[0063] Optionally, the index table in the second database is updated according to the second key-value pair. For example, if the index table contains data identical to the second key-value pair, the second database is not updated; if the index table does not contain data identical to the second key-value pair, the second key-value pair is added to the index table as new data.

[0064] Optionally, the purpose of updating the index table is to maintain the index structure of the metadata, ensuring that when performing a global query or needing to obtain all the data in a table, the data can be quickly located and obtained, thereby improving the overall processing capability of the component.

[0065] It should be noted that by separating metadata updates from index updates, the storage space requirements of the in-memory database can be effectively reduced, while optimizing data retrieval performance, thereby further improving component processing efficiency.

[0066] Optionally, in the data processing method provided in the embodiment of the present application, determining the first key-value pair based on the first identifier and the updated data includes: judging whether the data content of the updated data includes a first attribute, wherein the first attribute refers to an attribute whose attribute value is the same as a preset value; in the case that the data content of the updated data includes the first attribute, deleting the attribute value of the first attribute in the updated data to obtain the updated update data; determining the first key-value pair based on the first identifier and the updated update data.

[0067] Optionally, after determining the updated data, the first process may determine whether the updated data includes the first attribute. For example, if the updated value of a certain attribute is the same as the preset value corresponding to the attribute, the attribute is determined to be the first attribute; if the updated value of a certain attribute is different from the preset value corresponding to the attribute, the attribute is determined not to be the first attribute.

[0068] Optionally, when it is found that the update data includes the first attributes, the first process can remove the values ​​of these attributes to obtain the updated update data, thereby using the first identifier as the key and the updated update data as the value to obtain a first key-value pair.

[0069] Optionally, since the second process no longer caches the data in the first database, the original method of searching and submitting data by the second process needs to be changed. This is to reduce changes to the original code and avoid introducing problems due to modifying the logic of the original code. By designing some new mechanisms, the solution design can be implemented without modifying the original business processing logic. For example, before actually applying the second process, the method ovsdbapp.backend.ovs_idl.idlutils.row_by_value used to search for data from the second process's own memory can be replaced with obtaining data from the second database through the relevant protocol. The acquired data (such as the aforementioned target network data) is instantiated into a new MemRow object. The MemRow object inherits from the class ovs.id.Row used to get data from the second process's own memory, and the data will be cached according to the data structure of this class. And there are the following changes:

[0070] 1. After the MemRow object is instantiated, it is automatically added to the class variable rows of the newly designed transaction class MemTransaction. The newly designed transaction class MemTransaction is used to define how the data content that needs to be updated is submitted to the first database.

[0071] 2. Add the attribute _data to save the original data (ie, target network data) retrieved from the second database.

[0072] 3. Add the attribute _updates to save the data to be updated to the first database.

[0073] Fourth, overload the __getattr__ method. When querying data cached in the second process's local memory, first check whether the data exists in _updates. If not, then check whether it exists in _data. If the queried value is a UUID (Universally Unique Identifier) ​​resource type, first check whether it is cached in MemTransaction.rows. If so, return it directly. If not, retrieve it from the second database. For queries of other types, directly return the data in _data.

[0074] It should be noted that by eliminating the attribute values ​​in the default state and then storing the updated data in the second database, the data storage strategy is effectively optimized and unnecessary data redundancy and memory usage are reduced.

[0075] Optionally, in the data processing method provided in the embodiment of the present application, target network data associated with the network resource update request is obtained from the second database, including: determining the target update data based on the network resource update request, and judging whether there is network data in the second database with the same table name and data identifier as the target update data; if there is network data in the second database with the same table name and data identifier as the target update data, determining the network data as the target network data; and storing the target network data in the second database in the first storage area in the second process.

[0076] In an optional embodiment, when the second process receives a network resource update request, it first parses the request content to determine the network resource to be updated (i.e., the target update data) and its corresponding location in the first database, namely, the table name and data identifier of the target update data. The second process then checks whether a corresponding cache record of the target update data already exists in the second database, specifically, whether network data with the same table name and data identifier as the target update data exists in the second database.

[0077] Optionally, if there is a cache record of corresponding target update data in the second database, the cache record is determined as the target network data, so that the target network data in the second database is stored in the first storage area in the second process.

[0078] In an optional embodiment, when the data of a certain network resource is updated, the data of other network resources may also need to be updated based on the update of the former. For example, when the data of network resource A (such as the output port number) is updated, the data of network resource B (such as the port number of A recorded when establishing a communication connection with A) also needs to be updated. And in some business scenarios, the update of other network resource data depends not only on the update of the former, but also on the update of other network resources. For example, when the data of network resource A is updated, it is necessary not only to know the updated content of A, but also to know the data of network resources C and D, so as to update the data of network resource B based on the updated content of A and the data of C and D.

[0079] Therefore, based on the diversity of business scenarios, optionally, the second process can also determine the target update data based on the network resource update request, and determine whether there is network data in the second database that has an associated relationship with the target update data. For example, the data required to update the data of other network resources (such as the above-mentioned B) based on the target update data (such as the updated content of A mentioned above) is determined to be the network data that has an associated relationship with the target update data (such as the data of C and D mentioned above). Other network resources refer to network resources whose data updates depend on the target update data, and the data of other network resources have been recorded in the first database. Optionally, when there is network data in the second database that has an associated relationship with the target update data, the network data that has an associated relationship with the target update data is also determined as the target network data.

[0080] That is, the target network data may be only the historical cache of the target updated data, or may be the historical cache of the target updated data and the network data associated with the target updated data.

[0081] Optionally, when there is no network data in the second database with the same table name and data identifier as the target update data, the target update data is determined to be new data. In this case, the first database can be updated directly based on the target update data, such as directly writing the target update data into the first database.

[0082] It should be noted that, through the above-mentioned method, accurate acquisition of target network data is achieved, thereby improving the reliability of updating the first database.

[0083] Optionally, in the data processing method provided in the embodiment of the present application, the first database is updated according to the target network data and the network resource update request, including: determining whether the data content of the target update data is the same as that of the target network data; if the data content of the target update data is different from that of the network data, storing the target update data in the second storage area in the second process; and updating the first database based on the data in the second storage area.

[0084] Optionally, the update requested in the network resource update request is sometimes used to set an attribute value to a fixed value. In this case, the update requested in the network resource update request may be the same as the existing historical cache. If the two sets of data are exactly the same, it means that there is no need to update the first database, thereby avoiding unnecessary write operations. Therefore, in this embodiment, it is possible to determine whether the data content of the target update data is the same as the target network data.

[0085] Optionally, when the data content of the target update data is different from that of the network data, the target update data is stored in a second storage area in the second process (eg, the updates mentioned above).

[0086] During updating, the first database is updated based on the data in the second storage area.

[0087] Optionally, when the target update data and the network data have the same data content, the target update data is prohibited from being stored in the second storage area in the second process. Thus, during the update, if the data does not exist in the second storage area, the first database will not be updated.

[0088] In an optional embodiment, if there are data of other network resources that are updated depending on the update of the target update data, the second process can determine the update data of other network resources based on the target update data, and then store the update data of other network resources in the second storage area in the second process, so that the data of other network resources can be updated together during the update.

[0089] In an optional embodiment, before applying the second process, a transaction class MemTransaction can be designed that inherits from ovs.db.id.Transaction. The transaction class MemTransaction is a class for updating transactions in the first database. The following changes are made:

[0090] 1. Add a class variable rows. This class variable is thread- and coroutine-safe, of type HASH table, and is used to store data in Updates. This variable is reset each time a MemTransaction is created and committed.

[0091] 2. Overload the do_commit and commit methods. The do_commit and commit methods are used to traverse all the data in the class variable rows, assemble the data to be updated in the first database based on the changed data, and send it to the first database.

[0092] It should be noted that, through the above method, when the first database is updated, only the data that has actually changed is updated, thereby avoiding repeated writing of unchanged data, thereby improving component processing efficiency.

[0093] Optionally, in the data processing method provided in the embodiment of the present application, determining whether the data content of the target update data is the same as that of the target network data includes: determining whether a second attribute exists in the target network data, wherein the second attribute refers to an attribute whose attribute value is a null value; in the case that the second attribute exists in the target network data, updating the attribute value of the second attribute in the target network data according to a preset value to obtain updated target network data; and determining whether the data content of the target update data is the same as that of the updated target network data.

[0094] Optionally, in order to reduce memory usage, in the second database, the attribute value that is the same as the preset value of the attribute is set to empty. Therefore, after obtaining the target network data, the second process can determine whether the second attribute exists in the target network data.

[0095] Optionally, when a second attribute exists in the target network data, the second process can determine that the attribute value of the second attribute should actually be a preset value. Therefore, the second process can update the attribute value of the second attribute in the target network data based on the preset value to obtain the updated target network data.

[0096] Optionally, after obtaining the updated target network data, it is determined whether the data content of the target update data is the same as that of the updated target network data.

[0097] It should be noted that, through the above-mentioned method, the accuracy of the judgment result can be effectively improved, thereby improving the accuracy of updating the first database and improving the component processing efficiency.

[0098] Optionally, in the data processing method provided in the embodiment of the present application, the network service component also includes a third process, wherein the method also includes: when the third process monitors the update of the second database, updating the target data in the third database based on the updated data in the second database, wherein there is a dependency relationship between the target data and the updated data in the second database.

[0099] Optionally, the third process is used to perform external linkage processing. Specifically, data external to the network service component and network controller may also require updates due to updates to data in the first database. Therefore, data in the third database that is dependent on the data updated in the second database is identified as target data, with the data updated in the second database being substantially identical to the data updated in the first database. For example, the third database may be used to store data on network resources in a system outside the system in which the network service component and network controller reside.

[0100] Optionally, when the third process monitors the update of the second database, it updates the target data in the third database based on the updated data in the second database to achieve linkage processing. The third process may be pre-set with processing logic for linkage processing.

[0101] It should be noted that, through the above-mentioned method, external linkage is achieved through independent processes in the network service component, thereby further improving the component processing efficiency.

[0102] In an optional embodiment, Figure 3is a schematic diagram of the operation of the network service component provided in accordance with an embodiment of the present application, such as Figure 3 As shown, a first process in the network service component monitors updated data in a first database of a network controller. Upon detecting updated data, the first process updates a second database based on the updated data. Upon receiving a network resource update request, the second process retrieves target network data associated with the network resource update request from the second database and updates the first database based on the target network data and the network resource update request. Upon detecting an update to the second database, the third process retrieves relevant data from the second database and updates the target data in the third database based on the updated data in the second database, thereby achieving a coordinated update.

[0103] It can be seen that the method provided by the present application achieves the purpose of data caching and request processing through different processes, realizes the technical effect of improving the processing efficiency of network service components, and solves the technical problem in the related technology that a single process of the network service component in the cloud computing platform performs real-time data caching and request processing, resulting in the process processing many transactions and thus low component processing efficiency.

[0104] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0105] Example 2

[0106] The present application also provides a data processing device. It should be noted that the data processing device of the present application can be used to execute the data processing method provided in the present application. The data processing device provided in the present application is introduced below.

[0107] According to an embodiment of the present application, a device for implementing the above data processing method is also provided, such as Figure 4 As shown, the device includes:

[0108] A processing module 401 is configured to monitor, through a first process, updated data in a first database of a network controller, wherein the network controller is configured to manage a virtual network based on the network data in the first database, the virtual network being created by a cloud computing platform;

[0109] A first updating module 402 is configured to update a second database based on the updated data through a first process when updated data is monitored, where the second database is a database provided outside the network service component;

[0110] The second updating module 403 is configured to obtain target network data associated with the network resource update request from the second database upon receiving the network resource update request through the second process, and update the first database according to the target network data and the network resource update request.

[0111] In an embodiment of the present application, by setting a first process to monitor updated data in the first database of the network controller and updating the second database when the updated data is monitored, and setting a second process to update the first database based on the content in the second database and the network resource update request when receiving a network resource update request, compared with related technologies, the interaction mode between the network service component in the cloud computing platform and the first database of the network controller is changed, and the independent processing of data monitoring and caching and request transaction processing is achieved. This can reduce the workload of a single process and avoid the memory usage when each second process caches data separately, thereby effectively improving the processing efficiency of the network service component.

[0112] It can be seen that the method provided by the present application achieves the purpose of data caching and request processing through different processes, realizes the technical effect of improving the processing efficiency of network service components, and solves the technical problem in the related technology that a single process of the network service component in the cloud computing platform performs real-time data caching and request processing, resulting in the process processing many transactions and thus low component processing efficiency.

[0113] Optionally, in the data processing device provided in the embodiment of the present application, the processing module also includes: a first determination submodule, used to determine the table name of the data table to which the updated data belongs in the first database; a second determination submodule, used to combine the data identifier and the table name of the updated data to obtain a first identifier, and determine a first key-value pair based on the first identifier and the updated data; a third determination submodule, used to determine the second key-value pair based on the table name and the data identifier of the updated data; and a first update submodule, used to update the metadata table in the second database according to the first key-value pair, and update the index table in the second database according to the second key-value pair.

[0114] Optionally, in the data processing device provided in the embodiment of the present application, the second determination submodule also includes: a first judgment submodule, used to determine whether the data content of the updated data includes a first attribute, wherein the first attribute refers to an attribute whose attribute value is the same as a preset value; a deletion submodule, used to delete the attribute value of the first attribute in the updated data when the data content of the updated data includes the first attribute, to obtain updated update data; a fourth determination submodule, used to determine the first key-value pair based on the first identifier and the updated update data.

[0115] Optionally, in the data processing device provided in the embodiment of the present application, the second update module also includes: a second judgment sub-module, used to determine the target update data according to the network resource update request, and to determine whether there is network data with the same table name and data identifier as the target update data in the second database; a fifth determination sub-module, used to determine the network data as the target network data if there is network data with the same table name and data identifier as the target update data in the second database; and a first storage sub-module, used to store the target network data in the second database to the first storage area in the second process.

[0116] Optionally, in the data processing device provided in the embodiment of the present application, the second update module also includes: a third judgment submodule, used to determine whether the data content of the target update data is the same as that of the target network data; a second storage submodule, used to store the target update data in the second storage area in the second process when the data content of the target update data is different from that of the network data; and a second update submodule, used to update the first database based on the data in the second storage area.

[0117] Optionally, in the data processing device provided in the embodiment of the present application, the third judgment submodule also includes: a first judgment unit, used to determine whether a second attribute exists in the target network data, wherein the second attribute refers to an attribute whose attribute value is a null value; an update unit, used to update the attribute value of the second attribute in the target network data according to a preset value when the second attribute exists in the target network data, to obtain updated target network data; a second judgment unit, used to determine whether the data content of the target update data is the same as that of the updated target network data.

[0118] Optionally, in the data processing device provided in an embodiment of the present application, the data processing device also includes: a third update module, which is used to update the target data in the third database based on the updated data in the second database when monitoring the update of the second database through a third process, wherein the target data has a dependency relationship with the updated data in the second database.

[0119] It should be noted that the processing module 401, the first update module 402, and the second update module 403 correspond to steps S201 to S203 in Example 1. The examples and application scenarios implemented by the three modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules or units can be hardware components or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above-mentioned modules can also be part of the device and can be run in the computer terminal 10 provided in Example 1.

[0120] Example 3

[0121] An embodiment of the present application may provide an electronic device, Figure 5 This is a structural block diagram of an electronic device according to an embodiment of the present application. Figure 5 As shown, the electronic device may include: one or more ( Figure 5 Only one is shown) processor 1002, memory 1004, storage controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0122] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implementing the above-mentioned method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0123] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: monitor the updated data in the first database of the network controller through the first process, wherein the network controller is used to manage the virtual network based on the network data in the first database, and the virtual network is created by the cloud computing platform; when the updated data is monitored, update the second database based on the updated data through the first process, and the second database is a database set outside the network service component; when the network resource update request is received through the second process, obtain the target network data associated with the network resource update request from the second database, and update the first database according to the target network data and the network resource update request.

[0124] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: determine the table name of the data table to which the updated data belongs in the first database; combine the data identifier and table name of the updated data to obtain a first identifier, and determine a first key-value pair based on the first identifier and the updated data; determine a second key-value pair based on the table name and the data identifier of the updated data; update the metadata table in the second database according to the first key-value pair, and update the index table in the second database according to the second key-value pair.

[0125] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: determine whether the data content of the updated data includes a first attribute, where the first attribute refers to an attribute whose attribute value is the same as a preset value; if the data content of the updated data includes the first attribute, delete the attribute value of the first attribute in the updated data to obtain updated update data; determine the first key-value pair based on the first identifier and the updated update data.

[0126] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: determine the target update data according to the network resource update request, and judge whether there is network data with the same table name and data identifier as the target update data in the second database; if there is network data with the same table name and data identifier as the target update data in the second database, determine the network data as the target network data; store the target network data in the second database to the first storage area in the second process.

[0127] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: determine whether the data content of the target update data is the same as the target network data; if the data content of the target update data is different from the data content of the network data, store the target update data in the second storage area in the second process; and update the first database based on the data in the second storage area.

[0128] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: determine whether the second attribute exists in the target network data, where the second attribute refers to an attribute whose attribute value is a null value; if the second attribute exists in the target network data, update the attribute value of the second attribute in the target network data according to a preset value to obtain the updated target network data; determine whether the data content of the target update data is the same as that of the updated target network data.

[0129] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: when a third process monitors the update of the second database, it updates the target data in the third database based on the updated data in the second database, wherein the target data has a dependency relationship with the updated data in the second database.

[0130] In an embodiment of the present application, by setting a first process to monitor updated data in the first database of the network controller and updating the second database when the updated data is monitored, and setting a second process to update the first database based on the content in the second database and the network resource update request when receiving a network resource update request, compared with related technologies, the interaction mode between the network service component in the cloud computing platform and the first database of the network controller is changed, and the independent processing of data monitoring and caching and request transaction processing is achieved. This can reduce the workload of a single process and avoid the memory usage when each second process caches data separately, thereby effectively improving the processing efficiency of the network service component.

[0131] It can be seen that the method provided by the present application achieves the purpose of data caching and request processing through different processes, realizes the technical effect of improving the processing efficiency of network service components, and solves the technical problem in the related technology that a single process of the network service component in the cloud computing platform performs real-time data caching and request processing, resulting in the process processing many transactions and thus low component processing efficiency.

[0132] It can be understood by those skilled in the art that Figure 5 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Figure 5 It does not limit the structure of the above electronic device. For example, the electronic device may also include Figure 5 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Figure 5 Different configurations shown.

[0133] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0134] Example 4

[0135] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the data processing method provided in the first embodiment.

[0136] Optionally, in this embodiment, the above-mentioned storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0137] The present application also provides a computer program product, which, when executed on a data processing device, is adapted to perform the steps of the data processing method.

[0138] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0139] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0140] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0141] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0142] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0143] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0144] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A data processing method, characterized in that: A network service component applied to a cloud computing platform, the network service component including at least a first process and a plurality of second processes, the method comprising: monitoring, by the first process, updated data in a first database of a network controller, wherein the network controller is configured to manage a virtual network based on the network data in the first database, the virtual network being created by the cloud computing platform; When the update data is monitored, updating a second database based on the update data by the first process, where the second database is a database set outside the network service component; When a network resource update request is received, the second process obtains target network data associated with the network resource update request from the second database, and updates the first database according to the target network data and the network resource update request.

2. The method according to claim 1, characterized in that Updating the second database based on the updated data by the first process includes: Determine the name of the data table to which the updated data belongs in the first database; combining the data identifier of the updated data and the table name to obtain a first identifier, and determining a first key-value pair based on the first identifier and the updated data; Determine a second key-value pair according to the table name and the data identifier of the updated data; The metadata table in the second database is updated according to the first key-value pair, and the index table in the second database is updated according to the second key-value pair.

3. The method according to claim 2, characterized in that Determining a first key-value pair based on the first identifier and the update data includes: Determining whether the data content of the update data includes a first attribute, wherein the first attribute refers to an attribute whose attribute value is the same as a preset value; In a case where the data content of the update data includes the first attribute, deleting the attribute value of the first attribute in the update data to obtain updated update data; The first key-value pair is determined based on the first identifier and the updated update data.

4. The method according to claim 1, wherein Acquiring target network data associated with the network resource update request from the second database includes: Determine target update data according to the network resource update request, and judge whether there is network data in the second database with the same table name and data identifier as the target update data; If network data having the same table name and data identifier as the target update data exists in the second database, determining the network data as the target network data; The target network data in the second database is stored in the first storage area in the second process.

5. The method according to claim 4, characterized in that Updating the first database according to the target network data and the network resource update request includes: Determining whether the target update data and the target network data have the same data content; In a case where the data contents of the target update data are different from those of the network data, storing the target update data in a second storage area in the second process; The first database is updated based on the data in the second storage area.

6. The method according to claim 5, characterized in that Determining whether the target update data and the target network data have the same data content includes: Determining whether a second attribute exists in the target network data, wherein the second attribute is an attribute whose attribute value is a null value; When the second attribute exists in the target network data, updating the attribute value of the second attribute in the target network data according to a preset value to obtain updated target network data; It is determined whether the target update data and the updated target network data have the same data content.

7. The method according to claim 1, characterized in that The network service component also includes a third process, wherein the method further includes: When the third process monitors the update of the second database, the target data in the third database is updated based on the updated data in the second database, wherein the target data has a dependency relationship with the updated data in the second database.

8. A data processing device, characterized in that: A network service component applied to a cloud computing platform, the network service component including at least a first process and a plurality of second processes, the apparatus comprising: a processing module, configured to monitor, through the first process, updated data in a first database of a network controller, wherein the network controller is configured to manage a virtual network based on the network data in the first database, the virtual network being created by the cloud computing platform; a first updating module configured to update a second database based on the updated data through the first process when the updated data is monitored, where the second database is a database provided outside the network service component; The second update module is used to obtain target network data associated with the network resource update request from the second database when the network resource update request is received through the second process, and update the first database according to the target network data and the network resource update request.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the computer-readable storage medium is located is controlled to execute the data processing method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: a memory storing an executable program; A processor, configured to run the program, wherein the program, when running, executes the data processing method according to any one of claims 1 to 7.

Images