Data processing method, apparatus, device, and medium

Abstract

Embodiments of the present disclosure provide a data processing method and device, equipment and medium, relating to the technical field of communication, the method comprising: obtaining a service request sent by an application service layer to a communication layer; in the case where the request type of the service request is a first request type, forwarding the service request to a control plane processing module in the communication layer for processing; in the case where the request type of the service request is a second request type, forwarding the service request to a data plane processing module in the communication layer for processing; wherein the request type of the service request is determined based on the data type corresponding to the service request, or the request type of the service request is determined based on the operation type of the target operation corresponding to the service request. In this way, the communication efficiency of the system is effectively improved.

Description

Technical Field

[0001] This disclosure relates to the field of communication technology, and in particular to a data processing method, apparatus, device and medium. Background Technology

[0002] Currently, the Open Baseboard Management Controller (OpenBMC) is the mainstream open-source BMC implementation. Its internal services (such as sensor monitoring, power management, and event logging) generally use Desktop Bus (D-Bus) for inter-process communication. However, as server management functions become increasingly complex, the performance bottleneck of the existing D-Bus communication architecture is becoming increasingly apparent, seriously affecting the system's communication efficiency. Summary of the Invention

[0003] This disclosure provides a data processing method, apparatus, device, and medium to at least solve the above-mentioned technical problems existing in the prior art.

[0004] In a first aspect, embodiments of this disclosure provide a data processing method, the method comprising: Obtain the service request sent from the application service layer to the communication layer; If the service request type is the first request type, the service request will be forwarded to the control plane processing module in the communication layer for processing. If the service request is of type 2, the service request will be forwarded to the data plane processing module in the communication layer for processing. The request type of a service request is determined based on the data type of the service request, or the request type of a service request is determined based on the operation type of the target operation corresponding to the service request.

[0005] Secondly, embodiments of this disclosure provide a data processing method apparatus, the apparatus comprising: The acquisition module is used to acquire service requests sent from the application service layer to the communication layer. The forwarding module is used to forward the service request to the control plane processing module in the communication layer for processing when the service request type is the first request type. The forwarding module is used to forward the service request to the data plane processing module in the communication layer for processing when the service request type is the second request type. The request type of a service request is determined based on the data type corresponding to the service request, or the request type of a service request is determined based on the operation type of the target operation corresponding to the service request.

[0006] Thirdly, embodiments of this disclosure provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the data processing method of the first aspect.

[0007] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the data processing method according to the first aspect.

[0008] The beneficial technical effects of the data processing method provided in this disclosure are at least as follows: It can acquire service requests sent from the application service layer to the communication layer, and if the service request type is a first request type, forward the service request to the control plane processing module in the communication layer for processing; if the service request type is a second request type, forward the service request to the data plane processing module in the communication layer for processing. Thus, based on the communication architecture composed of the data plane processing module and the control plane processing module, it can intelligently distribute service requests to either the data plane processing module or the control plane processing module according to the request type, thereby avoiding the centralized star topology and message queue blocking in high-concurrency data scenarios present in existing communication architectures, and effectively improving the communication efficiency of the system.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of an existing communication architecture of a BMC system provided in an embodiment of this disclosure; Figure 2 This is a schematic diagram of the architecture of a target system provided in an embodiment of this disclosure; Figure 3 This is an interactive schematic diagram based on a target system provided in an embodiment of this disclosure; Figure 4 This is one of the flowcharts illustrating a data processing method provided in this embodiment of the disclosure; Figure 5 This is a second schematic flowchart of a data processing method provided in this embodiment of the disclosure; Figure 6 This is a third schematic flowchart of a data processing method provided in this embodiment of the disclosure; Figure 7This is a fourth flowchart illustrating a data processing method provided in this embodiment of the present disclosure; Figure 8 This is a schematic diagram of the structure of a data processing apparatus provided in an embodiment of this disclosure; Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0011] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0012] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0013] If the application documents contain similar descriptions such as "first / second", the following explanation shall be added: In the following description, the terms "first / second / third" are used only to distinguish similar objects and do not represent a specific order of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0014] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0015] As described in the background section, with the increasing complexity of server management functions, especially in scenarios with high-density computing and high real-time requirements, the existing D-Bus communication architecture is gradually showing the following performance bottlenecks: First, the bottleneck of centralized bus. For example... Figure 1 As shown, all synchronization requests or responses of the BMC system (such as producer service A, consumer service B, and consumer service C) must pass through a single D-Bus daemon, which is responsible for all message routing, serialization, authentication, and scheduling. In high-concurrency data access scenarios, this can easily become a performance bottleneck, causing message queue blocking and a surge in response latency, ultimately leading to a decrease in overall communication efficiency.

[0016] Second, there is a mismatch in communication paradigms. The existing D-Bus communication architecture's "request-response" or "publish-subscribe" model is mainly designed for general, discrete inter-service interactions, while sensor data is essentially a continuous data stream in a producer-consumer model. Using a complex service-oriented mechanism to handle simple sensor data distribution tasks introduces unnecessary serialization overhead, authentication operations, and context switching, which can easily lead to resource waste and communication delays.

[0017] Third, scalability is limited. The existing D-Bus communication architecture is based on a centralized star topology, making it difficult to linearly increase communication throughput by horizontally expanding nodes. This structural deficiency limits the BMC system's scalability and processing performance to handle more intensive management tasks in the future.

[0018] Based on this, the present disclosure provides a data processing method to at least solve the technical problem of low overall communication efficiency of the system caused by the D-Bus communication architecture having problems such as centralized bus bottleneck, communication paradigm mismatch, and limited scalability.

[0019] It should be noted that, since the data processing method provided in this disclosure involves a target system, the target system may be a system containing a microservice architecture that needs to process high-frequency data streams and low-frequency data streams. For example, the target system may be a BMC system, and no specific limitation is made here. Based on this, for ease of understanding, the target system involved in the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings.

[0020] Figure 2 This is a schematic diagram of the architecture of a target system provided in an embodiment of this disclosure.

[0021] like Figure 2 As shown, the target system involved in the data processing method provided in this embodiment adopts a layered structure, which can be divided into an application service layer 21 and a communication layer 22, without specific limitations here.

[0022] The aforementioned application service layer 21 can include two types of roles: consumer services and producer services. Among them, producer services are responsible for generating high-frequency data (such as sensor data), while consumer services express subscription interest in specific data objects.

[0023] The aforementioned communication layer 22 can be further divided into a data plane processing module for processing high-frequency data streams and a control plane processing module for processing low-frequency data streams. Together, they achieve a complete physical and logical separation between high-frequency and low-frequency data streams. The control plane processing module can be built on the D-Bus bus and is specifically designed to handle low-frequency meta-operations requiring strong consistency and complex interactions, such as service discovery, configuration management, and subscription registration. It may include control plane agents, subscription managers, and data object managers, among others, without specific limitations. The data plane processing module, independent of the D-Bus, is specifically responsible for the efficient, near-zero-copy distribution of high-frequency, unidirectional, data-centric information streams (such as sensor readings). It may include shared memory, a data distribution engine, and a channel manager, among others, without specific limitations.

[0024] The client-side data access library 23 is deployed between the application service layer 21 and the communication layer 22, providing a unified and transparent access interface for upper-layer application services. This library encapsulates routing logic, intelligently determining the routing path based on request type: directing low-frequency data streams (such as subscriptions and configurations) to the control plane processing module and high-frequency data streams to the data plane control module. It should be noted that the two only interact at critical configuration points (such as subscription synchronization and memory allocation), thus solidifying a separate architecture consisting of the control plane processing module and the data plane processing module at the system level to ensure its feasibility and efficiency.

[0025] based on Figure 2 The schematic diagram of the target system architecture shown is as follows: Figure 3 As shown, the data processing method provided in this embodiment can specifically include the following: Step 1: The consumer service sends a subscription request to the subscription manager.

[0026] Consumer services could be, for example, fan control services. Additionally, subscription requests can be used to declare interest in or subscription intent for a specific data object.

[0027] Step 2: The subscription manager authenticates and registers the subscription request to obtain subscription relationship information.

[0028] Since a subscription request can declare an interest in or subscription intent for a specific data object, the subscription request can at least include the service identifier of the consumer service and the object identifier of the data object. Therefore, the obtained subscription relationship information can include the service identifier of the consumer service and the object identifier of the data object.

[0029] Step 3: The subscription manager synchronizes subscription relationship information with the data distribution engine.

[0030] Step 4: The producer service will update the data and write it to the shared memory bound to the data object.

[0031] Step 5: The producer service sends the memory region information corresponding to the data object to the data distribution engine.

[0032] The memory region information may include information such as memory pointers or file descriptors used to represent data storage addresses.

[0033] Step 6: The data distribution engine determines the consumer service for the subscribed data object based on the subscription relationship information.

[0034] Step 7: The data distribution engine sends memory region information to the consumer service that subscribes to the data object.

[0035] In this way, a consumer service can declare its interest in a specific data object by sending a subscription request to the subscription manager in the control plane processing module. The subscription manager can then authenticate and register the subscription request, obtaining subscription relationship information consisting of the object identifier of the data object and the service identifier of the consumer service, and synchronize this subscription relationship information to the data distribution engine in the data plane processing module. Thus, through a single control operation, the coordination of permissions and relationships can be completed via the D-Bus (i.e., the control plane processing module).

[0036] Based on this, when updating data, the producer service can atomically write the updated data to the shared memory of the data plane processing module bound to the corresponding data object. After the write operation is complete, the producer service can then send the memory region information of the data object, i.e., a memory pointer or file descriptor, to the data distribution engine in the data plane processing module. The data distribution engine can then query subscription relationship information, identify all consumer services subscribed to the data object, and, through a high-performance point-to-point channel, such as the Socket Control Message-Rights (SCM_RIGHTS) mechanism of Unix Domain Sockets (UDS), directly and in parallel push control messages containing memory pointers or file descriptors to each consumer service. This allows the consumer service to directly access the updated data written to shared memory based on the received memory pointer or file descriptor. In this way, high-frequency data production and consumption form a complete closed loop within the data plane processing module. Furthermore, after the producer service writes to memory, it only triggers a notification. The data distribution engine, based on the pre-defined subscription relationship information, pushes the data access rights to the consumer service in parallel and directly, achieving zero-copy reading across processes. This process completely bypasses D-Bus.

[0037] It should be noted that during the subscription establishment process described in steps 1 to 3, after the subscription manager synchronizes the subscription relationship information with the data distribution engine, the data distribution engine can construct a dedicated data channel between itself and the consumer services in the subscription relationship information, namely the aforementioned high-performance point-to-point channel. This facilitates the subsequent pushing of memory pointers or file descriptors to the corresponding consumer services through this dedicated data channel, allowing the consumer services to read relevant data from shared memory. Furthermore, before the subscription establishment process described in steps 1 to 2, multiple data objects can be statically or dynamically defined when the target system starts. Each data object corresponds to a high-frequency data source (e.g., / system / cpu0 / temperature), which can contain the data object's object identifier (ID) and a predefined data structure. Simultaneously, the control plane processing module and the data plane processing module can be initialized.

[0038] In addition, the data distribution engine needs to obtain subscription relationship information from the subscription manager in the control plane processing module when the target system starts up. Furthermore, since the subscription manager synchronizes the subscription relationship information to the data distribution engine while generating the subscription relationship information, the data distribution engine can directly query the subscription relationship information locally.

[0039] It's also worth noting that all non-data distribution operations, such as service creation / destruction, configuration parameter modification, low-frequency attribute queries, and control command issuance (e.g., manually setting fan speed), are performed through the control plane processing module. This ensures the transactionality, security, and consistency required for such operations. Furthermore, after approving any metadata changes, the control plane processing module is responsible for synchronizing the updated subscription relationship information, data object information, etc., to the data plane processing module to ensure the accuracy and consistency of the data distribution topology. The data plane processing module itself maintains a "stateless" or "weakly stateful" design, faithfully executing the distribution strategy issued by the control plane.

[0040] Based on the architecture diagram of the target system described above, the data processing method provided by the present disclosure will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] Figure 4 This is a flowchart illustrating a data processing method provided in an embodiment of this disclosure.

[0042] like Figure 4 As shown, the execution entity of this data processing method can be the target system. Based on this, the data processing method provided in this embodiment may include the following steps: S410: Obtain the service request sent from the application service layer to the communication layer.

[0043] S420: If the service request type is the first request type, the service request is forwarded to the control plane processing module in the communication layer for processing.

[0044] S430: If the service request type is the second request type, the service request is forwarded to the data plane processing module in the communication layer for processing.

[0045] The aforementioned communication layer may include a data plane processing module and a control plane processing module.

[0046] In some embodiments, the request type of a service request can be determined based on the data type corresponding to the service request, or the request type of a service request can be determined based on the operation type of the target operation corresponding to the service request. It should be noted that the request type of the service request can be divided into a first request type and a second request type. The first request type may indicate that the service request conforms to the D-Bus "request-response" or "publish-subscribe" model, while the second request type may indicate that the service request conforms to the producer-consumer pattern.

[0047] Specifically, it can acquire service requests sent from the application service layer to the communication layer, and if the request type of the service request is the first request type, forward the service request to the control plane processing module in the communication layer for processing; if the request type of the service request is the second request type, forward the service request to the data plane processing module in the communication layer for processing.

[0048] Based on the data processing method provided in this disclosure, service requests sent from the application service layer to the communication layer can be obtained. If the service request type is a first request type, the request is forwarded to the control plane processing module in the communication layer for processing. If the service request type is a second request type, the request is forwarded to the data plane processing module in the communication layer for processing. In this way, based on the communication architecture composed of the data plane processing module and the control plane processing module, service requests can be intelligently routed to either the data plane processing module or the control plane processing module according to the request type. This avoids issues such as centralized star topology and message queue blocking in high-concurrency data scenarios present in existing communication architectures, thereby effectively improving the system's communication efficiency.

[0049] Since the foregoing embodiments can forward service requests to the corresponding processing modules for processing based on the request type of the service request, in order to comprehensively and thoroughly describe the data processing method provided by the embodiments of this disclosure, in one embodiment, the request type of the service request can be determined based on the data type corresponding to the service request.

[0050] Specifically, when the service request is of the first type of data, the first type of data indicates that the data access frequency is lower than the first frequency threshold, that is, the first type of data is a low-frequency data stream, and the request type of the service request is the first request type; when the service request is of the second type of data, the second type of data indicates that the data access frequency is greater than the second frequency threshold, that is, the second type of data is a high-frequency data stream, and the request type of the service request is the second request type.

[0051] The first frequency threshold is less than or equal to the second frequency threshold. The specific values ​​of the two can be determined according to the actual situation, and no specific limitation is made here.

[0052] It should be noted that the first type of data mentioned above can be a subscription request or a control instruction. Since the subscription process corresponding to a subscription request or the control process corresponding to a control instruction has low-frequency and one-time characteristics, the data access frequency of this first type of data is lower than the first frequency threshold. The second type of data mentioned above can be used to store requests for sensor data. Since sensor data belongs to a high-frequency data stream, the data access frequency of this second type of data is greater than the second frequency threshold, which is not specifically limited here.

[0053] In this embodiment, since the data type of the service request can reflect whether the service request belongs to a low-frequency data stream or a high-frequency data stream, the request type of the service request can be accurately determined based on the data type of the service request. This facilitates accurate request routing in the future and avoids the centralized star topology, message queue blocking in high-concurrency data scenarios, and communication paradigm mismatch that exist in the existing communication architecture, thereby effectively improving the communication efficiency of the system.

[0054] In another embodiment, the request type of the above service request can be determined based on the operation type of the target operation corresponding to the service request.

[0055] Specifically, in some embodiments, when the target operation is a read operation, the corresponding service request may be a low-frequency data stream such as a subscription request or control command, and the request type of the service request is the first request type. When the target operation is a write operation, the corresponding service request may be a high-frequency data stream such as a request for storing or updating sensor data, and the request type of the service request is the second request type. Further details are omitted here.

[0056] In this embodiment, since the operation type of the target operation of the service request can reflect whether the service request belongs to a low-frequency data stream or a high-frequency data stream, the request type of the service request can be accurately determined based on the operation type of the service request. This facilitates accurate request routing in the future and effectively avoids the centralized star topology, message queue blocking in high-concurrency data scenarios, and communication paradigm mismatch that exist in the existing communication architecture, thereby effectively improving the communication efficiency of the system.

[0057] Thus, if a service request includes a subscription request sent by the first consumer service in the application service layer, it indicates that the service request belongs to a low-frequency data stream. Based on this, such as Figure 5 As shown, the data processing method provided in this embodiment may further include: S510, determine the request type of the subscription request as the first request type.

[0058] Based on this, the steps described above for forwarding service requests to the control plane processing module in the communication layer for processing specifically include the following steps: S4201 forwards the subscription request to the control plane processing module in the communication layer.

[0059] In some embodiments, the subscription request is used by a first consumer service to subscribe to object data of a first data object, and it may indicate the first consumer service's intention to subscribe to the first data object. The subscription request includes the object identifier of the first data object and the service identifier of the first consumer service, which are not specifically limited here.

[0060] S4202, using the subscription manager in the control plane processing module, based on the object identifier of the first data object and the service identifier of the first consumer service, to generate subscription relationship information between the first data object and the first consumer service.

[0061] Specifically, when a service request includes a subscription request sent by a first consumer service in the application service layer, the request type of the subscription request can be determined to be a first request type, and the service request can be forwarded to the control plane processing module in the communication layer. Since the subscription request may include the object identifier of the first data object and the service identifier of the first consumer service, which is used to indicate the subscription intention of the first consumer service to the first data object, the subscription manager in the control plane processing module can then generate subscription relationship information between the first data object and the first consumer service based on the object identifier of the first data object and the service identifier of the first consumer service.

[0062] In this embodiment, when the subscription request is determined to be of the first request type, the subscription request can be accurately routed to the control plane processing module in the communication layer. Since the subscription request carries the object identifier of the first data object and the service identifier of the first consumer service, indicating the first consumer service's subscription intent for the first data object, the subscription manager in the control plane processing module can generate corresponding subscription relationship information based on the information carried in the subscription request. This achieves centralized and precise management of subscription relationships, facilitating subsequent updates to the data corresponding to the first data object by the data plane processing module, ensuring that data is only pushed to subscribed services, avoiding invalid data transmission, and improving the overall communication efficiency and resource utilization of the system.

[0063] To provide a comprehensive and detailed description of the data processing method provided in this disclosure, in one embodiment, prior to the step of forwarding the subscription request to the control plane processing module in the communication layer, the data processing method provided in this disclosure may further include the following steps: In response to the target system's startup command, multiple data objects are created using the data object manager in the control plane processing module.

[0064] The aforementioned startup command can be used to trigger the startup process or initialization operation of the target system, and is not specifically limited here. Additionally, in some embodiments, the aforementioned plurality of data objects may include a first data object, each data object uniquely indicating a second type of data. The second type of data represents a data access frequency greater than a second frequency threshold, i.e., the high-frequency data stream involved in the embodiments of this disclosure.

[0065] It should also be noted that each data object has its own unique object identifier and a predefined data structure, which are not specifically limited here. The object identifier is used to uniquely distinguish different data objects throughout the system, facilitating subsequent subscription, querying, and retrieval. The data structure is used to standardize the data format, field types, storage methods, and interaction rules of the corresponding high-frequency data streams, which are not specifically limited here.

[0066] In this embodiment, multiple high-frequency data sources can be statically or dynamically defined at system startup, each corresponding to a data object. This facilitates subsequent subscription of data objects by consumer services through the control plane processing module. By abstracting high-frequency data streams into data objects and centrally managing subscriptions through the control plane, it becomes easier to decouple the data plane and control plane, and efficiently distribute requests, thus significantly improving the system's communication efficiency.

[0067] In another embodiment, if a service request includes a write request sent by a producer service in the application service layer, it indicates that the data belongs to a high-frequency data stream. Based on this, as... Figure 6 As shown, the data processing method provided in this embodiment may further include: S610, determine that the request type of the write request is the second request type.

[0068] Based on this, the steps described above for forwarding service requests to the data plane processing module in the communication layer for processing may specifically include: S4301 forwards the write request to the data plane processing module.

[0069] In some embodiments, a write request may include data to be written, and the specific content of the data to be written is not specifically limited in this disclosure.

[0070] S4302 uses the data plane processing module to write the data to be written into the shared memory bound to the second data object, and obtains the memory area information corresponding to the second data object.

[0071] The aforementioned memory region information may include memory pointers or file descriptors, which are related information used to characterize the storage location in shared memory, and no further restrictions are imposed here.

[0072] Specifically, when a service request includes a write request sent by a producer service in the application service layer, the request type of the write request can be determined to be the second request type. Then, the write request can be forwarded to the data plane processing module in the communication layer. The data plane processing module can then write the data to be written into the shared memory bound to the second data object to obtain the memory area information corresponding to the second data object.

[0073] In this embodiment, when the service request includes a write request sent by the producer service in the application service layer, the request type of the write request can be determined to be the second request type, and the write request can be accurately diverted to the data plane processing module in the communication layer. The data plane processing module then writes the data to be written into the shared memory bound to the second data object, thereby obtaining the memory area information corresponding to the second data object. This effectively achieves request diversion and improves communication efficiency.

[0074] Based on this, in one embodiment, such as Figure 7 As shown, after the data plane processing module writes the data to be written into the shared memory bound to the second data object to obtain the memory region information corresponding to the second data object, the data processing method provided in this embodiment may further include: S710 obtains the subscription relationship information between the second data object and the second consumer service through the data distribution engine in the data plane processing module.

[0075] S720, based on the subscription relationship information between the second data object and the second consumer service, sends the memory region information corresponding to the second data object to the second consumer service, so that the second consumer service can access the data in the shared memory bound to the second data object based on the memory region information of the second data object.

[0076] Specifically, the data to be written in the write request can be written into the shared memory bound to the second data object. After obtaining the memory area information corresponding to the second data object, the subscription relationship information between the second data object and the second consumer service can be obtained through the data distribution engine in the data plane processing module. Then, based on the subscription relationship information between the second data object and the second consumer service, the memory area information corresponding to the second data object is sent to the second consumer service, so that the second consumer service can access the data in the shared memory bound to the second data object, i.e., the aforementioned data to be written, based on the memory area information of the second data object.

[0077] In this embodiment, after the producer service writes the updated data to the shared memory area bound to the data object, the data distribution engine directly and in parallel pushes the memory area information, including memory pointers or file descriptors, to each consumer subscribing to the data object based on the data object's subscription relationship. This allows the consumer to directly access the corresponding updated data in the shared memory based on the received memory pointer or file descriptor. Thus, by using a combination of shared memory and file descriptor passing, near-zero copy data access from the producer's memory to the consumer's memory can be achieved.

[0078] In order to provide a comprehensive and detailed description of the data processing method provided in the embodiments of this disclosure, in one embodiment, the data processing method provided in the embodiments of this disclosure may include the following steps: Use the channel manager in the data plane processing module to create a data channel between the data distribution engine and the second consumer service; Based on this, the step of sending the memory region information corresponding to the second data object to the second consumer service based on the subscription relationship information between the second data object and the second consumer service can specifically include: Based on the subscription relationship information between the second data object and the second consumer service, the memory region information corresponding to the second data object is sent to the second consumer service through the data channel.

[0079] Specifically, a data channel between the data distribution engine and the second consumer service is pre-created using the channel manager in the data plane processing module. In this way, based on the subscription relationship information between the second data object and the second consumer service, the memory region information corresponding to the second data object can be quickly sent to the second consumer service through the data channel.

[0080] In one example, after registering the subscription relationship information between a data object and a consumer service through the subscription manager in the control plane processing module, a dedicated data channel between the data distribution engine and the corresponding consumer service can be created using the channel manager in the data plane processing module. This facilitates the subsequent sending of the memory region information corresponding to the data object to the corresponding consumer service through the dedicated channel between the data distribution engine and the corresponding consumer service.

[0081] In this embodiment, a dedicated data channel between the data distribution engine and the consumer service can be created through the channel manager in the data plane processing module. This dedicated data channel between the data distribution engine and the consumer service can then be used to accurately and quickly send the memory region information corresponding to the data object to the corresponding consumer service.

[0082] Based on the same inventive concept, this disclosure provides a data processing apparatus, which can be specifically described in conjunction with the appendix. Figure 8 A data processing apparatus provided in the embodiments of this disclosure will be described in detail.

[0083] Figure 8 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this disclosure.

[0084] like Figure 8 As shown, the data processing device 800 may include: The acquisition module 810 is used to acquire service requests sent from the application service layer to the communication layer; The forwarding module 820 is used to forward the service request to the control plane processing module in the communication layer for processing when the request type of the service request is a first request type. The forwarding module 820 is used to forward the service request to the data plane processing module in the communication layer for processing when the request type of the service request is the second request type. The request type of the service request is determined based on the data type corresponding to the service request, or the request type of the service request is determined based on the operation type of the target operation corresponding to the service request.

[0085] In one embodiment, the request type of the service request is determined based on the data type corresponding to the service request; When the service request is for the first type of data, the data access frequency of the first type of data is lower than the first frequency threshold, and the request type of the service request is the first request type; when the service request is for the second type of data, the data access frequency of the second type of data is greater than the second frequency threshold, and the request type of the service request is the second request type.

[0086] In one embodiment, the request type of the service request is determined based on the operation type of the target operation corresponding to the service request; When the target operation is a read operation, the service request type is a first request type; when the target operation is a write operation, the service request type is a second request type.

[0087] In one embodiment, when the service request includes a subscription request sent by a first consumer service in the application service layer, the request processing apparatus provided in this disclosure includes: The determining module is used to determine that the request type of the subscription request is a first request type; The forwarding module is specifically used to forward the subscription request to the control plane processing module in the communication layer. The subscription request is used by the first consumer service to subscribe to the object data of the first data object. The subscription request includes the object identifier of the first data object and the service identifier of the first consumer service. The generation module is used to generate subscription relationship information between the first data object and the first consumer service based on the object identifier of the first data object and the service identifier of the first consumer service using the subscription manager in the control plane processing module.

[0088] In one embodiment, the request processing apparatus provided in this disclosure includes: A creation module is used to create multiple data objects using the data object manager in the control plane processing module in response to the startup command of the target system. The multiple data objects include the first data object, and each data object indicates a second type of data, the second type of data representing that the data access frequency is greater than a second frequency threshold.

[0089] In one embodiment, when the service request includes a write request sent by a producer service in the application service layer, the request processing apparatus provided in this disclosure includes: The determining module is used to determine that the request type of the write request is the second request type; The forwarding module is specifically used to forward the write request to the data plane processing module, and the write request includes the data to be written. The writing module is used to write the data to be written into the shared memory bound to the second data object using the data plane processing module, thereby obtaining the memory area information corresponding to the second data object.

[0090] In one embodiment, the request processing apparatus provided in this disclosure includes: The acquisition module is also used to acquire subscription relationship information between the second data object and the second consumer service through the data distribution engine in the data plane processing module; The sending module is used to send the memory region information corresponding to the second data object to the second consumer service based on the subscription relationship information between the second data object and the second consumer service, so that the second consumer service can access the data in the shared memory bound to the second data object based on the memory region information of the second data object.

[0091] In one embodiment, the request processing apparatus provided in this disclosure includes: A creation module is used to create a data channel between the data distribution engine and the second consumer service using the channel manager in the data plane processing module. The sending module is specifically used to send the memory region information corresponding to the second data object to the second consumer service through the data channel, based on the subscription relationship information between the second data object and the second consumer service.

[0092] It is understood that, when implementing the corresponding data processing method apparatus provided in the above embodiments, the processing can be distributed to different program modules as needed to complete all or part of the processing described above. Furthermore, the apparatus and the corresponding method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0093] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a data processing method.

[0094] This application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are executed by a processor, causing the processor to execute the data processing method provided in this application.

[0095] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0096] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0097] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0098] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0099] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure; as shown below. Figure 9 As shown, the electronic device 90 includes: a processor 901, and a memory 902 communicatively connected to the processor 901; the memory 902 stores instructions executable by the processor 901. The instructions are executed by the processor 901 to enable the processor 901 to perform: Obtain the service request sent from the application service layer to the communication layer; If the service request is of the first request type, the service request is forwarded to the control plane processing module in the communication layer for processing. If the service request is of the second request type, the service request is forwarded to the data plane processing module in the communication layer for processing. The request type of the service request is determined based on the data type corresponding to the service request, or the request type of the service request is determined based on the operation type of the target operation corresponding to the service request.

[0100] The electronic devices and corresponding data processing methods provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0101] In practical applications, the electronic device 90 may further include at least one network interface 903. The various components of the electronic device 90 are coupled together via a bus system 904. It is understood that the bus system 904 is used to implement communication between these components. In addition to a data bus, the bus system 904 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 9 All buses are labeled as bus system 904. The number of processors 901 and the number of memories 902 can be at least one. The network interface 903 is used for wired or wireless communication between the electronic device 90 and other devices.

[0102] The memory 902 in this embodiment is used to store various types of data to support the operation of the electronic device 90.

[0103] The methods disclosed in the above embodiments of this disclosure can be applied to or implemented by processor 901. Processor 901 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 901 or by instructions in software form. The processor 901 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 901 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this disclosure can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 902. Processor 901 reads information from memory 902 and, in conjunction with its hardware, completes the steps of the aforementioned data processing method.

[0104] In some embodiments, the electronic device 90 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned methods.

[0105] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0106] In the above description, the term "some embodiments" refers to a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0107] Unless otherwise defined, all technical and scientific terms used in this disclosure have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used in this disclosure is for the purpose of describing embodiments of this disclosure only and is not intended to be limiting of this disclosure.

[0108] It should be understood that in the various embodiments of this disclosure, the sequence number of each implementation process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0109] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0110] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, The method includes: Obtain the service request sent from the application service layer to the communication layer; If the service request is of the first request type, the service request is forwarded to the control plane processing module in the communication layer for processing. If the service request is of the second request type, the service request is forwarded to the data plane processing module in the communication layer for processing. The request type of the service request is determined based on the data type corresponding to the service request, or the request type of the service request is determined based on the operation type of the target operation corresponding to the service request.

2. The method according to claim 1, characterized in that, The request type of the service request is determined based on the data type corresponding to the service request; When the service request is for the first type of data, the data access frequency of the first type of data is lower than the first frequency threshold, and the request type of the service request is the first request type; when the service request is for the second type of data, the data access frequency of the second type of data is greater than the second frequency threshold, and the request type of the service request is the second request type.

3. The method according to claim 1, characterized in that, The request type of the service request is determined based on the operation type of the target operation corresponding to the service request: When the target operation is a read operation, the service request type is a first request type; when the target operation is a write operation, the service request type is a second request type.

4. The method according to any one of claims 1 to 3, characterized in that, If the service request includes a subscription request sent by a first consumer service in the application service layer, the method further includes: The request type of the subscription request is determined to be the first request type; The step of forwarding the service request to the control plane processing module in the communication layer for processing includes: The subscription request is forwarded to the control plane processing module in the communication layer. The subscription request is used by the first consumer service to subscribe to the object data of the first data object. The subscription request includes the object identifier of the first data object and the service identifier of the first consumer service. The subscription manager in the control plane processing module generates subscription relationship information between the first data object and the first consumer service based on the object identifier of the first data object and the service identifier of the first consumer service.

5. The method according to claim 4, characterized in that, Before forwarding the subscription request to the control plane processing module in the communication layer, the method further includes: In response to the startup command of the target system, multiple data objects are created using the data object manager in the control plane processing module. The multiple data objects include the first data object, and each data object indicates a second type of data, wherein the second type of data represents a data access frequency greater than a second frequency threshold.

6. The method according to any one of claims 1 to 3, characterized in that, If the service request includes a write request sent by the producer service in the application service layer, the method further includes: The request type of the write request is determined to be the second request type; The step of forwarding the service request to the data plane processing module in the communication layer for processing includes: The write request is forwarded to the data plane processing module, and the write request includes the data to be written. The data plane processing module is used to write the data to be written into the shared memory bound to the second data object, thereby obtaining the memory region information corresponding to the second data object.

7. The method according to claim 6, characterized in that, After using the data plane processing module to write the data to be written into the shared memory bound to the second data object, and obtaining the memory region information corresponding to the second data object, the method further includes: The subscription relationship information between the second data object and the second consumer service is obtained through the data distribution engine in the data plane processing module. Based on the subscription relationship information between the second data object and the second consumer service, the memory region information corresponding to the second data object is sent to the second consumer service, so that the second consumer service can access the data in the shared memory bound to the second data object based on the memory region information of the second data object.

8. The method according to claim 7, characterized in that, The method further includes: The data distribution engine and the second consumer service are created using the channel manager in the data plane processing module; The step of sending the memory region information corresponding to the second data object to the second consumer service based on the subscription relationship information between the second data object and the second consumer service includes: Based on the subscription relationship information between the second data object and the second consumer service, the memory region information corresponding to the second data object is sent to the second consumer service through the data channel.

9. A data processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire service requests sent from the application service layer to the communication layer. The forwarding module is used to forward the service request to the control plane processing module in the communication layer for processing when the request type of the service request is the first request type. The forwarding module is used to forward the service request to the data plane processing module in the communication layer for processing when the request type of the service request is the second request type. The request type of the service request is determined based on the data type corresponding to the service request, or the request type of the service request is determined based on the operation type of the target operation corresponding to the service request.

10. The apparatus according to claim 9, characterized in that, The request type of the service request is determined based on the data type corresponding to the service request; When the service request is for the first type of data, the data access frequency of the first type of data is lower than the first frequency threshold, and the request type of the service request is the first request type; when the service request is for the second type of data, the data access frequency of the second type of data is greater than the second frequency threshold, and the request type of the service request is the second request type.

11. The apparatus according to claim 9, characterized in that, The request type of the service request is determined based on the operation type of the target operation corresponding to the service request: When the target operation is a read operation, the service request type is a first request type; when the target operation is a write operation, the service request type is a second request type.

12. The apparatus according to any one of claims 9 to 11, characterized in that, If the service request includes a subscription request sent by the first consumer service in the application service layer, the apparatus further includes: The determining module is used to determine that the request type of the subscription request is a first request type; The forwarding module is specifically used to forward the subscription request to the control plane processing module in the communication layer. The subscription request is used by the first consumer service to subscribe to the object data of the first data object. The subscription request includes the object identifier of the first data object and the service identifier of the first consumer service. The generation module is used to generate subscription relationship information between the first data object and the first consumer service based on the object identifier of the first data object and the service identifier of the first consumer service using the subscription manager in the control plane processing module.

13. The apparatus according to any one of claims 9 to 11, characterized in that, If the service request includes a write request sent by the producer service in the application service layer, the apparatus further includes: The determining module is used to determine that the request type of the write request is the second request type; The forwarding module is specifically used to forward the write request to the data plane processing module, and the write request includes the data to be written; The writing module is used to write the data to be written into the shared memory bound to the second data object using the data plane processing module, thereby obtaining the memory area information corresponding to the second data object.

14. An electronic device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the data processing method according to any one of claims 1 to 8.

15. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the data processing method according to any one of claims 1 to 8.

Images