Data processing method and device, electronic equipment and storage medium

Abstract

The invention provides a data processing method and device, electronic equipment and a storage medium, and relates to the technical field of communication. According to the scheme, each application in the first network domain configures the corresponding processing parameter in the cross-network transmission service, and the processing parameter corresponding to the data is identified through the cross-network transmission service, so that the application and the cross-network transmission service can be decoupled, and the specific processing method can be realized by each application in a self-defined manner; therefore, the application only needs to pay attention to the configuration of the processing method of the application and does not need to pay attention to the implementation of data cross-network transmission, when a new processing method needs to be added, only new processing parameters need to be added in the cross-network transmission service, and the expansibility of a cross-network transmission system architecture can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of communication technology, and more specifically, to a data processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] In some application scenarios, cross-network data transmission is often involved. For example, in the practical application of situational awareness and coordination command systems, the platform is often deployed across multiple isolated networks, with one-way data communication between these networks achieved through devices such as one-way gateways / one-way control systems. The platform's business processes, such as issuing and responding to instructions, submitting and receiving plans, are executed by different personnel in different network domains. Therefore, it is necessary to synchronize data between these network domains to form a closed loop for complex business processes, while simultaneously achieving real-time synchronization between PC and mobile devices across different network domains.

[0003] Currently, the cross-network transmission system only supports unidirectional transmission of non-business data, not cross-network transmission of business data. This is because non-business data can be simply transmitted without requiring processing by each network domain, while business data involves specific processing for each application. If the existing cross-network transmission system is used to handle business data transmission, applications in each network domain will still need to pay attention to the cross-network transmission process of business data, resulting in poor scalability of the entire system. Summary of the Invention

[0004] The purpose of this application is to provide a data processing method, apparatus, electronic device, and storage medium to improve the poor scalability of existing methods.

[0005] In a first aspect, embodiments of this application provide a data processing method applied to cross-network transmission services in a first network domain, the method comprising:

[0006] Receive application data transmitted from the second network domain, wherein the application data refers to data transmitted by the application of the second network domain to the application of the first network domain, and which requires the application of the first network domain to perform relevant processing;

[0007] Based on the application-related information of the first network domain carried by the application data, the processing parameters corresponding to the application data are determined, wherein each application in the first network domain is configured with corresponding processing parameters in the cross-network transmission service.

[0008] The application interface corresponding to the processing parameters is invoked, and the application data is transmitted to the corresponding application for processing through the application interface.

[0009] In the above implementation process, each application in the first network domain of this solution configures corresponding processing parameters in the cross-network transmission service. The cross-network transmission service is used to identify the processing parameters corresponding to the data, thus decoupling the application from the cross-network transmission service. The specific processing method can be customized by each application. This allows the application to only focus on the configuration of its own processing method without having to worry about the implementation of cross-network data transmission. When a new processing method needs to be added, only the new processing parameters need to be added to the cross-network transmission service, which can effectively improve the scalability of the cross-network transmission system architecture.

[0010] Optionally, the application-related information includes the processing parameter name corresponding to the application, and determining the processing parameters corresponding to the application data based on the application-related information of the first domain carried by the application data includes:

[0011] Obtain the processing parameter name carried by the application data;

[0012] The processing parameters corresponding to the application data are determined based on the processing parameter name.

[0013] In the above implementation process, the corresponding processing parameters can be quickly found by using the processing parameter names carried by the application data.

[0014] Optionally, the application-related information further includes the application name, the processing parameter name includes the implementation method name, and the step of determining the processing parameters corresponding to the application data based on the processing parameter name includes:

[0015] Obtain the application name corresponding to the application data, and determine the corresponding application based on the application name;

[0016] The implementation class corresponding to the application is determined based on the implementation method name;

[0017] The invocation of the application interface corresponding to the processing parameters includes:

[0018] The implementation method corresponding to the implementation class is obtained through reflection, and the application interface corresponding to the implementation method is called.

[0019] In the above implementation process, the application only needs to expose the corresponding application interface to the cross-network transmission service, and then use the corresponding implementation method to process the application data accordingly. This allows the application to only focus on its own business logic implementation process without having to understand the application data transmission process, which decouples data transmission and data processing. This is beneficial for optimizing system resource allocation and improving system scalability.

[0020] Optionally, the application-related information includes the processing parameter name corresponding to the application, and determining the processing parameters corresponding to the application data based on the application-related information of the first domain carried by the application data includes:

[0021] Obtain the filename of the application data;

[0022] Extract a string of a preset length from a preset position in the filename, and use the extracted string as a processing parameter name;

[0023] The processing parameters corresponding to the application data are determined based on the processing parameter name.

[0024] In the above implementation process, by including the processing parameter name in the file name of the application data, the corresponding processing parameter can be quickly determined without parsing the content of the application data.

[0025] Optionally, determining the processing parameters corresponding to the application data based on the application-related information of the first domain carried in the application data includes:

[0026] The integrity of the application data is verified;

[0027] After successful verification, the processing parameters corresponding to the application data are determined based on the application-related information of the first domain carried in the application data.

[0028] In the above implementation process, the integrity of the application data is verified to determine whether the application data has been transmitted completely, thereby ensuring the accuracy of subsequent determination of processing parameters and the accurate processing of application data.

[0029] Optionally, verifying the integrity of the application data includes:

[0030] The initial file size of the application data is determined from the filename of the application data;

[0031] Verify whether the application data transmission is complete based on the received application data file size and the initial file size;

[0032] If so, then the integrity verification of the application data is confirmed to be successful.

[0033] In the above implementation process, the integrity of application data can be determined directly by the file size, which allows for rapid judgment.

[0034] Optionally, the file format of the application data is a preset file format that the cross-network transmission service of the second network domain converts before transmission;

[0035] And / or, the file content of the application data is the content of a preset format converted by the cross-network transmission service of the second network domain before transmission.

[0036] In the above implementation process, standardizing the file format and file content format for data transmission can improve the universality of cross-network data transmission.

[0037] Optionally, after receiving the application data transmitted from the second network domain, and before determining the processing parameters corresponding to the application data based on the application-related information of the first network domain carried in the application data, the method further includes:

[0038] Obtain sequence information that characterizes the sending order of the application data;

[0039] The application data is stored according to the sequence information in the order of transmission.

[0040] In the above implementation process, by storing application data in the order of transmission, it can be ensured that application data is processed in sequence in some application scenarios.

[0041] Secondly, embodiments of this application provide a data processing apparatus, including a cross-network transmission service in a first network domain, the apparatus comprising:

[0042] The data receiving module is used to receive application data transmitted from the second network domain. The application data refers to data transmitted from the application of the second network domain to the application of the first network domain, and which requires the application of the first network domain to perform relevant processing.

[0043] The processing parameter determination module is used to determine the processing parameters corresponding to the application data based on the application-related information of the first network domain carried by the application data, wherein each application in the first network domain is configured with corresponding processing parameters in the cross-network transmission service.

[0044] The interface call module is used to call the application interface corresponding to the processing parameters, and transmit the application data to the corresponding application for processing through the application interface.

[0045] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the method provided in the first aspect above are performed.

[0046] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method provided in the first aspect above.

[0047] Fifthly, embodiments of this application provide a computer program product, including computer program instructions, which, when read and executed by a processor, perform the steps of the method provided in the first aspect above.

[0048] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0049] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 A flowchart illustrating a data processing method provided in an embodiment of this application;

[0051] Figure 2 This application provides a schematic diagram of the structure of a cross-network transmission system according to an embodiment of the present application.

[0052] Figure 3 This is a schematic diagram illustrating data transmission via cross-network transmission service, provided as an embodiment of this application.

[0053] Figure 4 A structural block diagram of a data processing apparatus provided in an embodiment of this application;

[0054] Figure 5 This is a schematic diagram of the structure of an electronic device for performing a data processing method, provided as an embodiment of this application. Detailed Implementation

[0055] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0056] It should be noted that the terms "system" and "network" in the embodiments of this invention can be used interchangeably. "Multiple" refers to two or more; therefore, in the embodiments of this invention, "multiple" can also be understood as "at least two". "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / ", unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0057] This application provides a data processing method applied to a cross-network transmission service in a first network domain. The method receives application data transmitted from a second network domain through the first network domain's cross-network transmission service. Then, based on the application-related information of the first network domain carried in the application data, it determines the processing parameters corresponding to the application data. Finally, it calls the application interface corresponding to the processing parameters and transmits the application data to the corresponding application for processing through the application interface. In this solution, each application in the first network domain configures corresponding processing parameters in the cross-network transmission service, using the cross-network transmission service to identify the processing parameters corresponding to the data. This decouples the application from the cross-network transmission service, and the specific processing method can be customized by each application. This allows the application to focus only on configuring its own processing method without needing to worry about the implementation of cross-network data transmission. When a new processing method needs to be added, only the new processing parameters need to be added to the cross-network transmission service, effectively improving the scalability of the cross-network transmission system architecture.

[0058] To facilitate understanding of this solution, we will first briefly introduce its application scenarios.

[0059] like Figure 1 As shown, Figure 1 This is a schematic diagram of a cross-network transmission system provided in an embodiment of this application. Taking three isolated networks as an example, data transmission between network domain 1 and network domain 2, and between network domain 1 and network domain 3, can be achieved through two unidirectional gateways. Network domain 2 and network domain 3 are not connected and require network domain 1 as a bridge for data transmission. Each network domain deploys a set of the data processing methods described in this solution.

[0060] When a user performs a page operation (such as sending a message) on application 1 of domain 1, users in domain 2 and domain 3 need to view and operate on application 1 of their respective domain systems (such as signing for receipt or providing feedback). This scenario requires transmitting the information sent by the user in domain 1 to domain 2 and domain 3 via a network gateway, and then storing it in the databases of domain 2 and domain 3 for users within those domains to view. Simultaneously, the operation structures (such as signing for receipt and providing feedback) of users in domain 2 and 3 need to be transmitted back to domain 1 via the network gateway, and some data in domain 2 and 3 need to be synchronized through the path of domain 1 to ensure that the designated business data of the three domains remain consistent.

[0061] This scenario involves cross-network data transmission. Since applications in different network domains process data differently, in order to optimize resource allocation and improve system scalability, this solution configures a cross-network transmission service in each network domain. The cross-network transmission service can be configured on the server in each network domain and is used to perform corresponding processing on the data received and before it is sent. In other words, the cross-network transmission service is used to handle the cross-network transmission of data, while the application itself only needs to focus on its own business processing logic, thereby decoupling the application from the cross-network transmission service.

[0062] Please refer to Figure 2 , Figure 2 A flowchart of a data processing method provided in this application embodiment, the method including the following steps:

[0063] Step S110: Receive application data transmitted from the second network domain.

[0064] The implementation subject of this scheme is the cross-network transmission service of the first network domain. The first network domain and the second network domain can be relative. For example, if network domain 1 transmits data to network domain 2, then network domain 2 can be regarded as the first network domain and network domain 1 can be regarded as the second network domain. If network domain 2 transmits data to network domain 1, then network domain 1 can be regarded as the first network domain and network domain 2 can be regarded as the second network domain.

[0065] Each network domain is configured with a cross-network transmission service. When the second network domain transmits application data to the first network domain, the application data is first processed by the cross-network transmission service of the second network domain, and then the application data is transmitted to the cross-network transmission service of the first network domain through the network gateway.

[0066] The application data here refers to data transmitted from the application in the second network domain to the application in the first network domain, and which requires processing by the application in the first network domain. Application data can be business data, i.e., data requiring some business processing, such as data sent from application 1 in network 1 to application 1 in network 2, which requires application 1 in network 2 to parse the data and provide corresponding information. Of course, application data can also refer to non-business data. The data processing method in this solution is also adaptable to the transmission of non-business data. For example, if the application data is non-business data, each application can configure the corresponding processing parameters in the cross-network transmission service for non-business data.

[0067] Step S120: Determine the processing parameters corresponding to the application data based on the application-related information of the first domain carried in the application data.

[0068] The cross-network transmission system in this solution is a multi-application microservice deployment architecture. Considering the minimal intrusion of cross-network scenarios on various business applications, the cross-network transmission service is specifically responsible for interacting with applications and network gateways.

[0069] Therefore, after receiving application data, the cross-network transmission service performs corresponding preprocessing before sending it to the application. Considering scalability, this solution allows each application in the first network domain to configure corresponding processing parameters in the cross-network transmission service. Understandably, for each network domain, each application within that domain can also configure corresponding processing parameters in its own cross-network transmission service. These processing parameters can be used in the application interface of the corresponding application, and the application data can be passed as input parameters to the application interface for processing.

[0070] In cross-network transmission services, a mapping relationship between various applications and their corresponding processing parameters is stored. In some implementations, the application-related information of the application data may include an application identifier, which indicates the application to which the application data is to be transmitted. Therefore, the cross-network transmission service can find the processing parameters corresponding to the application in the first network domain from the mapping relationship based on the application identifier. For example, if the application-related information includes the application identifier of application 1 in the first network domain, the processing parameters corresponding to application 1 can be found based on the mapping relationship, and the processing parameters of the application data can then be determined.

[0071] Step S130: Call the application interface corresponding to the processing parameters and transmit the application data to the corresponding application for processing through the application interface.

[0072] After determining the processing parameters corresponding to the application data, the cross-network transmission service can call the application interface corresponding to those parameters. Understandably, the processing parameters are bound to the application interface, so the cross-network transmission service can directly use the application data as input parameters to the application interface, passing it to the corresponding application for subsequent processing. The application's processing of the application data can be determined according to actual business needs, such as feeding back relevant data or saving the application data to a corresponding database. In this case, the processing methods of each application can be flexibly customized. The cross-network transmission service only needs to maintain the processing parameters of each application, thus eliminating the need for each application to configure a separate data preprocessing method. Each application only needs to focus on its own business logic implementation, reducing the complexity of application implementation. This decouples the application from the cross-network transmission service, improving the scalability of the entire system.

[0073] In the above implementation process, each application in the first network domain of this solution configures corresponding processing parameters in the cross-network transmission service. The cross-network transmission service is used to identify the processing parameters corresponding to the data, thus decoupling the application from the cross-network transmission service. The specific processing method can be customized by each application. This allows the application to only focus on the configuration of its own processing method without having to worry about the implementation of cross-network data transmission. When a new processing method needs to be added, only the new processing parameters need to be added to the cross-network transmission service, which can effectively improve the scalability of the cross-network transmission system architecture.

[0074] Based on the above embodiments, when determining the processing parameters corresponding to the application data, the application-related information may include the processing parameter name corresponding to the application. Then, the processing parameter name carried by the application data can be obtained, and the processing parameters corresponding to the application data can be determined according to the processing parameter name.

[0075] Understandably, before transmitting application data, the second network domain can encapsulate the processing parameter names within the application data. This way, after the first network domain's cross-network transmission service obtains the application data, it can retrieve the corresponding processing parameter names from the application data. Different processing parameters can correspond to different processing parameter names, so the corresponding processing parameter can be determined directly based on the processing parameter name.

[0076] In the above implementation process, the corresponding processing parameters can be quickly found by using the processing parameter names carried by the application data.

[0077] Based on the above embodiments, the processing parameter name can be carried in the file name of the application data. When determining the corresponding processing parameter, the file name of the application data can be obtained, and then a string of a preset length can be extracted at a preset position in the file name. The extracted string is used as the processing parameter name, and then the processing parameter corresponding to the application data can be determined according to the processing parameter name.

[0078] Each network domain can pre-determine the format of the filenames for application data transmitted across networks, such as how the filenames are generated and what content they contain. When generating the filenames, the position of the processing parameter names in the filenames and the length of the strings they occupy can also be agreed upon. In this way, after the cross-network transmission service of the first network domain receives the application data transmitted by the second network domain, it can process and extract the string of the preset length from the preset position of its filename, which can then be used as the processing parameter name.

[0079] In the above implementation process, by including the processing parameter name in the file name of the application data, the corresponding processing parameter can be quickly determined without parsing the content of the application data.

[0080] Based on the above embodiments, in order to facilitate accurate identification of processing parameters, the application-related information may also include the application name, and the processing parameter name may include the implementation method name. When determining the processing parameters, the application name corresponding to the application data can be obtained, and the corresponding application can be determined according to the application name. Then, the implementation class corresponding to the application can be determined according to the implementation method name. When calling the corresponding application interface, the implementation method corresponding to the implementation class can be obtained through reflection mechanism, and the application interface corresponding to the implementation method can be called.

[0081] For example, when designing the filename for application data, information such as data type, destination network domain, application name (which can be an application identifier), and parsing method can be included in the filename. The corresponding processing parameters can be found based on this information. Specific specifications are as follows:

[0082] data_order_size_dest_mod_method_uuid.txt;

[0083] Note: Prefix (file type)_Sequence number (default current timestamp)_File size_Destination domain (net1 / net2 / net3)_Application name_Implementation method name_Unique identifier uuid.txt

[0084] The "data prefix" is used to identify the type of application data, which is divided into two types: data files and attachment files.

[0085] The "order number" indicates the transmission order of application data. It follows the FIFO (First In First Out) principle and defaults to the 13-bit timestamp of the current time when the application data was generated, ensuring the order of filenames.

[0086] "size file size" is used to represent the physical size of the current application data, in bytes. During the network gateway transfer process, the data consumer uses it to check whether the current data has been transferred completely, so as to ensure that the consumer processes the transferred file and ensures data integrity.

[0087] "dest destination domain" indicates the destination domain to which the application data needs to be transmitted. When processing on the consumer side, the subsequent processing flow will be identified based on the destination domain, such as: net1 / net2 / net3, etc.

[0088] "mod module name" and "method implementation method name" are the mappings and implementation method names of the Java classes registered by each application, which are used as identifiers to indicate the Java methods that this application data needs to execute on the consumer side;

[0089] The "unique identifier UUID" is a randomly generated UUID value to ensure that filenames are not duplicated. It is automatically generated by the system when generating application data.

[0090] For example, the filename of an application's data is as follows:

[0091] data_1701828515669_98_net3_activity_addReport_f35c5d9c9c5a4caf9dba4f03223b6575.txt.

[0092] Here, "mod" represents the application name, and "method" represents the implementation method name. For example, in the filename example above, "activity" represents the application name, and "addReport" represents the implementation method name. When determining processing parameters, the application name can be used to identify the corresponding application, and then the implementation method name can be used to find the corresponding implementation class of that application. The cross-network transmission service stores the mapping between application names and implementation method names. Different applications may have different implementation methods. The cross-network transmission service also stores the mapping between implementation method names and implementation classes. This allows the corresponding implementation class to be found. Then, the cross-network transmission service can use Java's reflection mechanism to obtain the implementation method corresponding to the implementation class, thereby calling the application interface corresponding to the implementation method.

[0093] Java's reflection mechanism allows, at runtime, the ability to know all the attributes and methods of any class, and to invoke any method and attribute of any object. Therefore, in cross-network transmission services, reflection can be used to obtain the implementation methods corresponding to the implementing class.

[0094] Each application can pre-register its implementation class with the cross-network transmission service. For example, each application's code project in the cross-network transmission service follows a mapping between custom mods and implementation classes. For instance, in the code, the value of `code` is the mod identifier, consistent with the mod in the filename, and `className` is the application's data processing class path. When the cross-network transmission service parses application data, it extracts the mod and method from the filename. The mod identifies the `className`, and the `method` identifies the corresponding implementation method, i.e., the processing function. Then, reflection can be used to call the implementation method in the implementation class using the application data as input parameters. This allows invoking the application interface corresponding to the implementation method. Different implementation methods correspond to different application interfaces, so application data can be transferred to the application for processing by calling the application interface.

[0095] In this solution, application data can be transmitted by calling the application's RESTful API interface. The cross-network transmission service parses the application data and calls the application's method, which in turn calls the corresponding API interface of the application via HTTPS, passing the application data as a request parameter. The application then performs the appropriate processing. Therefore, the content of the application data to be transmitted across the network and the processing method for transmission to the destination network domain can be customized by each application. This decouples the application from the cross-network transmission service, allowing each application to focus on data processing without worrying about data transmission, resulting in greater scalability.

[0096] Each application registers its corresponding implementation class and methods with the cross-network transmission service through a registration mechanism. This mechanism allows the system to introduce new applications without modifying existing code. When new business processes need to be added, only the new implementation class and methods need to be registered with the cross-network transmission service, without requiring large-scale changes to the existing system. Furthermore, each application can customize the registered implementation classes and methods, meaning that each application can implement specific data processing logic according to its own business needs, thus adapting to various business scenarios and changing requirements.

[0097] In some implementations, the cross-network transport service can control access to application interfaces registered by various applications, such as allowing only authorized applications to access and process data. In specific implementations, the cross-network transport service can also configure permissions for each application interface. Upon determining the corresponding application interface, it can be checked whether its permissions satisfy the need to process application data (different processing may correspond to different permissions). If so, the application interface is invoked to transmit the application data to the application for processing; otherwise, the transmission of application data to the application for processing is refused.

[0098] In the above implementation process, the application only needs to expose the corresponding application interface to the cross-network transmission service, and then use the corresponding implementation method to process the application data accordingly. This allows the application to only focus on its own business logic implementation process without having to understand the application data transmission process, which decouples data transmission and data processing. This is beneficial for optimizing system resource allocation and improving system scalability.

[0099] Based on the above embodiments, in order to ensure that the cross-network transmission service of the first network domain receiving the application data can know whether the application data has been transmitted completely, the integrity of the application data can be verified first. After the verification is passed, the processing parameters corresponding to the application data can be determined according to the application-related information of the first network domain carried by the application data.

[0100] For example, before transmitting application data, the cross-network transmission service of the second network domain can generate a hash value for the application data content and store it in the application data, or include it in the application data filename. When the cross-network transmission service of the first network domain receives the application data, it can obtain the hash value by parsing the application data content or filename. Then, it generates a new hash value using the received application data content and compares the two hash values. If they match, the application data transmission is considered complete, the integrity verification is passed, and subsequent processing can proceed. If the two hash values ​​do not match, the application data transmission is considered incomplete or has suffered losses. In this case, the cross-network transmission service of the first network domain can continue to wait for the application data transmission and, after a certain period, continue to perform integrity verification in the same manner. If the verification still fails, it can send a feedback message to the second network domain to inform it that the application data transmission is incorrect. Upon receiving this feedback message, the cross-network transmission service of the second network domain can retransmit the application data.

[0101] In the above implementation process, the integrity of the application data is verified to determine whether the application data has been transmitted completely, thereby ensuring the accuracy of subsequent determination of processing parameters and the accurate processing of application data.

[0102] Based on the above embodiments, in the method of verifying the integrity of application data, the initial file size of the application data can be determined from the file name of the application data. Then, based on the file size of the received application data and the initial file size, it can be verified whether the application data has been transmitted completely. If so, the integrity verification of the application data is determined to be successful.

[0103] For example, before sending application data, the cross-network transmission service of the second network domain encapsulates the file size of the application data into the file name of the application data. In the example above, "98" in the file name represents the initial file size. After receiving the application data, the cross-network transmission service of the first network domain can extract the initial file size from the file name and compare it with the file size of the currently received application data. If the two file sizes are the same, it means that the application data transmission is complete and the integrity verification is successful. Conversely, if the two file sizes are different, it means that the application data integrity verification fails. At this time, the cross-network transmission service of the first network domain can send a feedback message to the second network domain to inform that the application data transmission is incorrect. After receiving the feedback message, the cross-network transmission service of the second network domain can retransmit the application data.

[0104] In the above implementation process, the integrity of application data can be determined directly by the file size, which allows for rapid judgment.

[0105] Based on the above embodiments, in order to cope with the cross-network transmission scenario of complex heterogeneous business data, the format of the relevant information of the application data can be unified. For example, the file format of the application data is a preset file format converted by the cross-network transmission service of the second network domain before transmission, and / or, the file content of the application data is the content of the preset format converted by the cross-network transmission service of the second network domain before transmission.

[0106] The preset file format can be a txt text format, or other formats such as JSON. This can be flexibly configured according to actual needs. For example, the preset file format can be pre-configured in the cross-network transmission service of each domain. In this way, the cross-network transmission service can convert the application data into the corresponding preset file format before sending the application data, and the application itself does not need to care about the data format conversion.

[0107] The preset format content can be in string format as text content, which facilitates the parsing of application data by various cross-network transmission services. Of course, other preset formats can also be flexibly configured according to actual needs, such as binary format, so as to solve the parsing problem of heterogeneous business data of various applications in conjunction with the registration classes and methods of various applications.

[0108] The cross-network transfer service (transfer) provides a general file-to-disk interface for applications to call. For example, when an application needs to transfer data across networks, it can call the cross-network transfer service's file-to-disk interface to write the application data to be transferred into the corresponding preset file format, and then transfer it to the local network gateway. Figure 3 As shown.

[0109] In the above implementation process, standardizing the file format and file content format for data transmission can improve the universality of cross-network data transmission.

[0110] Based on the above embodiments, in order to ensure the sequentiality of data transmission, relevant information for characterizing its transmission order can also be carried in the application data. Therefore, after the cross-network transmission service of the first network domain receives the application data, it can obtain the sequence information characterizing the sending order of the application data, and then store the application data according to the transmission order based on the sequence information.

[0111] The sequence information representing the transmission order of application data can be the timestamp of the application data, which can be carried in the filename of the application data. In the example filename above, "1701828515669" represents the timestamp. The application data can be sorted according to the timestamp. Since the timestamp is the second character in the filename (after the file type), sorting by filename ensures that it is sorted according to the transmission order of the application data, thus ensuring the FIFO order.

[0112] When retrieving and processing application data from the first domain, a single-threaded polling method can be used to process the data according to its sorted order. This ensures that the data is processed in the order it was generated, guaranteeing its sequential nature.

[0113] Of course, in other implementations, the sequence information representing the sending order of application data can also be other information, such as a sequence number. This sequence number can be generated by the cross-network transmission service of the second network domain before sending the application data and then carried into the application data, or it can be carried in the file name of the application data. In this way, the transmission order of the application data can also be determined by identifying the sequence number.

[0114] In the above implementation process, by storing application data in the order of transmission, it can be ensured that application data is processed in sequence in some application scenarios.

[0115] In this application's solution, the cross-network transmission service of the first network domain can initiate periodic or real-time scheduling tasks to monitor application data transmitted from the second network domain's gateway to this side's network domain, and extract and process the application data sequentially. During processing, the cross-network transmission service first determines the integrity of the application data based on the size in the filename, reads the complete application data, parses the data content, and, based on the mod and method in the filename, reflectsively calls the corresponding implementation methods of the Java implementation class registered by the application for appropriate processing. This solves the problem of synchronizing complex heterogeneous business data transmission. Furthermore, the overall architecture of this solution adopts a microservice approach, decoupling the cross-network transmission service from the application, which effectively improves the overall architecture's scalability.

[0116] Based on the above embodiments, in cross-network transmission scenarios, the transmission of both business data and non-business data is involved. Non-business data, such as traffic logs, alarm logs, system logs, etc., is generally transmitted transparently and does not need to go through the data processing logic described above. Of course, to improve the system's versatility, the above processing logic can also be applied to non-business data, including the processing of file format and file content format. The only difference is that the processing parameters determined for non-business data and business data may differ. The processing parameters determined for non-business data are used to instruct the non-business data to be synchronized into the database.

[0117] Therefore, to facilitate the distinction between business data and non-business data, information such as data type can be included in the filename of the application data. In this way, the cross-network transmission service can identify whether the application data is business data or non-business data based on the data type. For non-business data, the cross-network transmission service can customize the corresponding processing logic. For business data, the cross-network transmission service can process it according to the processing logic described in the above embodiments.

[0118] Please refer to Figure 4 , Figure 4 This is a structural block diagram of a data processing apparatus 200 provided in an embodiment of this application. The apparatus 200 runs a cross-network transmission service in a first network domain. The apparatus 200 can be a module, program segment, or code on an electronic device, where the electronic device can refer to a server in the first network domain. It should be understood that this apparatus 200 is related to the above-described... Figure 1 The method implementation corresponds to this and can be executed. Figure 1 The various steps involved in the method embodiment and the specific functions of the device 200 can be found in the description above. To avoid repetition, detailed descriptions are omitted here.

[0119] Optionally, the device 200 includes:

[0120] Data receiving module 210 is used to receive application data transmitted from the second network domain. The application data refers to data transmitted from the application of the second network domain to the application of the first network domain, and which requires the application of the first network domain to perform relevant processing.

[0121] The processing parameter determination module 220 is used to determine the processing parameters corresponding to the application data based on the application-related information of the first network domain carried by the application data, wherein each application in the first network domain is configured with corresponding processing parameters in the cross-network transmission service.

[0122] The interface call module 230 is used to call the application interface corresponding to the processing parameters and transmit the application data to the corresponding application for processing through the application interface.

[0123] Optionally, the application-related information includes the processing parameter name corresponding to the application, and the processing parameter determination module 220 is used to obtain the processing parameter name carried by the application data; and determine the processing parameter corresponding to the application data according to the processing parameter name.

[0124] Optionally, the application-related information also includes the application name, the processing parameter name includes the implementation method name, the processing parameter determination module 220 is used to obtain the application name corresponding to the application data, and determine the corresponding application based on the application name; determine the implementation class corresponding to the application based on the implementation method name; the interface call module 230 is used to obtain the implementation method corresponding to the implementation class through reflection mechanism, and call the application interface corresponding to the implementation method.

[0125] Optionally, the application-related information includes the processing parameter name corresponding to the application. The processing parameter determination module 220 is used to obtain the file name of the application data; extract a string of a preset length at a preset position in the file name, and use the extracted string as the processing parameter name; and determine the processing parameters corresponding to the application data based on the processing parameter name.

[0126] Optionally, the processing parameter determination module 220 is used to verify the integrity of the application data; after the verification is passed, it determines the processing parameters corresponding to the application data based on the application-related information of the first network domain carried by the application data.

[0127] Optionally, the processing parameter determination module 220 is used to determine the initial file size of the application data from the file name of the application data; verify whether the application data has been transmitted completely based on the file size of the received application data and the initial file size; if so, determine that the integrity verification of the application data has passed.

[0128] Optionally, the file format of the application data is a preset file format that the cross-network transmission service of the second network domain converts before transmission;

[0129] And / or, the file content of the application data is the content of a preset format converted by the cross-network transmission service of the second network domain before transmission.

[0130] Optionally, the device 200 further includes:

[0131] The storage module is used to acquire sequence information representing the sending order of the application data; and to store the application data in the order of transmission according to the sequence information.

[0132] It should be noted that those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0133] Please refer to Figure 5 , Figure 5This is a schematic diagram of an electronic device for executing a data processing method, provided in an embodiment of this application. The electronic device may include: at least one processor 310, such as a CPU; at least one communication interface 320; at least one memory 330; and at least one communication bus 340. The communication bus 340 is used to establish communication between these components. In this embodiment, the communication interface 320 is used for signaling or data communication with other node devices. The memory 330 may be a high-speed RAM or a non-volatile memory, such as at least one disk storage device. Optionally, the memory 330 may also be at least one storage device located remotely from the aforementioned processor. The memory 330 stores computer-readable instructions. When these computer-readable instructions are executed by the processor 310, the electronic device performs the aforementioned... Figure 1 The method and process are shown.

[0134] Understandable. Figure 5 The structure shown is for illustrative purposes only; the electronic device may also include components that are more advanced than those shown. Figure 5 The more or fewer components shown, or having the same Figure 5 The different configurations shown. Figure 5 The components shown can be implemented using hardware, software, or a combination thereof.

[0135] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, performs the following... Figure 1 The method process executed by the electronic device in the illustrated method embodiment.

[0136] This embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can perform the methods provided in the above-described method embodiments, such as including:

[0137] Receive application data transmitted from the second network domain, wherein the application data refers to data transmitted by the application of the second network domain to the application of the first network domain, and which requires the application of the first network domain to perform relevant processing;

[0138] Based on the application-related information of the first network domain carried by the application data, the processing parameters corresponding to the application data are determined, wherein each application in the first network domain is configured with corresponding processing parameters in the cross-network transmission service.

[0139] The application interface corresponding to the processing parameters is invoked, and the application data is transmitted to the corresponding application for processing through the application interface.

[0140] In summary, the embodiments of this application provide a data processing method, apparatus, electronic device, and storage medium. In this solution, each application in the first network domain configures corresponding processing parameters in the cross-network transmission service. The cross-network transmission service is used to identify the processing parameters corresponding to the data, thus decoupling the application from the cross-network transmission service. The specific processing method can be customized by each application. This allows the application to focus only on the configuration of its own processing method without needing to worry about the implementation of cross-network data transmission. When a new processing method needs to be added, only the new processing parameters need to be added to the cross-network transmission service, which can effectively improve the scalability of the cross-network transmission system architecture.

[0141] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0142] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0143] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0144] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0145] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A data processing method, characterized in that, A cross-network transmission service applied to a first network domain, the method comprising: Receive application data transmitted from the second network domain, wherein the application data refers to data transmitted by the application of the second network domain to the application of the first network domain, and which requires the application of the first network domain to perform relevant processing; Based on the application-related information of the first network domain carried by the application data, the processing parameters corresponding to the application data are determined, wherein each application in the first network domain is configured with corresponding processing parameters in the cross-network transmission service. The application interface corresponding to the processing parameters is invoked, and the application data is transmitted to the corresponding application for processing through the application interface.

2. The method according to claim 1, characterized in that, The application-related information includes the processing parameter name corresponding to the application. Determining the processing parameters corresponding to the application data based on the application-related information of the first domain carried by the application data includes: Obtain the processing parameter name carried by the application data; The processing parameters corresponding to the application data are determined based on the processing parameter name.

3. The method according to claim 2, characterized in that, The application-related information also includes the application name, and the processing parameter name includes the implementation method name. Determining the processing parameters corresponding to the application data based on the processing parameter name includes: Obtain the application name corresponding to the application data, and determine the corresponding application based on the application name; The implementation class corresponding to the application is determined based on the implementation method name; The invocation of the application interface corresponding to the processing parameters includes: The implementation method corresponding to the implementation class is obtained through reflection, and the application interface corresponding to the implementation method is called.

4. The method according to claim 1, characterized in that, The application-related information includes the processing parameter name corresponding to the application. Determining the processing parameters corresponding to the application data based on the application-related information of the first domain carried by the application data includes: Obtain the filename of the application data; Extract a string of a preset length from a preset position in the filename, and use the extracted string as a processing parameter name; The processing parameters corresponding to the application data are determined based on the processing parameter name.

5. The method according to claim 1, characterized in that, The step of determining the processing parameters corresponding to the application data based on the application-related information of the first domain carried in the application data includes: The integrity of the application data is verified; After successful verification, the processing parameters corresponding to the application data are determined based on the application-related information of the first domain carried in the application data.

6. The method according to claim 5, characterized in that, The verification of the integrity of the application data includes: The initial file size of the application data is determined from the filename of the application data; Verify whether the application data transmission is complete based on the received application data file size and the initial file size; If so, then the integrity verification of the application data is confirmed to be successful.

7. The method according to any one of claims 1-6, characterized in that, The file format of the application data is a preset file format converted by the cross-network transmission service of the second network domain before transmission; And / or, the file content of the application data is the content of a preset format converted by the cross-network transmission service of the second network domain before transmission.

8. The method according to any one of claims 1-6, characterized in that, After receiving the application data transmitted from the second network domain, and before determining the processing parameters corresponding to the application data based on the application-related information of the first network domain carried by the application data, the method further includes: Obtain sequence information that characterizes the sending order of the application data; The application data is stored according to the sequence information in the order of transmission.

9. A data processing apparatus, characterized in that, The device includes cross-network transmission services in the first network domain: The data receiving module is used to receive application data transmitted from the second network domain. The application data refers to data transmitted from the application of the second network domain to the application of the first network domain, and which requires the application of the first network domain to perform relevant processing. The processing parameter determination module is used to determine the processing parameters corresponding to the application data based on the application-related information of the first network domain carried by the application data, wherein each application in the first network domain is configured with corresponding processing parameters in the cross-network transmission service. The interface call module is used to call the application interface corresponding to the processing parameters, and transmit the application data to the corresponding application for processing through the application interface.

10. An electronic device, characterized in that, It includes a processor and a memory, the memory storing computer-readable instructions that, when executed by the processor, perform the method as described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it performs the method as described in any one of claims 1-8.

12. A computer program product, characterized in that, It includes computer program instructions, which, when read and executed by a processor, perform the method as described in any one of claims 1-8.

Images