Financial information reporting method, device, computer equipment and readable storage medium

Abstract

The present application discloses a financial information reporting method, apparatus, computer device, and readable storage medium. By constructing a long-connection data transmission channel and a heartbeat service mechanism based on the MCWebSocket protocol, dynamic, low-latency, and high-reliability data transmission is achieved. The method comprises: a server initiates a process for creating a long-connection data transmission channel, calls the MCWebSocket communication protocol to construct the long-connection data transmission channel, and establishes a heartbeat service mechanism to detect the status of the long-connection data transmission channel; the server obtains data to be reported, divides the data to be reported into message queues according to data type, and, based on a streaming computing engine, converts static structured messages in the message queue into dynamic data streams to be reported; an API service encrypts the data stream to be reported and transmits the data stream to be reported to the client via a dedicated channel and / or a shared channel.

Description

Technical Field

[0001] The present application relates to the fields of computer technology and financial technology, and in particular to a method, apparatus, computer equipment and readable storage medium for reporting financial information. Background Art

[0002] In related technologies, financial information is often transmitted using non-real-time protocols such as HTTP / 1.1 and FTP, with batch data updates performed at fixed intervals (e.g., daily or hourly). However, these fixed-interval batch data updates result in poor timeliness of financial information, making it difficult to meet the ever-changing demands of the financial market. Therefore, a reporting device and method that can significantly improve the timeliness of financial information is urgently needed. Summary of the Invention

[0003] In view of this, the present application provides a financial information reporting method, apparatus, computer equipment and readable storage medium, the main purpose of which is to solve the problem that batch data updates at fixed time intervals will lead to poor timeliness of financial information.

[0004] According to the first aspect of the present application, a method for reporting financial information is provided, the method comprising:

[0005] The API service responds to the data transmission request and forwards the data transmission request to the server;

[0006] The server initiates a process for creating a long connection data transmission channel so that the API service notifies the client to create a data receiving node, and upon receiving the node number returned by the client, the server calls the MCWebSocket communication protocol for the node number to build a long connection data transmission channel, and establishes a heartbeat service mechanism to detect the status of the long connection data transmission channel, wherein the long connection data transmission channel includes an exclusive channel and a shared channel, the exclusive channel is used to transmit metadata, and the shared channel is used to transmit business data, task data and process data;

[0007] The server obtains the data to be reported based on the data transmission request, standardizes the data to be reported into structured messages, and divides the data to be reported into Kafka message queues according to data types. Based on the stream computing engine Apache Flink, the server converts the static structured messages in the Kafka message queue into a dynamic data stream to be reported, and transmits the data stream to the API service.

[0008] The API service encrypts the data stream to be reported, and when a metadata stream exists in the data stream to be reported, transmits the encrypted metadata stream to the client through the exclusive channel, and receives the data reception progress synchronized by the client; when the data reception progress indicates that the client has completed data reception, transmits the other data streams to be reported except the encrypted metadata stream to the client through the shared channel, and receives the data reception progress synchronized by the client;

[0009] The server and the API service collect process data in the financial information reporting process and transmit it to the management end for visual display.

[0010] According to a second aspect of the present application, there is provided a financial information reporting device, the device comprising: an API service, a server, and a management terminal;

[0011] The API service, in response to a data transmission request, forwards the data transmission request to the server, and upon detecting that the server has initiated a process for creating a persistent connection data transmission channel, notifies the client to create a data receiving node, and encrypts the data stream to be reported. When a metadata stream exists in the data stream to be reported, the encrypted metadata stream is transmitted to the client via the exclusive channel, and the client receives a synchronized data reception progress. When the data reception progress indicates that the client has completed data reception, the remaining data streams to be reported, excluding the encrypted metadata stream, are transmitted to the client via the shared channel, and the client receives a synchronized data reception progress. Furthermore, the collected process data in the financial information reporting process is transmitted to the management end for visual display.

[0012] The server initiates a long connection data transmission channel creation process, and upon receiving the node number returned by the client, calls the MCWebSocket communication protocol for the node number to build a long connection data transmission channel, and establishes a heartbeat service mechanism to detect the status of the long connection data transmission channel, and obtains the data to be reported based on the data transmission request, standardizes the data to be reported into structured messages, and divides the data to be reported into Kafka message queues according to data types, and converts the static structured messages in the Kafka message queue into dynamic data streams to be reported based on the streaming computing engine Apache Flink, passes the data streams to be reported to the API service, and collects process data in the financial information reporting process and transmits it to the management end for visual display, wherein the long connection data transmission channel includes an exclusive channel and a shared channel, the exclusive channel is used to transmit metadata, and the shared channel is used to transmit business data, task data and process data.

[0013] According to a third aspect of the present application, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0014] According to a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the methods in the first aspect are implemented.

[0015] By means of the above technical solution, the present application provides a method, apparatus, computer equipment and readable storage medium for reporting financial information. In the embodiment of the present application, by constructing a long-connection data transmission channel and a heartbeat service mechanism based on the MCWebSocket protocol, dynamic, low-latency and high-reliability data transmission is achieved. Moreover, through the intelligent division strategy of exclusive channels and shared channels, metadata and business data are classified and transmitted, which improves the utilization rate of channel resources while reducing the risk of network congestion caused by mixed data transmission. And through the collaborative processing of the Apache Flink streaming computing engine and the Kafka message queue, real-time streaming conversion of structured messages is achieved, thereby improving data flow efficiency. In addition, the embodiment of the present application also improves the security protection capability of financial data transmission through multi-level encrypted transmission and progress synchronization mechanism, and eliminates the process black box through the visual monitoring system of the management end, and realizes visual tracking of the transmission status of the entire link.

[0016] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0018] Figure 1 A schematic diagram of a financial information reporting method provided in an embodiment of the present application is shown;

[0019] Figure 2 A schematic diagram of the structure of a financial information reporting device provided in an embodiment of the present application is shown;

[0020] Figure 3 A schematic diagram of a financial information reporting method provided in an embodiment of the present application is shown;

[0021] Figure 4 A schematic diagram showing module interactions of a financial information reporting method provided in an embodiment of the present application is shown;

[0022] Figure 5 A schematic diagram of the device structure of a computer device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present application.

[0024] Those skilled in the art will understand that, unless otherwise stated, the singular forms "a," "an," "said," and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of this application refers to the presence of the stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0025] Those skilled in the art will understand that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and, unless specifically defined as such, will not be interpreted in an idealized or overly formal sense.

[0026] Those skilled in the art will appreciate that the term "terminal" as used herein includes both devices that are wireless signal receivers, i.e., devices that only have wireless signal receivers without transmission capabilities, and devices that have receiving and transmitting hardware capable of performing two-way communication over a two-way communication link. Such devices may include: cellular or other communication devices with single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service) devices that may combine voice, data processing, fax, and / or data communication capabilities; PDAs (Personal Digital Assistants) that may include a radio frequency receiver, a pager, Internet / Intranet access, a web browser, a notepad, a calendar, and / or a GPS (Global Positioning System) receiver; and conventional laptop and / or palmtop computers or other devices that have and / or include a radio frequency receiver. As used herein, a "terminal" can be portable, transportable, installed in a vehicle (air, sea, and / or land), or adapted and / or configured to operate locally and / or in a distributed manner at any other location on Earth and / or in space. A "terminal" as used herein can also refer to a communication terminal, an Internet access terminal, or a music / video playback terminal, such as a PDA, a mobile internet device (MID), and / or a mobile phone with music / video playback capabilities, as well as devices such as smart televisions and set-top boxes.

[0027] like Figure 1 As shown, an embodiment of the present application provides a financial information reporting method. Furthermore, as a specific implementation of the financial information reporting method, an embodiment of the present application also provides a financial information reporting device. The carrier of the financial information reporting method, namely the financial information reporting device, is first introduced below.

[0028] The financial information reporting device includes three parts: API service, server and management. Among them, the server can build a long connection data transmission channel for real-time transmission of financial information. In the actual operation process, the sensitive real-time data of the financial institution is first extracted, processed, assembled, and encrypted through the server, and then the sensitive real-time data is transmitted to the client's local business application via the long connection data transmission channel through the API service. Provide visual full-process management and control capabilities based on the management end. In the specific design, the embodiment of the present application is based on message queues, stream computing, WebSocket and other technologies to solve the problem that financial information data service providers cannot provide real-time data security and efficient interface services, so that users can quickly and accurately obtain the latest financial data information and realize data connectivity and sharing.

[0029] For example, the functional architecture of the financial information reporting device is as follows: Figure 2 As shown in the figure, the server side consists of a data source control module, a data control module, a real-time data module, a data assembly module, and a permission control module. The API service consists of an identity security module, a data receiving module, a running data collection module, and a coding language plug-in.

[0030] The data source management module on the server side is designed in a layered manner, consisting of metadata services and data source services. The metadata service provides attribute information management for data sources, enabling resource identification, dynamic evaluation, and change tracking of networked data sources, fulfilling the requirements of managing large amounts of networked data source data. The data source service responds to data transmission requests to obtain the corresponding raw data, and internally performs internal governance operations such as data source registration, updates, and status monitoring. By decoupling the metadata service from the data source service and adopting a two-tier architecture that separates internal and external services, the flexibility and efficiency of large-scale data source management are guaranteed.

[0031] The data control module on the server side provides a calling unit, a service control unit, a streaming control unit, and an asset permission control unit. Among them, the calling unit receives external data requests and internal scheduling instructions, and forwards metadata and original data to the service control unit. On the one hand, the service control unit relies on metadata and original data to synchronize historical jobs, manage job status, manage logs, and allocate tasks. On the other hand, it calls the data assembly module to perform data processing and relies on the streaming control unit for real-time data flow. The streaming control unit receives streaming data transmitted by the data assembly module and dynamically controls the transmission status, transmission points, link scheduling, and traffic of the real-time data flow. The asset permission control unit provides unified permission management and security control capabilities for the service control unit and API. Based on task splitting, streaming processing, and refined log and permission management, the data control module improves the ability to respond to customized needs and solves the problem of functional operation complexity.

[0032] The server-side real-time data module provides the underlying support for data storage and real-time data operations within financial institutions. It supports the integrated management of heterogeneous data sources such as Oracle, MySQL, Hive, DB2, and external APIs. It also distributes data and transmits messages based on message queues such as Kafka. The Kafka message queue connects data sources and real-time data processing and storage devices such as Flink and Hudi. Real-time data processing and storage devices provide real-time data processing to ensure that data quality and format meet expectations. Specifically, Flink obtains raw data from the Kafka message queue, performs data processing, format standardization, and dynamic stream conversion, and then transmits the processed data back to the Kafka message queue, transforming static data source information into dynamic, real-time data stream information.

[0033] The server-side data assembly module consists of a task service unit, a task executor, a task generation unit, a task scheduling unit, and a result output unit. The task service unit receives service requests from the calling unit in the data control module, promptly reports heartbeat information to the service control unit, and sends task calls to the task executor. The task executor initiates specific task calls to the task scheduling unit. The task scheduling unit manages task execution through the task scheduler and provides services for task generation, task submission, task status updates, and task metric updates. Task generation relies on the relevant services of the task generation unit, which implements data streaming by invoking the execution engine services in the real-time data module. The task generation unit includes a task job generator and task job context management, providing more refined and accurate task management. The result output module assembles result data into streaming data packets. After undergoing streaming control in the data control module, the calling unit communicates with the external world. Through the full process management of task generation, scheduling, and execution, customer operations are simplified, operational efficiency is enhanced, and a technical foundation is provided for data asset management.

[0034] The server-side permission control module serves as the security backbone of the entire solution, providing security control over access requests and output content. It offers password, verification code, and certificate authentication for access requests. It also supports dynamic encryption of access links using dynamic tokens. It also provides message encryption for transmission links and data file encryption for transmitted content. It also offers frame extraction verification, which can sensitively identify eavesdropping or message interception and forgery on outgoing links. The permission control module effectively enhances system security through multiple security mechanisms, such as dynamic encryption and frame extraction auditing.

[0035] For example, identity security for API services provides four key services: interface login, encrypted transmission, credential refresh, and certificate management. These services work in conjunction with the server-side permission management module to ensure API service security, building an end-to-end security protection system that addresses interface exposure risks and insufficient client-side security protection capabilities, laying a trusted foundation for subsequent data transmission.

[0036] The API service's data receiving module is a key module for effectively delivering real-time data to clients. It provides services for data stream reception, data stream verification, data queue sorting, and data landing. After performing security analysis, the data receiving module retrieves data from the Kafka message queue after message structure conversion. It then performs processes such as information acquisition, block transmission, automatic parsing, queue management, and data landing on the received data stream. It then transmits the data to the client in real time via the MCWebSocket protocol, ensuring real-time and immediacy.

[0037] The API service's coding language plug-in dynamically identifies development languages such as Java, Python, Go, and C# based on the Accept-Language or User-Agent fields in the client's request header. It then loads the corresponding language SDK from the plug-in library, such as the Java JAR package or Python Wheel package, to generate a standardized API interface. This eliminates client integration costs caused by language differences and improves solution customization capabilities.

[0038] The data collection module of the API service performs data collection operations by real-time monitoring of the terminal's operating status, including indicators such as CPU / memory usage, network latency, and exception logs, to achieve comprehensive awareness of the client's operating status data. By performing lightweight computing operations such as data cleaning, aggregation, and compression locally, preliminary processing of raw data and optimization of transmission load are achieved. It is understandable that the data collection module can use intelligent scheduling algorithms to trigger upload strategies during idle periods based on device resource usage such as power, network bandwidth, and CPU load, thereby achieving a balanced control of data reporting and business performance.

[0039] For example, the management end provides management services such as process visualization, user management, permission management, asset group management, link monitoring, link reset, data retransmission, breakpoint resumption, data statistics, traffic management, and plug-in management based on the uploaded process data. At the same time, user management, permission management, and asset group management solve the problem of black box data service process and insufficient visualization monitoring and handling capabilities. Among them, the data acquisition module of the API service and the calling unit in the data control module of the server provide the process data uploading function on both ends. In addition to collecting the operating data generated by the data control module itself, the calling unit also collects real-time data and process data in the data assembly module, and completes the data uploading in a time-sharing manner. It can be understood that the management end draws a complete data transmission service process by handling the operating data of both ends, monitors the data service status in real time, and provides various types of management services such as plug-in management and emergency operations. It is responsible for the synchronization management of different data files. Multiple synchronization strategies can be set, and different synchronization strategies can be selected for each data file. It can also choose whether to enable synchronization. For data that is not needed temporarily, the user can choose to turn it off.

[0040] The device provided in the embodiment of the present application realizes dynamic, low-latency and high-reliability data transmission by constructing a long-connection data transmission channel and a heartbeat service mechanism based on the MCWebSocket protocol. Moreover, through the intelligent division strategy of exclusive channels and shared channels, metadata and business data are classified and transmitted, which improves the utilization rate of channel resources while reducing the risk of network congestion caused by mixed data transmission. And through the collaborative processing of the Apache Flink streaming computing engine and the Kafka message queue, real-time streaming conversion of structured messages is realized, thereby improving data flow efficiency. In addition, the embodiment of the present application also improves the security protection capability of financial data transmission through multi-level encrypted transmission and progress synchronization mechanism, and eliminates the process black box through the visual monitoring system of the management end, and realizes visual tracking of the transmission status of the entire link.

[0041] The following describes the process of a financial information reporting method. The present embodiment provides a financial information reporting method, including:

[0042] 101. The API service responds to the data transmission request and forwards the data transmission request to the server.

[0043] In an embodiment of the present application, the API service can identify the client's development language, such as Java, Python, Go, C#, etc., based on the Accept-Language or User-Agent field in the client request header, and then match the corresponding communication plug-in from the plug-in library. The API service loads the target communication plug-in and generates a standardized API interface so that the client can interact with the server through the communication plug-in to achieve protocol compatibility transmission in a cross-language environment. This eliminates the integration cost caused by differences in programming languages on the client side and improves the solution customization capability.

[0044] Furthermore, the client submits a login request containing an identity identifier, a dynamic token signature, and credential information through the communication plug-in. The API service calls the identity security module to parse the request message, extract the identity identifier, dynamic token signature, and credential information, and pass them to the server-side permission control and verification module. The permission control and verification module verifies the identity identifier, dynamic token signature, and credential information, including digital signature verification, credential validity check, and identity permission matching. If the verification result indicates a pass, an access token with a time limit is generated and passed to the API service.

[0045] It is understandable that the token contains the client identification, permission scope and validity period information, is digitally signed using an asymmetric encryption algorithm, and is returned to the client through a secure channel.

[0046] After the client obtains the token, the communication plug-in automatically injects it into the protocol message header of subsequent requests, such as the HTTP Authorization header.

[0047] It is understandable that when the server subsequently processes the client's data request, it can verify the validity and digital signature of the token in real time, and then complete the identity authentication and permission verification of the request.

[0048] 102. The server initiates a process of creating a long connection data transmission channel so that the API service notifies the client to create a data receiving node. When receiving the node number returned by the client, the server calls the MCWebSocket communication protocol for the node number to build a long connection data transmission channel, and establishes a heartbeat service mechanism to detect the status of the long connection data transmission channel, wherein the long connection data transmission channel includes an exclusive channel and a shared channel, the exclusive channel is used to transmit metadata, and the shared channel is used to transmit business data, task data and process data.

[0049] In the embodiments of this application, Figure 3 and Figure 4As shown in the figure, after the server receives the data transmission request, it prepares to start the workflow for creating a persistent data transmission channel. Based on the expanded MCWebSocket protocol, the server classifies and manages the data and plans persistent data transmission channels. Specifically, the server first calls the data management module to start the process of creating a persistent data transmission channel. In this process, the API service is used to notify the client to create a metadata receiving node. It should be noted that metadata contains key information that describes the attributes of the transmitted data, such as data format, data volume, data structure, data field type, etc. It provides the basis for the client to receive, store and use data, and once the metadata changes, the server will push updates to the client in a timely manner.

[0050] After the client completes the creation of the metadata receiving node NodeS, it returns the node number, such as NSid-XXX, to the server. After receiving the node number, the server calls the MCWebSocket protocol to build the first long connection data transmission channel, such as NSid-XXX-Channel, for the NSid-XXX node. This channel is an exclusive channel specifically used to transmit metadata. The two parties establish a heartbeat service mechanism, where the client initiates a ping command and the server returns a pong command to confirm the establishment of the long connection. In particular, two primary and backup channels will be created for NSid-XXX-Channel to support active switching in abnormal situations.

[0051] Furthermore, the API service notifies the client to create a data receiving node cluster, which includes task data receiving nodes, business data receiving nodes, and process data receiving nodes. After the client completes node cluster creation, it returns a node ID cluster to the server. This node ID cluster includes the task data receiving node IDs NTid-XXX, business data receiving node IDs NBid-XXX, and process data receiving node IDs NMid-XXX. Upon receiving the node ID cluster, the server invokes the MCWebSocket protocol to establish a second persistent data transmission channel for the business data stream, a third persistent data transmission channel for the task data stream, and a fourth persistent data transmission channel for the process data stream, targeting the NBid-XXX, NTid-XXX, and NMid-XXX nodes. These three channels are shared. These shared channels utilize multiplexing technology for efficient sharing, and the channel load balancing capabilities integrated into the channel management component ensure balanced transmission capacity. Both parties also establish a heartbeat service mechanism: the client initiates a ping command, and the server responds with a pong command to establish a shared persistent connection. Furthermore, two primary and backup channels are created for these three channels to support active failover in the event of an anomaly.

[0052] To ensure the reliability and stability of persistent data transmission channels, each persistent data transmission channel is equipped with a primary and backup transmission channel. A heartbeat service mechanism is also established to monitor channel status in real time. If a heartbeat timeout is detected on the primary transmission channel, the server invokes the backup transmission channel to ensure that the API service can continue transmitting the corresponding data stream to be reported via the backup transmission channel. This mechanism effectively avoids data transmission interruptions caused by single channel failures, greatly improving data transmission reliability and ensuring the continuity and stability of financial information reporting.

[0053] It should be noted that the MCWebSocket communication protocol in the embodiments of this application is based on the standard WebSocket duplex communication protocol, with optimizations made to transmission channel management, transmission data compression algorithm optimization, and protocol header extension. This protocol can enhance the real-time performance, accuracy, security, and efficiency of the financial information reporting device, specifically including the following improvements:

[0054] First, transmission channel management supports multi-channel customization, automatic transmission management, and multiplexing. By modifying the WebSocket constructor, it can parse the multi-channel identifier mchlId (composed of the channel type chlType and the single channel identifier schlId) from a URL containing a channel ID (e.g., ws: / / mchlId / example.com / ws). The protocol reserved field RSV1 is enabled to indicate whether to use the multi-channel dispatch mechanism. The mchlId is passed as a parameter to subsequent customized events such as sendbychl(), onopenbychl(), onmessagebychl(), onerrorbychl(), and onclosebychl(). The server adds channel and data management components, assembling the mchlId information according to the principle of using dedicated channels for metadata and shared channels for other data. Channel load balancing strategies are then planned based on data volume and channel traffic. The mchlId is passed as a parameter to the constructor to complete WebSocket construction. Based on the planned load balancing strategy, five APIs, including sendbychl(), are called to establish and manage multi-type persistent channels. In terms of channel multiplexing, a shared channel mechanism was established by extending MultiplexingExtension. By modifying the receive() method, channel subscription method, and cleanupChannels() method, a multi-channel sharing mechanism based on mchlId was implemented. Extended multiplexing balanced channel pressure and optimized channel transmission bandwidth to further improve data transmission efficiency. Through technical measures such as refactoring constructors, modifying API interfaces, adding channel and data management, and expanding multiplexing, the real-time and efficient data transmission was continuously improved.

[0055] The transmission data compression algorithm has been optimized by rewriting the CompressionExtension constructor to set parameters such as the compression threshold and compression algorithm level, and continuously monitoring the compression ratio through the compression.ratio parameter. Regarding compression algorithm settings, considering the diversity of data formats transmitted by real-time data services, the configuration mechanism for the compression algorithm level parameter has been expanded, and the protocol reserved field RSV2 has been enabled to indicate whether to adaptively select compression algorithms such as zlib and gzip. The data sending server supports autonomous switching of compression algorithms based on different data types, such as automatically selecting gzip for file stream compression and zlib for data stream files. Data compression is then completed based on the selected algorithm, and transmission is completed after data compression is completed.

[0056] The basic WebSocket message format consists of two parts: the message header and the message body. The message body is the actual data transmitted, which supports text and binary data. This extension focuses on the message header. The expanded message header content is as follows:

[0057] FIN: Indicates the integrity of the information, 1 indicates the completion of the message.

[0058] RSV1, RSV2, RSV3: Enable the two spare fields RSV1 and RSV2, where RSV1 stores the identification information corresponding to the multi-channel mchlId, RSV2 stores the compression algorithm selection identification information, and RSV3 remains as a reserved field and is not used.

[0059] Opcode: identifies the type of message, including text message, binary message, client sends ping message to server, server responds to pong message to client, among which for.

[0060] Mask: Indicates whether the message is encrypted.

[0061] Payload length: indicates the length of the message body.

[0062] Masking key: appears only when the message needs to be encrypted and is used to encrypt and decrypt the message.

[0063] Payload data: Adds extended data. Bytes 0-4 represent the specific multi-channel information mchlId, used with RSV1. Bytes 5-6 represent the specific compression algorithm, used with RSV2. Subsequent bytes are used to store application data.

[0064] 103. Based on the data transmission request, the server obtains the data to be reported, standardizes the data to be reported into structured messages, and divides the data to be reported into Kafka message queues according to data types. Based on the streaming computing engine Apache Flink, the server converts the static structured messages in the Kafka message queue into dynamic data streams to be reported, and passes the data streams to be reported to the API service.

[0065] After receiving a data transmission request, the server invokes the data control module to receive the request and pass it to the data assembly module. Based on the data identifier indicated in the data transmission request, the data assembly module invokes the data source control module to retrieve metadata and raw data from heterogeneous data sources and process this data as the data to be reported. It should be noted that the raw data originates from heterogeneous data sources, including but not limited to Oracle, MySQL, Hive, DB2, and external APIs. The data assembly module standardizes the data to be reported into structured messages, enabling subsequent processing of data from different sources and formats in a unified format. This facilitates data storage, analysis, and transmission, improving the system's compatibility and processing efficiency for various data types. The data source identifier, timestamp, and version information are then added to the structured messages, which are then queued to a Kafka message queue. The server invokes the real-time data module, leveraging the streaming computing engine Apache Flink, to convert the static structured messages in the Kafka message queue into a dynamic data stream to be reported, and then passes this data stream to the API service. Kafka offers high throughput and strong scalability. It can rapidly process large amounts of data, buffering and distributing pending data, avoiding data congestion and improving data transmission efficiency. Kafka also supports persistent message storage, ensuring data loss and providing reliable support for subsequent data processing. Flink offers low latency, high concurrency, and real-time data processing capabilities. By converting static messages into dynamic data streams, it enables real-time data transmission and processing, meeting the needs of business scenarios with high data real-time requirements, such as financial transaction monitoring and real-time data analysis.

[0066] 104. The API service encrypts the data stream to be reported. When there is a metadata stream in the data stream to be reported, the encrypted metadata stream is transmitted to the client through an exclusive channel, and the data reception progress synchronized by the client is received. When the data reception progress indicates that the client has completed data reception, the other data streams to be reported except the encrypted metadata stream are transmitted to the client through a shared channel, and the data reception progress synchronized by the client is received.

[0067] In an embodiment of the present application, the API service calls the data receiving module, which is responsible for receiving the data stream to be reported. At the same time, the API service calls the identity security module to perform dynamic token encryption on the data stream to be reported, and marks the encryption status in the MCWebSocket communication protocol message header to ensure the security of the data during transmission. In the actual operation process, the data stream to be reported may include at least one of the metadata stream, business data stream, task data stream and process data stream. When there is a metadata stream in the data stream to be reported, the API service initiates real-time transmission of the metadata stream through the exclusive channel NSid-XXX-Channel. The client receives the data synchronously and sends the data reception progress to the API service in a timely manner. The client decrypts the encrypted metadata stream based on the Masking key field in the MCWebSocket communication protocol message header to obtain metadata. When the data reception progress indicates that the client has completed the metadata stream reception, the client completes the metadata-related business logic execution and makes basic preparations for the subsequent use of the data. Afterwards, the client notifies the server that the metadata has been successfully received and waits for the server's response to start the transmission of business data and task data streams.

[0068] After receiving notification from the client that metadata has been successfully received, the server transmits all pending data streams, excluding the encrypted metadata stream, to the client in real time via a shared channel. The client synchronously receives the data and sends the progress of data reception to the API service. Simultaneously, the client decrypts the encrypted pending data stream using the Masking Key field in the protocol message header to obtain the business data and task data. The client processes the business data according to the rules indicated in the task data and, upon completion of receiving the pending data stream, notifies the server that this round of data transmission is complete.

[0069] The API service receives the data reception progress sent by the client, updates the transmission status of the data stream to be reported based on the progress, and records the breakpoint sequence number. The data reception progress is then transmitted to the management end for visual display, allowing managers to understand the data transmission status in real time.

[0070] As you can understand, the client sends data reception progress to the server at regular intervals. If data is missing during the reception process, the client initiates a resume request. The API service retransmits the missing data based on the breakpoint sequence number carried in the resume request to ensure complete data transmission.

[0071] 105. The server and API services collect process data in the financial information reporting process and transmit it to the management side for visual display.

[0072] In an embodiment of the present application, the management end provides management services such as process visualization, user management, authority management, asset group management, link monitoring, link reset, data retransmission, breakpoint resumption, data statistics, traffic management and plug-in management based on the uploaded process data. At the same time, the problems of black box of data service process and insufficient visualization monitoring and disposal capabilities are solved through user management, authority management and asset group management. Among them, the calling unit in the data acquisition module of the API service and the data control module of the server provides the function of uploading process data at both ends. In addition to collecting the operating data generated by the data control module itself, the calling unit also collects real-time data and process data in the data assembly module, and completes the data upload in time. It can be understood that the management end draws a complete data transmission service process by handling the operating data of both ends, monitors the data service status in real time, and provides various types of control services such as plug-in control and emergency operation.

[0073] It is understandable that in the embodiment of the present application, the server can also obtain the data to be reported in response to the autonomous scheduling instruction according to the scheduling plan. The data to be reported is standardized into a structured message, and the data to be reported is divided into the Kafka message queue according to the data type. And based on the Apache Flink streaming computing engine, the static data to be reported in the Kafka message queue is converted into dynamic streaming data, and the streaming data is encrypted to obtain an encrypted data stream to be reported. When there is an encrypted metadata stream in the data stream to be reported, the server calls the API service to transmit the encrypted metadata stream to the client through an exclusive channel, and receives the data reception progress synchronized by the client. When the data reception progress indicates that the client has completed data reception, the other data streams to be reported except the encrypted metadata stream are transmitted to the client through a shared channel, and the data reception progress synchronized by the client is received. The server and the API service collect process data in the financial information reporting process and transmit it to the management end for visual display.

[0074] The method provided in the embodiment of the present application realizes dynamic, low-latency and high-reliability data transmission by constructing a long-connection data transmission channel and a heartbeat service mechanism based on the MCWebSocket protocol. Moreover, through the intelligent division strategy of exclusive channels and shared channels, metadata and business data are classified and transmitted, which improves the utilization rate of channel resources while reducing the risk of network congestion caused by mixed data transmission. And through the collaborative processing of the Apache Flink streaming computing engine and the Kafka message queue, real-time streaming conversion of structured messages is realized, thereby improving data flow efficiency. In addition, the embodiment of the present application also improves the security protection capability of financial data transmission through multi-level encrypted transmission and progress synchronization mechanism, and eliminates the process black box through the visual monitoring system of the management end, and realizes visual tracking of the transmission status of the entire link.

[0075] To solve the above technical problems, the embodiment of the present invention also provides a computer device. Figure 5 , Figure 5 This is a basic structural block diagram of the computer device in this embodiment.

[0076] like Figure 5 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a non-volatile storage medium, a memory and a network interface connected via a system bus. Among them, the non-volatile storage medium of the computer device stores an operating system, a database and computer-readable instructions, and the database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement a financial information reporting method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute a financial information reporting method. The network interface of the computer device is used to connect and communicate with the terminal. Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0077] The memory stores the program code and various data required to execute the aforementioned modules. The network interface is used to transmit data to and from a user terminal or server. In this embodiment, the memory stores the program code and data required to execute all submodules within the financial information reporting device. The server can call upon the server's program code and data to execute the functions of all submodules.

[0078] The present invention also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the financial information reporting method of any of the above embodiments.

[0079] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed, the program can include the processes in the above-described method embodiments. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0080] Those skilled in the art will appreciate that the steps, measures, and schemes in the various operations, methods, and processes discussed in this application may be interchanged, modified, combined, or deleted. Furthermore, other steps, measures, and schemes in the various operations, methods, and processes discussed in this application may also be interchanged, modified, rearranged, decomposed, combined, or deleted. Furthermore, steps, measures, and schemes in the prior art that are similar to those disclosed in this application may also be interchanged, modified, rearranged, decomposed, combined, or deleted.

[0081] The above description is only part of the implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for reporting financial information, characterized in that: include: The API service responds to the data transmission request and forwards the data transmission request to the server; The server initiates a process for creating a long connection data transmission channel so that the API service notifies the client to create a data receiving node, and upon receiving the node number returned by the client, the server calls the MCWebSocket communication protocol for the node number to build a long connection data transmission channel, and establishes a heartbeat service mechanism to detect the status of the long connection data transmission channel, wherein the long connection data transmission channel includes an exclusive channel and a shared channel, the exclusive channel is used to transmit metadata, and the shared channel is used to transmit business data, task data and process data; The server obtains the data to be reported based on the data transmission request, standardizes the data to be reported into structured messages, and divides the data to be reported into Kafka message queues according to data types. Based on the stream computing engine Apache Flink, the server converts the static structured messages in the Kafka message queue into a dynamic data stream to be reported, and transmits the data stream to the API service. The API service encrypts the data stream to be reported, and when a metadata stream exists in the data stream to be reported, transmits the encrypted metadata stream to the client through the exclusive channel, and receives the data reception progress synchronized by the client; when the data reception progress indicates that the client has completed data reception, transmits the other data streams to be reported except the encrypted metadata stream to the client through the shared channel, and receives the data reception progress synchronized by the client; The server and the API service collect process data in the financial information reporting process and transmit it to the management end for visual display.

2. The method according to claim 1, characterized in that The method further includes, before the API service responds to the data transmission request and forwards the data transmission request to the server: The API service calls the coding language plug-in, determines a communication plug-in that matches the coding language type according to the coding language type of the client, and loads the communication plug-in, so that the client can interact with the server through the communication plug-in. The API service calls the identity security module to parse the login request message uploaded by the client through the communication plug-in, obtain the identity identifier, dynamic token signature and credential information carried in the login request message, and pass the identity identifier, dynamic token signature and credential information to the authority control and verification module of the server; The server calls the permission control verification module to verify the identity, the dynamic token signature, and the credential information. If the verification result indicates a pass, it generates an access token with a time limit and passes the access token to the API service. The API service sends the access token to the client, so that the client injects the access token into the protocol message header of the data transmission request through the communication plug-in.

3. The method according to claim 1, characterized in that The server initiates a process of creating a persistent connection data transmission channel so that the API service notifies the client to create a data receiving node, and upon receiving the node number returned by the client, the server calls the MCWebSocket communication protocol to build a persistent connection data transmission channel for the node number, including: The server calls the data control module to start the process of creating a persistent connection data transmission channel so that the API service notifies the client to create a metadata receiving node, and transmits the node number returned by the client to the server; Upon receiving the node number, the server creates a first persistent connection data transmission channel, wherein the first persistent connection data transmission channel is the exclusive channel; The API service notifies the client to create a data receiving node cluster, and transmits the node ID cluster returned by the client to the server; When receiving the node number cluster, the server creates a second long connection data transmission channel for transmitting the business data stream, a third long connection data transmission channel for transmitting the task data stream, and a fourth long connection data transmission channel for transmitting the process data stream, wherein the second long connection data transmission channel, the third long connection data transmission channel, and the fourth long connection data transmission channel are shared channels; Among them, each long connection data transmission channel includes a main transmission channel and a backup transmission channel. A heartbeat service mechanism is established to detect the status of the long connection data transmission channel. If the heartbeat of the main transmission channel is detected to have timed out, the server is called to switch to the backup transmission channel, so that the API service transmits the corresponding data stream to be reported based on the backup transmission channel. The data receiving node cluster includes a task data receiving node, a business data receiving node, and a process data receiving node. Among them, the MCWebSocket communication protocol is expanded based on the standard WebSocket duplex communication protocol. The MCWebSocket communication protocol distinguishes the data transmission channel type through the channel identifier, selects the compression algorithm according to the data type, and identifies the compression algorithm type through the RSV2 field of the protocol message header. A new RSV1 field is added to the WebSocket message header to store the channel identification information. Among them, the channel type includes the exclusive channel and the shared channel.

4. The method according to claim 1, wherein The server obtains the data to be reported based on the data transmission request, standardizes the data to be reported into structured messages, and divides the data to be reported into Kafka message queues according to data types. Based on the stream computing engine Apache Flink, the server converts the static structured messages in the Kafka message queue into a dynamic data stream to be reported, and transmits the data stream to the API service, including: The server calls the data control module to receive the data transmission request, and passes the data transmission request to the data assembly module; The server, through the data assembly module, calls the data source control module according to the data identifier indicated by the data transmission request to obtain metadata and original data as the data to be reported, and standardizes the data to be reported into a structured message, and adds a data source identifier, a timestamp and version information to the structured message, and divides the structured message with the added data source identifier, timestamp and version information into a Kafka message queue according to the data type, and calls the real-time data module, uses the streaming computing engine Apache Flink, converts the static structured message in the Kafka message queue into a dynamic data stream to be reported, and passes the data stream to be reported to the API service; The original data comes from heterogeneous data sources, including but not limited to Oracle, MySQL, Hive, DB2 and external API.

5. The method according to claim 1, wherein The API service encrypts the data stream to be reported, and when a metadata stream exists in the data stream to be reported, transmits the encrypted metadata stream to the client through the exclusive channel, and receives the data reception progress synchronized by the client; when the data reception progress indicates that the client has completed data reception, transmits the other data streams to be reported except the encrypted metadata stream to the client through the shared channel, and receives the data reception progress synchronized by the client, including: The API service calls the data receiving module to receive the data stream to be reported, and calls the identity security module to perform dynamic token encryption on the data stream to be reported, and marks the encryption status in the MCWebSocket communication protocol message header; When there is a metadata stream in the data stream to be reported, the API service transmits the encrypted metadata stream to the client through the exclusive channel; The client receives the encrypted metadata stream through a dedicated channel, sends the data reception progress to the API service, and decrypts the encrypted metadata stream based on the Masking key field in the protocol message header to obtain metadata; When the data reception progress indicates that the client has completed receiving the metadata stream, the API service transmits other data streams to be reported except the encrypted metadata stream to the client through the shared channel; The client receives the encrypted data stream to be reported through the shared channel, sends the data reception progress to the API service, decrypts the encrypted data stream to be reported based on the Masking key field in the protocol message header, obtains business data and task data, and processes the business data according to the rules indicated by the task data; The API service receives the data reception progress sent by the client, updates the transmission status of the data stream to be reported according to the data reception progress, records the breakpoint sequence number, and transmits the data reception progress to the management end for visual display.

6. The method according to claim 5, characterized in that The method further comprises: The client sends data reception progress to the server at regular intervals. If data missing is detected, a resume request is initiated to enable the API service to transmit data again based on the breakpoint sequence number carried in the resume request.

7. The method according to claim 1, characterized in that The method further comprises: The server, in response to the autonomous scheduling instruction, obtains the data to be reported, standardizes the data to be reported into structured messages, and divides the data to be reported into Kafka message queues according to data types; and based on the Apache Flink streaming computing engine, converts the static data to be reported in the Kafka message queue into dynamic streaming data, and encrypts the streaming data to obtain an encrypted data stream to be reported; When an encrypted metadata stream exists in the data stream to be reported, the server calls the API service to transmit the encrypted metadata stream to the client through the exclusive channel, and receives the data reception progress synchronized by the client; when the data reception progress indicates that the client has completed data reception, the server transmits the other data streams to be reported except the encrypted metadata stream to the client through the shared channel, and receives the data reception progress synchronized by the client; The server and the API service collect process data in the financial information reporting process and transmit it to the management end for visual display.

8. A financial information reporting device, characterized in that: include: API service, server and management side; The API service responds to a data transmission request by forwarding the data transmission request to the server, and upon detecting that the server has initiated a process for creating a persistent connection data transmission channel, notifies the client to create a data receiving node, and encrypts the data stream to be reported. When a metadata stream is included in the data stream to be reported, the encrypted metadata stream is transmitted to the client via a dedicated channel, and the client receives the synchronized data reception progress. When the data reception progress indicates that the client has completed data reception, the remaining data streams to be reported, excluding the encrypted metadata stream, are transmitted to the client via a shared channel, and the client receives the synchronized data reception progress. Furthermore, the collected process data in the financial information reporting process is transmitted to the management end for visual display. The server initiates a long connection data transmission channel creation process, and upon receiving the node number returned by the client, calls the MCWebSocket communication protocol for the node number to build a long connection data transmission channel, and establishes a heartbeat service mechanism to detect the status of the long connection data transmission channel, and obtains the data to be reported based on the data transmission request, standardizes the data to be reported into structured messages, and divides the data to be reported into Kafka message queues according to data types, and converts the static structured messages in the Kafka message queue into dynamic data streams to be reported based on the streaming computing engine Apache Flink, passes the data streams to be reported to the API service, and collects process data in the financial information reporting process and transmits it to the management end for visual display, wherein the long connection data transmission channel includes an exclusive channel and a shared channel, the exclusive channel is used to transmit metadata, and the shared channel is used to transmit business data, task data and process data.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images