Data hub communication method for large-scale distributed telescope system and electronic device

Abstract

The application provides a data hub communication method for a large distributed telescope system and an electronic device, and can be applied to the technical field of distributed system communication. In the method, any heterogeneous service subsystem is taken as a publishing end, and another heterogeneous service subsystem is taken as a receiving end. The publishing end obtains service data of itself, encapsulates the service data into a target message containing a first integrity check value, and sends the target message to a data hub. The data hub performs theme authority verification on the target message, and sends the target message that passes the verification to the receiving end. The receiving end calculates a second integrity check value of the target message that passes the verification, and outputs the target message that passes the verification in a case where the first integrity check value and the second integrity check value satisfy a preset condition.

Description

Technical Field

[0001] This application relates to the field of distributed system communication technology, and in particular to a data hub communication method and electronic device for a large distributed telescope system. Background Technology

[0002] Large distributed telescopes are complex heterogeneous systems with multiple devices, modules, and deployment nodes. They consist of several independent yet highly collaborative subsystems, including operation control, telescope control, observation mission control, data management, data processing, dome control, and terminal instrument control. Throughout the entire astronomical observation mission, these subsystems need to continuously exchange real-time control commands, equipment operating status, observation configuration parameters, and fault alarm information, while simultaneously transmitting large files such as observation images, raw detection data, and log packets.

[0003] In existing systems, each subsystem typically selects its communication scheme independently based on its own development language, operating environment, and real-time requirements. The mainstream approach employs a hybrid network architecture combining direct network communication, general message queues, database polling, local file sharing, and network interfaces. This heterogeneous communication architecture presents the following problems in scenarios involving long-term operation and continuous multi-task observations of large telescopes:

[0004] First, the communication interfaces are not standardized, and each subsystem needs to implement its own communication logic, resulting in redundant development and high integration costs.

[0005] Second, the formats of various messages are inconsistent, and there is a lack of a unified data structure;

[0006] Third, messages may be damaged or tampered with during transmission due to network fluctuations or middleware malfunctions, and the receiving end may not be able to effectively identify them, which may easily lead to incorrect control commands or inaccurate observation data. Summary of the Invention

[0007] In view of the above problems, this application provides a data hub communication method and electronic device for large distributed telescope systems.

[0008] This application provides a data hub communication method for a large-scale distributed telescope system. The large-scale distributed telescope system includes a data hub and multiple heterogeneous business subsystems communicatively connected to the data hub. One heterogeneous business subsystem acts as the publishing end, and another heterogeneous business subsystem acts as the receiving end. The method includes: the publishing end acquiring its own business data, encapsulating the business data into a target message containing a first integrity check value, and sending it to the data hub; the data hub performing topic permission verification on the target message, and sending the verified target message to the receiving end; the receiving end calculating a second integrity check value for the verified target message, and outputting the verified target message if the first integrity check value and the second integrity check value satisfy a preset condition.

[0009] According to an embodiment of this application, the method further includes: before the data hub performs topic permission verification on the target message, the data hub creates a unified set of communication topics and generates topic permission control rules based on multiple heterogeneous business subsystems and the business interaction logic between the multiple heterogeneous business subsystems; wherein, the set of communication topics includes multiple pre-planned communication topics covering all business interaction scenarios in the large distributed telescope system; the topic permission control rules characterize the authorization relationship between each communication topic in the set of communication topics and the heterogeneous business subsystems in publishing or receiving operations.

[0010] According to an embodiment of this application, the data hub performs topic permission verification on the target message, including: obtaining the target communication topic in the target message; verifying whether the target communication topic exists in the communication topic set, and verifying whether the publishing end has been authorized to publish the target message to the target communication topic based on topic permission control rules; and determining that the target message verification is successful if it is determined that the target communication topic exists in the communication topic set and the publishing end has been authorized to publish the target message to the target communication topic.

[0011] According to an embodiment of this application, encapsulating business data into a target message containing a first integrity check value includes: dividing the business data into first data and second data according to a preset division rule, wherein the first data represents the semantic information of the business data, and the second data includes identification information for determining the uniqueness of the first data and attribute information of the first data; calculating the first integrity check value of the first data; encapsulating the first integrity check value in the second data; and encapsulating the first data and the encapsulated second data again to obtain the target message.

[0012] According to an embodiment of this application, the method further includes: the data hub creating a unified communication interface based on multiple predefined communication backends and multiple communication modes; multiple heterogeneous business subsystems establishing connections with the data hub through the unified communication interface; the data hub determining a communication adapter based on the communication configuration parameters of each heterogeneous business subsystem; the publishing end sending the target message to the data hub through the unified communication interface; and the data hub sequentially sending the verified target message to the receiving end through the unified communication interface and the communication adapter.

[0013] According to an embodiment of this application, the receiving end calculates a second integrity check value for the verified target message, and outputs the verified target message when it is determined that the first integrity check value and the second integrity check value meet preset conditions. This includes: splitting the verified target message into first data and second data, where the first data represents the semantic information of the business data, and the second data includes identification information for determining the uniqueness of the first data, attribute information of the first data, and a first integrity check value; calculating a second integrity check value for the first data; comparing the first integrity check value and the second integrity check value, and when the first integrity check value and the second integrity check value are equal, confirming that the target message is complete, and outputting the verified target message.

[0014] According to an embodiment of this application, the method further includes: when the first integrity check value is not equal to the second integrity check value, confirming that the target message is abnormal, discarding the target message and issuing an alarm.

[0015] According to an embodiment of this application, the method further includes: after outputting the verified target message, the receiving end determines the business module associated with the attribute information in the receiving end based on the attribute information of the first data in the second data; and calls the processing interface of the business module with the first data as a parameter to execute the business processing logic corresponding to the business data.

[0016] According to embodiments of this application, the large distributed telescope system further includes a file transfer service area isolated from the data hub; the method further includes: when the type of business data obtained by the publishing end is a file type, the publishing end uploads the business data to the file transfer service area, obtains the access address and file metadata of the business data in the file transfer service area, and encapsulates the access address and file metadata into a target message; the data hub receives the target message and performs topic permission verification on the target message, and sends the verified target message to the receiving end; the receiving end obtains the business data from the file transfer service area based on the access address in the verified target message, and performs integrity verification on the business data based on the file metadata in the verified target message.

[0017] Another aspect of this application provides an electronic device, including: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0018] This application provides a data hub communication method and electronic device for a large-scale distributed telescope system, which has at least the following technical advantages:

[0019] 1. By uniformly encapsulating business data and calculating the first integrity check value at the publishing end, the differences in message format between heterogeneous subsystems are eliminated and a benchmark is provided for the integrity verification of the target message; the data hub performs topic permission verification on the target message, intercepting illegal topics or unauthorized publishing requests before the message enters the communication link, thereby achieving security control; the receiving end recalculates the second integrity check value and compares it with the first integrity check value, and only outputs the message when the preset conditions are met, ensuring that damage or tampering during transmission can be identified and isolated in a timely manner.

[0020] 2. By creating a unified communication interface through the data hub, all heterogeneous business subsystems rely on only one standard application programming interface (API). This unified communication interface shields the underlying differences between various communication backends, reducing the integration complexity of each subsystem. Simultaneously, the data hub dynamically selects and binds the adapter to the specific communication backend based on the message topic or configuration. Adding or replacing a communication backend only requires supplementing the corresponding adapter; no modifications to the upper-layer business code are necessary. This reduces the cost of telescope system integration and subsequent maintenance, providing the flexibility of on-demand selection for different business scenarios. Attached Figure Description

[0021] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0022] Figure 1 This illustration schematically depicts an application scenario of a data hub communication method for a large distributed telescope system according to an embodiment of this application.

[0023] Figure 2 This illustration schematically shows a sequence diagram of a data hub communication method for a large distributed telescope system according to an embodiment of this application;

[0024] Figure 3 A flowchart illustrating the verification of the target message by the receiving end according to an embodiment of this application is shown schematically.

[0025] Figure 4 A block diagram of an electronic device suitable for implementing a data hub communication method for a large distributed telescope system, according to an embodiment of this application, is illustrated schematically. Detailed Implementation

[0026] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0027] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0028] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0029] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0030] Figure 1 The diagram illustrates an application scenario of a data hub communication method for a large distributed telescope system according to an embodiment of this application.

[0031] like Figure 1 As shown, the application scenario 100 according to the embodiments of this application may include a first business subsystem 101, a second business subsystem 102, a third business subsystem 103, a fourth business subsystem 104, and a data hub 105.

[0032] Among them, the first business subsystem 101, the second business subsystem 102, the third business subsystem 103, and the fourth business subsystem 104 are all heterogeneous business subsystems of the large-scale distributed telescope system. Any one of the heterogeneous business subsystems can act as a publishing end, used to collect local business data, encapsulate and generate a target message carrying a first integrity verification value, and send it to the data hub 105. The remaining heterogeneous business subsystems can act as receiving ends, used to receive the target message forwarded by the data hub, and complete the integrity verification and message output. The data hub 105 is used to execute the data hub communication method provided in this application embodiment, establish communication connections with each heterogeneous business subsystem, perform topic permission verification on the received target message, and forward the verified target message to the corresponding receiving end.

[0033] For example, the first business subsystem 101, the second business subsystem 102, the third business subsystem 103, and the fourth business subsystem 104 can be heterogeneous business equipment or service nodes with data acquisition, message encapsulation, command interaction, and data parsing capabilities, such as observation planning system, telescope control system, dome control system, operation control system, data processing system, and data management system.

[0034] The data hub 105 can be a communication server that performs message relay and permission verification functions. It receives target messages sent by the publishing end, performs topic permission verification, and forwards legitimate messages to the receiving end (this is just an example). The data hub 105 can identify, verify, and route received target messages to ensure the security and integrity of cross-subsystem data interaction. The data hub 105 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services such as cloud databases, cloud message queues, communication middleware, cloud storage, network services, and big data. The data hub 105 is used to realize message forwarding and topic permission control between various heterogeneous business subsystems, supporting the secure transmission of business data and control commands within the distributed telescope system.

[0035] It should be noted that the data hub communication method provided in this application embodiment can generally be executed by the data hub 105. Accordingly, the data hub communication device provided in this application embodiment can generally be disposed in the data hub 105.

[0036] It should be understood that Figure 1 The number of heterogeneous business subsystems and data hubs shown is merely illustrative. Depending on implementation needs, there can be any number of heterogeneous business subsystems and data hubs.

[0037] Figure 2A sequence diagram illustrating a data hub communication method for a large distributed telescope system according to an embodiment of this application is shown. Figure 2 As shown, the task scheduling method 200 according to an embodiment of this application may include steps S210 to S230.

[0038] The large distributed telescope system includes a data hub and multiple heterogeneous service subsystems that are connected to the data hub. Each heterogeneous service subsystem serves as the publishing end and another heterogeneous service subsystem serves as the receiving end.

[0039] In step S210, the publishing end obtains its own business data, encapsulates the business data into a target message containing a first integrity verification value, and sends it to the data hub.

[0040] The publishing end can be a heterogeneous business subsystem that actively generates and sends messages during communication, such as a telescope control subsystem or a data acquisition subsystem. Each publishing end has a unique subsystem identifier, used by the data hub for identity verification and authorization. The publishing end sends messages to one or more specified communication topics.

[0041] Business data can be the raw information that the publishing end needs to pass to other subsystems, including control commands, device status, observation parameters, alarm information, file metadata, etc.

[0042] A target message can be a data unit encapsulated in a unified format by the publishing end. For example, a target message consists of two parts: a fixed message header structure and a message body format that varies depending on the command type. The message body carries control commands, device status, observation parameters, alarm information, file metadata, etc., while the message header carries metadata (such as message identifier, message type, message name, timestamp, priority, request identifier, timeout parameters, message body checksum, etc.). Target messages ensure that all communication software encapsulates and parses messages according to a unified standard, and also facilitates data management software to automatically store message content in the database according to format rules.

[0043] The first integrity check value can be a hash value or checksum calculated by the sending end based on the message body content using a preset algorithm (such as CRC32, MD5, SHA-256). This value is written into the message header and used by the receiving end to verify whether the message body has been tampered with or corrupted during transmission.

[0044] For example, the telescope control subsystem (publisher) needs to send a pointing command to the actuator. First, the control subsystem generates the pointing command internally. Second, the control subsystem serializes the pointing command into a byte stream to form a message body, calculates a hash on the message body using the SHA-256 algorithm to obtain a first integrity check value. Then, it encapsulates the first integrity check value, the message's unique identifier, message type, message name, timestamp, priority, request identifier, and timeout parameters into a message header. The message header and message body together form the target message.

[0045] In step S220, the data center performs topic permission verification on the target message and sends the verified target message to the receiving end.

[0046] The data hub can be the core communication component in a large-scale distributed telescope system. It is responsible for uniformly receiving messages from all publishing ends, performing topic permission verification, routing and forwarding messages, and providing a unified communication interface and scalable adapters. The data hub pre-establishes a set of communication topics and topic permission control rules.

[0047] Topic permission verification can be a two-step check performed by the data hub on the target communication topic carried by the target message: 1) Legality verification: Does the target communication topic exist in the predefined set of communication topics? 2) Permission verification: Is the publisher authorized to publish messages to the target communication topic? Only when both verifications pass can the message be allowed to enter the subsequent distribution process. Messages that fail verification will be intercepted and an alarm will be recorded.

[0048] For example, once the receiving end has subscribed to the target communication topic, the data hub will push the message to the receiving end as soon as it receives a message related to the target communication topic.

[0049] For example, the telescope status monitoring subsystem, acting as the receiver, has subscribed to the communication topic "Status / Rack". The actuator subsystem, acting as the publisher, sends a status update message to this topic. The actuator subsystem constructs a target message with the target communication topic "Status / Rack" in the message header and data such as the rack's current azimuth and elevation angles in the message body. It then calls the data hub's publishing interface to send the message to the data hub. The data hub checks if "Status / Rack" is in a pre-created set of communication topics. Assuming the topic is defined, the check passes. The data hub then checks the publisher (actuator subsystem) against topic permission control rules to see if it is authorized to publish messages to the "Status / Rack" topic. The rules stipulate that only the "actuator subsystem" and "power management subsystem" are allowed to publish to this topic. The current publisher meets the conditions, and the permission check passes. After both checks pass, the data hub searches the list of receivers subscribed to the "Status / Rack" topic and finds that the status monitoring subsystem has subscribed. The data hub then pushes the target message to this receiver via the unified communication interface.

[0050] For example, if any validation fails (e.g., the topic is misspelled as "status / rack typo", or the publisher is "control subsystem" and the topic is not allowed to be published), the data hub will intercept the message and return an error to the publisher.

[0051] In step S230, the receiving end calculates the second integrity check value of the target message that has passed the verification, and outputs the target message that has passed the verification if it is determined that the first integrity check value and the second integrity check value meet the preset conditions.

[0052] The receiving end can be a heterogeneous business subsystem that actively subscribes to one or more communication topics and waits to receive messages. The receiving end receives target messages through a data hub and is responsible for message integrity verification and service distribution. For example, the receiving end could be a telescope status monitoring subsystem or a data storage subsystem.

[0053] The target message that passes the verification retains the original structure when it was encapsulated on the publishing end.

[0054] The second integrity check value can be a check value recalculated by the receiving end using the same preset algorithm on the message body content after receiving the message. The second integrity check value is calculated independently by the receiving end.

[0055] For example, the receiving end passes the validated message body to subsequent business processing modules (such as business listeners). If the validation fails, the message is discarded and an alarm is recorded, but no output is output.

[0056] For example, the data hub pushes a target message to the status monitoring subsystem. The status monitoring subsystem extracts the message body content, calculates the hash of the message body byte stream using the same SHA-256 algorithm as the publishing end, and obtains the second integrity check value; it extracts the first integrity check value from the message header, and the check passes when the first integrity check value is exactly equal to the second integrity check value.

[0057] The data hub communication method based on the embodiments of this application eliminates message format differences between heterogeneous subsystems and provides a benchmark for integrity verification by uniformly encapsulating business data and calculating a first integrity check value at the publishing end. The data hub performs topic permission verification on target messages, intercepting illegal topics or unauthorized publishing requests before the message enters the communication link, thus achieving security control. The receiving end recalculates a second integrity check value and compares it with the first integrity check value, only outputting the message when preset conditions are met, ensuring that damage or tampering during transmission can be identified and isolated in a timely manner. This significantly improves the reliability, security, and scalability of communication in large distributed telescope systems.

[0058] In embodiments of this application, the method further includes: before the data hub performs topic permission verification on the target message, the data hub creates a unified set of communication topics and generates topic permission control rules based on multiple heterogeneous business subsystems and the business interaction logic between the multiple heterogeneous business subsystems; wherein, the set of communication topics includes multiple pre-planned communication topics covering all business interaction scenarios in the large distributed telescope system; the topic permission control rules characterize the authorization relationship between each communication topic in the set of communication topics and the heterogeneous business subsystems in publishing or receiving operations.

[0059] Business interaction logic can be the actual collaborative relationships between the various subsystems in a telescope system. For example, the telescope control subsystem needs to send instructions to the actuators (issue), and the status monitoring subsystem needs to obtain the equipment status (subscribe).

[0060] The communication topic set can be a pre-planned and defined list of all communication channel names, covering all operational interaction scenarios in the telescope system (such as control commands, status reporting, alarm broadcasts, observation data transmission, etc.). Each topic corresponds to a specific type of operational data stream.

[0061] Topic access control rules can be applied to each communication topic, explicitly specifying which heterogeneous business subsystems have the authority to publish messages to that topic and which have the authority to receive (subscribe to) messages from that topic. Only authorized heterogeneous business subsystems can participate in communication on a specific topic.

[0062] Authorization relationships can be defined in two ways: in publishing operations, it refers to whether a heterogeneous business subsystem is allowed to send messages to a certain communication topic; and in receiving operations, it refers to whether a heterogeneous business subsystem is allowed to subscribe to a certain communication topic and retrieve messages from it.

[0063] The data hub communication method based on the embodiments of this application establishes a unified set of communication topics and topic permission control rules in advance before topic permission verification. On the one hand, the communication topic set covering all business scenarios avoids naming conflicts and communication chaos caused by each subsystem arbitrarily creating topics. On the other hand, it clarifies the publish / receive authorization relationship between each topic and heterogeneous business subsystems, so that only legitimate subsystems can publish messages to or subscribe to messages from designated topics, thereby preventing unauthorized access, mispublishing, and unauthorized subscription from the source and significantly reducing the risk of mispublishing and missubscription.

[0064] In the embodiments of this application, the data hub performs topic permission verification on the target message, including: obtaining the target communication topic in the target message; verifying whether the target communication topic exists in the communication topic set, and verifying whether the publishing end has been authorized to publish the target message to the target communication topic based on the topic permission control rules; and determining that the target message verification is successful if it is determined that the target communication topic exists in the communication topic set and the publishing end has been authorized to publish the target message to the target communication topic.

[0065] The target communication topic can be the topic name bound by the publishing end when encapsulating the message. It is used to identify the business category to which the message belongs (such as "instruction / direction" or "status / rack") and serves as the basis for the data hub to perform routing and security verification.

[0066] The data hub communication method based on the embodiments of this application performs dual verification on the target communication topic. First, it confirms whether the topic exists in a predefined set to intercept illegal topics. Then, it verifies whether the publishing end has publishing authorization according to topic permission control rules, thus achieving security filtering before the message enters the forwarding link. This not only prevents mispublishing due to incorrect topic naming or system failure, but also eliminates unauthorized subsystems from sending control commands or sensitive data, effectively ensuring the orderliness and security of telescope communication.

[0067] In embodiments of this application, encapsulating business data into a target message containing a first integrity check value includes: dividing the business data into first data and second data according to a preset division rule, wherein the first data represents the semantic information of the business data, and the second data includes identification information for determining the uniqueness of the first data and attribute information of the first data; calculating the first integrity check value of the first data; encapsulating the first integrity check value in the second data; and encapsulating the first data and the encapsulated second data again to obtain the target message.

[0068] The preset partitioning rules can be logical rules that decompose business data into first data and second data based on the characteristics of the business data (such as size, sensitivity, and functional role). For example, business semantic content can be partitioned into first data (such as the aforementioned message body), and descriptive and control information can be partitioned into second data (such as the aforementioned message header).

[0069] The data hub communication method based on the embodiments of this application divides business data into semantic data (first data) and metadata (second data) according to functional roles, and calculates an integrity verification value based on the first data and stores it in the second data, thus achieving an organic integration of message structure and verification mechanism. The format of the divided target message is clear and uniform, and the receiving end can quickly locate metadata and business content, improving parsing efficiency.

[0070] In embodiments of this application, the method further includes: the data hub creating a unified communication interface based on multiple predefined communication backends and multiple communication modes; multiple heterogeneous business subsystems establishing connections with the data hub through the unified communication interface; the data hub determining a communication adapter based on the communication configuration parameters of each heterogeneous business subsystem; the publishing end sending the target message to the data hub through the unified communication interface; and the data hub sequentially sending the verified target message to the receiving end through the unified communication interface and the target adapter.

[0071] Communication configuration parameters can be configuration information used to define the connection method between the data hub and the communication backend, including the communication backend type (such as Kafka), connection address, communication mode (such as publish-subscribe, request-response, etc.), topic list, subscription start position, etc.

[0072] The communication backend can be the underlying infrastructure or middleware software responsible for message storage, routing, and transmission, including message queues (such as Kafka), real-time communication libraries (such as ZeroMQ), etc.

[0073] A communication pattern can be a behavioral paradigm or communication paradigm for message interaction between the publisher and receiver. It defines how messages are sent, routed, received, and how the two parties collaborate. Communication patterns include PubSub (Publish-Subscribe), ReqRep (Request-Reply), and Pair (Peer-to-Peer communication), etc.

[0074] A unified communication interface is a set of standardized application programming interfaces (APIs) provided by the data hub. All heterogeneous business subsystems, regardless of the language or protocol they use, interact with the data hub through this interface. This interface shields the underlying communication backend from differences, keeping the upper-layer business code stable.

[0075] Communication adapters can be scalable components within the data hub, with each adapter bound to a specific communication backend. Each communication adapter implements a unified communication interface agreement and can provide capabilities such as publish / subscribe, request / response, historical retrieval, low-latency transmission, routing, or streaming transmission based on its own backend characteristics. The communication adapter is responsible for converting interface calls into native backend application interface calls and converting backend data into a unified message format.

[0076] The communication backend type is not limited to a preset category and can be connected to communication components with message publishing, subscription, persistence, historical reading, low-latency transmission, routing distribution, or streaming transmission capabilities. The communication modes support publish-subscribe, point-to-point, request-response, push-pull, routing distribution, and streaming communication. New communication modes can be added according to business needs and flexibly combined and selected.

[0077] Different communication adapters can support the following capabilities based on their own characteristics: For data requiring historical tracing, fault recovery, and playback of observation processes, a communication backend with message persistence and historical reading capabilities can be selected; for low-latency control commands, status synchronization, or real-time events, a communication backend with low-latency transmission capabilities can be selected; for one-to-one control interaction, request-response or point-to-point communication modes can be used; for task distribution or data processing pipelines, push-pull communication modes can be used; for multi-client routing, identity differentiation, or complex distribution, routing distribution communication modes can be used; for raw TCP (Transmission Control Protocol) streams or special instrument data access, streaming communication modes can be used.

[0078] This application's embodiments are compatible with various communication methods, from point-to-point (P2P) connections to centralized publish / subscribe. Specifically, different communication backends support this; for example, Kafka supports a centralized publish / subscribe model. Other communication backends, such as ZeroMQ's Pair, Push / Pull, and Req / Rep, support P2P communication. A flexible and selectable communication layer is constructed from different communication backends (Kafka, Redis, ZeroMQ, etc.) and communication modes (publish / subscribe, push / pull, etc.).

[0079] The data hub communication method based on embodiments of this application creates a unified communication interface through the data hub, enabling all heterogeneous business subsystems to rely on only one standard application programming interface. This unified communication interface shields the underlying differences between different communication backends, reducing the complexity of subsystem integration. Simultaneously, the data hub dynamically selects the communication adapter bound to the specific communication backend based on the message topic or configuration. Adding or replacing a communication backend only requires supplementing the corresponding communication adapter; no modifications to the upper-layer business code are necessary. This reduces the cost of telescope system integration and subsequent maintenance, providing the flexibility of on-demand selection for different business scenarios.

[0080] Figure 3 A flowchart illustrating the verification of the target message by the receiving end according to an embodiment of this application is shown. Figure 3 As shown, in some embodiments, the receiving end verifies the target message, including steps S231 to S2352.

[0081] In step S231, the target message that has passed the verification is split into first data and second data. The first data represents the semantic information of the business data, and the second data includes identification information used to determine the uniqueness of the first data, attribute information of the first data, and a first integrity verification value.

[0082] The receiving end deconstructs the target message that has passed the topic permission verification, separating the first data (such as the message body) that carries business semantics and the second data (such as the message header) that carries metadata, laying the foundation for subsequent verification of the integrity of the first data.

[0083] In step S232, the second integrity check value of the first data is calculated.

[0084] The receiving end recalculates the checksum of the first data using the same preset algorithm (such as SHA-256) as the transmitting end, obtaining a second integrity checksum. A checksum independent of the transmitting end is generated and compared with the first integrity checksum to achieve end-to-end proactive verification.

[0085] In step S233, the first integrity check value and the second integrity check value are compared.

[0086] The first integrity check value provided by the publishing end is compared byte by byte or numerically with the second integrity check value calculated by the receiving end to detect whether any bit changes have occurred in the first data during transmission, storage or forwarding.

[0087] In step S234, it is determined whether the first integrity check value and the second integrity check value are equal.

[0088] A binary judgment is made based on the comparison results: equal or unequal. The comparison results are then converted into discrete pass / fail signals.

[0089] In step S2351, if they are equal, the target message is confirmed to be complete, and the target message that has passed the verification is output.

[0090] If the verification passes, the complete target message (or the first data output directly) is handed over to the business listener for processing. This ensures that only tamper-proof business data can enter the telescope's control, monitoring, or data processing flow, preventing erroneous commands or data contamination.

[0091] In step S2352, if they are not equal, the target message is confirmed to be abnormal, the target message is discarded and an alarm is triggered.

[0092] If the verification fails, the entire message is discarded, and an alarm log (including message ID, source, timestamp, etc.) is generated. This isolates corrupt or malicious messages to prevent them from affecting the system; it also provides traceable anomaly records.

[0093] Based on the data hub communication method of this application embodiment, the receiving end independently completes the end-to-end integrity verification of the business data (first data), ensuring that key information such as telescope control commands, equipment status, and observation parameters remain original and intact after passing through the entire link of the publishing end, data hub, and receiving end.

[0094] In embodiments of this application, the method further includes: after outputting the verified target message, the receiving end determines the business module associated with the attribute information in the receiving end based on the attribute information of the first data in the second data; and calls the processing interface of the business module with the first data as a parameter to execute the business processing logic corresponding to the business data.

[0095] The attribute information can be a set of fields used to describe the characteristics of the first data, including but not limited to: data length (the size of the first data in bytes), data format (such as JSON, Protobuf, binary), data category (such as control commands / status data / observation data), data version, etc. This information is used to help the receiving end correctly parse and process the first data.

[0096] Business modules can be components or listeners that implement specific business logic within the receiving end. Each business module will pre-register one or more topics that it is interested in.

[0097] The processing interface can be a standard callback method exposed by the business module. The receiving end calls this method to pass the first data to the business module and trigger the corresponding business processing logic.

[0098] The data hub communication method based on the embodiments of this application extracts the target communication topic from the second data and dynamically routes it to the pre-bound business module in the receiving end based on the topic. The receiving end does not need to make lengthy condition judgments to distinguish message types. It only needs to maintain a mapping table from topic to module. When adding a new business scenario, it only needs to register a new module, which improves the modularity of each subsystem in the large distributed telescope system.

[0099] In embodiments of this application, the large distributed telescope system further includes a file transfer service area isolated from the data hub; the method further includes: when the type of business data obtained by the publishing end is a file type, the publishing end uploads the business data to the file transfer service area, obtains the access address and file metadata of the business data in the file transfer service area, and encapsulates the access address and file metadata into a target message; the data hub receives the target message and performs topic permission verification on the target message, and sends the verified target message to the receiving end; the receiving end obtains the business data from the file transfer service area based on the access address in the verified target message, and performs integrity verification on the business data based on the file metadata in the verified target message.

[0100] The file transfer service area can be a storage and distribution infrastructure deployed independently of the data hub, specifically designed for storing large files such as observation images, log packages, and configuration packages. This service area provides file upload, download, access address generation, and metadata management capabilities, and is physically isolated from the message transmission link of the data hub to avoid large files consuming communication bandwidth.

[0101] The access address is a unique access path generated by the file transfer service area after business data is uploaded to it.

[0102] File metadata is a collection of data describing business data attributes and verification information, including filename, file size, upload time, file hash value, file type, and fragmentation information. The file hash value is used by the receiving end to verify file integrity.

[0103] The data hub communication method based on this application introduces a file transfer service area isolated from the data hub. The data hub only transmits file access addresses and metadata, avoiding the occupation of real-time communication links by large files such as observation images and log packets. This effectively prevents transmission delays caused by bandwidth contention for small messages such as control commands and status synchronization, ensuring the real-time response performance of the telescope system under continuous observation tasks. The receiving end independently verifies file integrity using the file hash value in the metadata, mitigating the risk of damage during file transmission due to network fluctuations or storage medium errors.

[0104] In some embodiments, the subscriber (such as a data analysis subsystem or a status monitoring subsystem) sends a subscription request to the data hub. The subscriber can flexibly choose the starting position of message subscription according to business needs. The starting position includes three options: receiving from the current time, receiving from the earliest available message in the backend, and receiving from a specified observation time point.

[0105] After receiving a subscription request, the data hub parses out the subscription start position.

[0106] If it is the current time or the earliest message, the message will be pushed directly from the corresponding position.

[0107] If a specific observation point is specified, the data hub converts the observation time point in the subscription request into a unified timestamp. Subsequently, based on the configuration of the subscribed communication topic, the data hub selects a target adapter bound to a communication backend with historical read capabilities and calls the adapter's historical read interface, passing in the target communication topic and timestamp parameters. The target adapter converts the timestamp into query parameters recognizable by the communication backend and calls the timestamp location interface provided by the communication backend. The communication backend locates the first message position corresponding to the timestamp in persistent storage based on the timestamp index. Starting from the message position located by the backend, the target adapter pulls historical messages in batches. The target adapter converts the pulled historical messages into a unified message format and pushes them to the subscriber end one by one or in batches through the data hub's unified communication interface. After receiving the messages, the subscriber end performs integrity verification and business processing, enabling observation process replay, fault recovery, or task interruption resumption.

[0108] Figure 4A block diagram of an electronic device suitable for implementing a data hub communication method for a large distributed telescope system, according to an embodiment of this application, is illustrated schematically.

[0109] like Figure 4 As shown, an electronic device 400 according to an embodiment of this application includes a processor 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory 402 or a program loaded from a storage portion 408 into a random access memory 403. The processor 401 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a dedicated microprocessor. The processor 401 may also include onboard memory for caching purposes. The processor 401 may include a single processing unit or multiple processing units for executing different steps of the method flow according to an embodiment of this application.

[0110] Random access memory 403 stores various programs and data required for the operation of electronic device 400. Processor 401, read-only memory 402, and random access memory 403 are interconnected via bus 404. Processor 401 executes various steps of the method flow according to embodiments of this application by executing programs stored in read-only memory 402 and / or random access memory 403. It should be noted that the programs may also be stored in one or more memories other than read-only memory 402 and random access memory 403. Processor 401 may also execute various steps of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0111] According to embodiments of this application, the electronic device 400 may further include an input / output interface 405, which is also connected to a bus 404. The electronic device 400 may also include one or more of the following components connected to the input / output interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a cathode ray tube, liquid crystal display, etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card, such as a local area network card, modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the input / output interface 405 as needed. A removable medium 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 410 as needed so that computer programs read from it can be installed into the storage section 408 as needed.

[0112] Embodiments of this application also provide a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0113] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include the read-only memory 402 described above, and / or random access memory 403, and / or one or more memories other than read-only memory 402 and random access memory 403.

[0114] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this application.

[0115] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via communication section 409, and / or installed from removable medium 411. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0116] In embodiments of this application, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by processor 401, it performs the functions defined in the system of this application embodiment. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0117] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0118] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0119] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A data hub communication method for a large distributed telescope system, the large distributed telescope system comprising a data hub and multiple heterogeneous service subsystems communicatively connected to the data hub, characterized in that, Using any one heterogeneous business subsystem as the publishing end and another heterogeneous business subsystem as the receiving end, the method includes: The publishing end obtains its own business data, encapsulates the business data into a target message containing a first integrity verification value, and sends it to the data hub. The data center performs topic permission verification on the target message and sends the target message that passes the verification to the receiving end. The receiving end calculates the second integrity check value of the verified target message, and outputs the verified target message when it is determined that the first integrity check value and the second integrity check value meet the preset conditions.

2. The data hub communication method according to claim 1, characterized in that, The method further includes: before the data hub performs topic permission verification on the target message. The data hub creates a unified set of communication topics and generates topic permission control rules based on the multiple heterogeneous business subsystems and the business interaction logic between the multiple heterogeneous business subsystems. The communication topic set includes multiple pre-planned communication topics covering all business interaction scenarios in the large distributed telescope system; the topic permission control rules represent the authorization relationship between each communication topic in the communication topic set and the heterogeneous business subsystem in publishing or receiving operations.

3. The data hub communication method according to claim 2, characterized in that, The data hub performs topic permission verification on the target message, including: Obtain the target communication topic from the target message; Verify whether the target communication topic exists in the communication topic set, and Based on the topic permission control rules, verify whether the publishing end has obtained authorization to publish the target message to the target communication topic; If it is determined that the target communication topic exists in the communication topic set, and the publishing end is authorized to publish the target message to the target communication topic, then the target message verification is deemed successful.

4. The data hub communication method according to claim 1, characterized in that, The step of encapsulating the business data into a target message containing a first integrity verification value includes: According to the preset division rules, the business data is divided into first data and second data. The first data represents the semantic information of the business data, and the second data includes identification information for determining the uniqueness of the first data and attribute information of the first data. Calculate the first integrity check value of the first data; The first integrity check value is encapsulated in the second data; The first data and the encapsulated second data are encapsulated again to obtain the target message.

5. The data hub communication method according to claim 1, characterized in that, The method further includes: The data hub creates a unified communication interface based on multiple predefined communication backends and multiple communication modes, and the multiple heterogeneous business subsystems establish a connection with the data hub through the unified communication interface; The data hub determines the communication adapter based on the communication configuration parameters of each of the heterogeneous service subsystems; The publishing terminal sends the target message to the data hub through the unified communication interface; The data hub sequentially sends the verified target message to the receiving end through the unified communication interface and the communication adapter.

6. The data hub communication method according to claim 1, characterized in that, The receiving end calculates a second integrity check value for the verified target message, and outputs the verified target message when it is determined that the first integrity check value and the second integrity check value satisfy a preset condition, including: The verified target message is split into first data and second data. The first data represents the semantic information of the business data, and the second data includes identification information for determining the uniqueness of the first data, attribute information of the first data, and the first integrity verification value. Calculate the second integrity check value of the first data; The first integrity check value is compared with the second integrity check value. When the first integrity check value and the second integrity check value are equal, the target message is confirmed to be complete, and the target message that has passed the check is output.

7. The data hub communication method according to claim 6, characterized in that, The method further includes: When the first integrity check value is not equal to the second integrity check value, the target message is confirmed to be abnormal, the target message is discarded and an alarm is triggered.

8. The data hub communication method according to claim 6, characterized in that, The method further includes: after outputting the target message that has passed the verification. The receiving end determines the service module associated with the attribute information in the receiving end based on the attribute information of the first data in the second data; Using the first data as a parameter, the processing interface of the business module is called to execute the business processing logic corresponding to the business data.

9. The data hub communication method according to claim 1, characterized in that, The large distributed telescope system also includes a file transfer service area isolated from the data hub; the method further includes: When the type of business data obtained by the publishing end is a file type, the publishing end uploads the business data to the file transfer service area, obtains the access address and file metadata of the business data in the file transfer service area, and encapsulates the access address and the file metadata into the target message; The data hub receives the target message and performs topic permission verification on the target message, and sends the target message that passes the verification to the receiving end; The receiving end obtains the service data from the file transfer service area based on the access address in the verified target message, and performs integrity verification on the service data based on the file metadata in the verified target message.

10. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 9.

Images