System for realizing disaster recovery and rapid recovery of national standard superior service by using Redis record cascade and equipment information

Abstract

The invention relates to a system for realizing disaster recovery and rapid recovery of national standard superior services by using Redis record cascade and equipment information. A Redis storage management system is constructed to record cascading and equipment information, and when a service is restarted abnormally, link recovery or waiting for registration online is quickly judged according to Redis information. Multi-thread storage is adopted for searching shared equipment information for the first time, and efficiency is improved. And comparing the Redis with the reported information at the time during secondary retrieval, carrying out difference processing and regularly checking the consistency. And the lower level synchronously retrieves resource information from the upper level, a database and Redis are accurately operated according to information content by means of a Kafka component, and timely and complete data synchronization is guaranteed. All the steps work cooperatively, disaster recovery and rapid recovery are achieved when existing national standard cascade services face various emergencies such as service faults and network anomalies, the overall performance, disaster recovery capacity and data accuracy are improved, and the requirements of a video monitoring system for efficient, stable and reliable operation and accurate resource sharing are met.

Description

[0001] This application is a divisional application of the application with the application number 202411846567.0, the application date of December 16, 2024, and the patent name of "Method for Realizing Rapid Recovery of National Standard Upper-Level Service Disaster Tolerance by Using Redis to Record Cascade and Device Information". Technical Field

[0002] The present invention belongs to the field of secure transmission of video surveillance bitstreams, and particularly relates to a method for realizing rapid recovery of national standard upper-level service disaster tolerance by using Redis to record cascade and device information. Background Art

[0003] During the operation of traditional national standard upper-level services, temporary caches are relied on to record link states. Once the service experiences an abnormal restart, according to national standard regulations, it is necessary to wait for a timeout of approximately 3 minutes before the registration and online process can be restarted to restore the normal link state. For example, in an actual video surveillance system, if the service restarts due to server hardware failures or software errors, the system cannot automatically and quickly restore the link within these 3 minutes, and users cannot perform any operations on cascaded devices, seriously affecting the real-time performance and availability of the system. Especially in scenarios with extremely high real-time requirements such as security surveillance and traffic monitoring, important information may be missed.

[0004] With the widespread application of video surveillance technology, the number of devices connected to the cascade platform has increased significantly, often reaching the million level. When facing the task of initially retrieving shared devices (including manual and automatic retrieval) for such a large number of devices, the traditional cascade upper-level service adopts a strategy of entering device information into the database one by one in a single-threaded manner. Taking 500,000 devices as an example, when performing the initial retrieval and synchronization, it is expected to take up to 6 hours. During this process, a large amount of system resources are occupied, and accurate device information cannot be provided to users in a timely manner, severely restricting the response speed and overall performance of the system, and reducing the practicality and efficiency of the system in actual applications.

[0005] When the cascade platform is connected to a large number of devices, it is relatively common for the information of lower-level devices to change. If the lower level is unable to notify and report device information changes due to certain reasons, to ensure data integrity, the traditional cascade upper-level service will initiate a secondary full-scale retrieval (manual or automatic) synchronization operation, and still adopts the method of entering data one by one in a single-threaded manner. For a device scale of 500,000, the expected time consumption is also about 6 hours. This method not only causes a waste of a large amount of time and computing resources, but also, since all device information is reprocessed and it is impossible to distinguish between changed and unchanged information, further reduces the system efficiency, affects the response timeliness of the system to other operation requests during processing, and has a negative impact on the overall performance and stability of the system.

[0006] In the conventional national standard docking scenario, when some information of the lower-level device changes and supports notification reporting of updates, the lower-level device will report the device change notification to the upper-level service through a SIP message. However, in the case of an unstable network environment, such as large network fluctuations or when the service is offline, the SIP message may be lost or unable to be delivered in time, causing the upper-level service to miss some status update messages, making it impossible to synchronize data between platforms, affecting the system's accurate judgment of the device status, and further interfering with subsequent operational decisions, such as device scheduling and resource allocation, reducing the accuracy and reliability of the system.

[0007] In addition, the existing relevant technologies still have at least the following deficiencies:

[0008] 1. Slow service restart and recovery: The waiting time specified by the national standard is too long, which makes it impossible for users to operate cascade devices normally during the service restart period, which is extremely unfavorable for video surveillance systems with high real-time requirements. In key monitoring scenarios, such as security monitoring and traffic monitoring, 3 minutes of offline time may miss the monitoring and processing of important events, seriously affecting the security and management efficiency of the system, and failing to meet the system's requirements for real-time and continuity.

[0009] 2. Low efficiency of first retrieval: The single-threaded storage method is extremely inefficient when searching for large-scale devices, and cannot meet the system's needs for rapid acquisition of device information. The long retrieval process consumes a lot of system resources, causing users to wait too long when querying device information, seriously affecting the user experience, reducing the practicality and competitiveness of the system, and making the system unable to cope with large-scale device management.

[0010] 3. Serious waste of secondary search resources and inefficiency: The traditional secondary full search method does not distinguish whether the device information has changed, and re-enters all devices into the warehouse, which causes a huge waste of time and computing resources when the number of devices is large. At the same time, due to the long processing time, the system cannot respond to other operation requests in time during the processing period, which seriously affects the overall performance and stability of the system and hinders the efficient operation of the system.

[0011] 4. Change notifications are easily missed and unreliable: The SIP message reporting method has obvious defects when the network fluctuates or the service is offline, which can easily lead to the loss of device change notifications and make the data between platforms out of sync. This may cause the system to misjudge the device status, which in turn affects subsequent operational decisions, such as device scheduling and resource allocation, reducing the accuracy and reliability of the system and failing to ensure the consistency and integrity of system data.

[0012] Traditional national standard cascading services face severe challenges in the above-mentioned multiple key aspects. From the timeliness of service restart and recovery, to the efficiency of device retrieval and synchronization, and then to the reliability of device information change notifications, they cannot meet the strict requirements of modern video surveillance systems for efficient, stable, and reliable operation. These problems seriously restrict the efficient sharing and accurate synchronization of video surveillance resources between different platforms. There is an urgent need for an innovative solution to effectively address these issues, comprehensively improve the overall performance and disaster tolerance of national standard cascading services, ensure the stable and reliable operation of video surveillance systems, and meet the diverse needs of various industries for video surveillance. Summary of the Invention

[0013] In view of the deficiencies in the current related prior art, the present invention provides a method for achieving rapid disaster recovery of the national standard upper-level service by using Redis to record cascading and device information, aiming to overcome the defects in the existing national standard cascading services such as slow service restart and recovery, low efficiency of device retrieval and synchronization, unreliable device information change notifications, and difficulty in ensuring data consistency. By using Redis to record cascading and device information and combining technologies such as multi-threading and Kafka, the national standard upper-level service can quickly restore the connection state during abnormal restart, improve the first retrieval efficiency, optimize the secondary retrieval process to save resources, ensure the timeliness and integrity of device information change notifications with the help of Kafka, and perform regular verification to ensure the consistency of data between Redis and the database, thereby comprehensively improving the overall performance, disaster tolerance, and data accuracy of the national standard cascading service, and meeting the requirements of modern video surveillance systems for efficient, stable, reliable operation, and accurate resource sharing.

[0014] To achieve the above object, the technical solution adopted by the present invention is: a method for achieving rapid disaster recovery of the national standard upper-level service by using Redis to record cascading and device information, which is characterized in that: through the recording, management of information, and collaborative operations between various processes, the national standard cascading upper-level service can quickly resume normal operation in the face of various emergencies, specifically including the following steps:

[0015] S1: During the operation of the national standard cascading upper-level service, construct a Redis storage management system for platform cascading and device information, classify and record the key information closely related to the service operation status and device management, establish a storage structure convenient for quick query and precise management, and update the information in Redis in a timely manner according to the dynamic changes during service operation, providing basic data support for subsequent service recovery, device retrieval, and information synchronization;

[0016] S2: When the national standard cascaded upper-level service encounters an abnormal restart, quickly perform multi-dimensional status judgments based on the information stored in Redis, make intelligent decisions to restore the cascaded link status or wait for the lower-level to register and go online according to the judgment results, and establish a continuous and stable information update mechanism after successfully restoring the link to ensure the real-time and accurate reflection of the service status and reduce the impact of abnormal restarts on the overall business;

[0017] S3: When performing the first operation of retrieving shared device information, implement an optimized multi-stage device information warehousing process, covering precise classification and processing of information, efficient queue management of organized and device information, and warehousing operations with single-thread and multi-thread cooperation, to improve the efficiency of device information warehousing and save time and server resources;

[0018] S4: When carrying out the task of retrieving shared device information for the second time, use a comparison algorithm to distinguish the differences between the information reported this time and the information recorded in Redis in the past, implement a differential processing strategy based on the comparison results, and establish a timing verification mechanism to ensure the high consistency of device information in Redis and the database, avoid repeated processing of unchanged information, and improve the efficiency of the second retrieval;

[0019] S5: In the process of the lower-level synchronizing and retrieving resource information to the upper-level, use the Kafka component to build an information reporting and sequential consumption channel, perform precise database operations and Redis information updates according to the content of the reported information, ensure the timeliness, continuity and integrity of data synchronization between platforms, and solve the problem of information loss caused by network fluctuations or service anomalies.

[0020] Furthermore, in S1, the key information closely related to the service running status and device management includes the basic platform cascading information and the shared device resource information. Among them, the key-value of the basic platform cascading information is "AS: cascading port", and the internal record includes the cascading link status, the last heartbeat time, the last registration information, the registration validity period, and the basic information of the upper and lower platforms. The shared device resource information includes the device ID, the device network status, and the information of the organization to which the device belongs; use the cascaded service ID, that is, spid, as the primary key of the Redis hash device table, and the device ID as the internal key-value to construct the storage structure. By using binary encoding to represent the cascading link status, recording the heartbeat timestamp as the cascading heartbeat information, and recording the registration time and registration validity period for the lower-level registration and online status management, efficient storage and fast query of information are realized; the cascading status information in Redis is updated according to the heartbeat timing to keep alive, which means that every time the heartbeat signaling arrives, the heartbeat timestamp is immediately updated. If the heartbeat signaling is not received continuously for multiple times, it is automatically determined that the cascade may be abnormal and corresponding handling measures are triggered in a timely manner to ensure the real-time monitoring and effective management of the cascade status.

[0021] Further, in S2, in the recovery process of abnormal restart of the national standard cascaded upper-level service, based on the Redis stored information, key status and conditions are judged, and according to the judgment results, the recovery link is determined or waiting for registration and online. After restoring the link, a stable information update mechanism is established to ensure the continuous and stable operation of the service. The judgment basis includes judging the cascaded status by specific status flags, comparing information conditions with configuration file parameters, and determining the keep-alive time period according to the set time range. The information update mechanism is to update the cascaded service status information to Redis in real time according to the heartbeat.

[0022] Further, the specific implementation method of the cascaded platform quick recovery process at least includes the following steps:

[0023] S11: After the national standard cascaded upper-level service is successfully started for the first time, when the lower level is successfully registered and online for the first time, the cascaded service is successfully linked;

[0024] S12: Record the cascaded online status into Redis, and update the cascaded service status information in Redis according to the heartbeat timing keep-alive mechanism;

[0025] S13: Service abnormal restart;

[0026] S14: Immediately read the cascaded service status information from Redis and the cascaded basic information from the database. If the following conditions are met: (1) The status of the cascade recorded in Redis currently is linked; (2) The conditions of the cascaded basic information read from the database and the cascaded basic information in Redis meet the configuration requirements of this level, that is, the cascaded information in Redis is consistent with the cascaded configuration information in the database; (3) The heartbeat timestamp recorded in Redis is within the keep-alive time period of the cascade heartbeat; if the conditions are met, the service directly resumes the linked state, otherwise wait for the next registration signal from the lower level to register and link online;

[0027] S15: If the conditions in step S14 are met, the service resumes the normal linked state, continues to process the cascaded business normally, and updates the cascaded service status information to Redis in real time according to the heartbeat.

[0028] Further, in S3, when retrieving the shared device information for the first time, through the classification processing and multi-threaded operation of the information reported by the lower level, efficient information warehousing is realized;

[0029] The warehousing process of retrieving the shared device information for the first time includes the following steps:

[0030] S31: After the national standard cascaded upper-level service is started, send a retrieval signal to the lower-level platform;

[0031] S32: The lower-level cascaded platform sequentially reports the retrieved organization information and retrieved device information in hierarchical levels;

[0032] S33. The national standard cascaded upper-level service receives and parses the retrieval information reported by the lower-level cascaded platform, and then determines whether the reported information is an organization. If it is an organization, it is written into the organization queue to be warehoused. If it is a device, it is written into the device queue to be warehoused;

[0033] S34. Repeat step S32 and step S33 until the lower-level cascaded platform finishes reporting retrieval information;

[0034] S35. While the lower-level cascaded platform is reporting retrieval information, there is also a distribution thread for processing retrieval tasks to be processed running simultaneously: including

[0035] (1) First, process the retrieval information. The organization and the device enter the organization queue and the device queue to be warehoused respectively;

[0036] (2) First, through the single-organization warehousing thread, sequentially process all the organization information in the organization queue to be warehoused;

[0037] (3) After synchronously waiting for all organizations to be warehoused, start to process the device information in the device queue to be warehoused by multiple threads;

[0038] In the overall process, except that the warehousing of organization information is a single-thread operation, after the organizations are warehoused, multi-threaded operation for device warehousing begins;

[0039] Intelligently allocate different thread pools according to the device type, and the number of threads in each thread pool is dynamically optimized and adjusted according to the device quantity and the real-time performance of the system.

[0040] Further, during the secondary retrieval in S4, use an algorithm to compare the information reported this time with the past record information in Redis, warehouse the changed or first-reported information into the database and record it in Redis, and discard the unchanged information, and regularly verify the consistency of the device information between Redis and the database.

[0041] Further, the optimization process for sharing device information in the secondary retrieval includes the following steps:

[0042] The optimization process for sharing device information in the secondary retrieval includes step S41 and / or step S42, where

[0043] Step S41 is as follows:

[0044] S411: After the national standard cascaded upper-level service is successfully started, send the retrieval signaling for the first time, and the lower-level cascaded platform reports the retrieval response.

[0045] S412: Process the first retrieval response, warehouse the information into the database, and after the warehousing is completed, record the information into the device information hash table of the Redis service.

[0046] S413: The national standard cascaded upper-level service reissues the retrieval signaling for the second time, and the lower-level cascaded platform reports the retrieval response.

[0047] S414: After receiving the retrieval device response information, compare the current report with the previous report information recorded in Redis according to the device ID to check if there is any change or if it is the first report.

[0048] S415: If the information has changed or it is the first report, store the information in the database and, after the storage is completed, promptly record the information in the device information hash table of the Redis service. If the information has not changed, directly discard this message and continue to process the next message.

[0049] The steps of S42 are as follows:

[0050] S421: After the full second-level retrieval is completed, start to regularly verify the consistency between the local cascaded device information recorded in Redis and the local cascaded device information recorded in the database. The verification period can be flexibly adjusted according to the real-time load of the system and the importance of the data.

[0051] S422: When verifying, use the device data in the database as the standard. If data inconsistency is found, accurately adjust the information in Redis according to the accurate device data in the database to ensure the accuracy and integrity of the data.

[0052] Furthermore, the Kafka retrieval information reporting process of the national standard cascaded upper-level service includes step S51 and step S52;

[0053] The steps of S51 are as follows:

[0054] S511: The national standard cascaded upper-level service starts successfully.

[0055] S512: The lower-level cascaded platform actively generates and reports device retrieval information. Set the number of partitions of the Kafka message queue according to the number of lower-level devices and the data traffic, and send the device retrieval information into the Kafka message queue. The messages within each partition are stored in chronological order.

[0056] S513: The national standard cascaded upper-level service consumes the device retrieval message data from Kafka in the order of offsets. If the upper-level service restarts abnormally, rely on the sequential consumption feature of Kafka to continue consuming the data in the Kafka message queue from the offset position where the interruption occurred last time.

[0057] The steps of S52 are as follows:

[0058] S521: The national standard cascaded upper-level service judges the received retrieval device response information; compare the current report with the previous report information recorded in Redis according to the device ID to check if there is any change; or if it is the first report.

[0059] S522: If the information has changed or is reported for the first time, the information is stored in the database. After the database storage is completed, the information is recorded in the device information hash table of the Redis service; if the information has not changed, the message is directly discarded and the next message is processed, so as to ensure the timeliness, continuity and integrity of data synchronization between platforms.

[0060] Furthermore, the above steps work in coordination with each other. The information recorded in S1 provides a key judgment basis for the service restart and recovery in S2, provides a basic data query source for the device retrieval and synchronization in S3 and S4, and provides a device information reference for the change notification in S5; after restoring the link status in S2, it interacts with the lower-level service to ensure normal device operation and data transmission, and collaborates with the continuous update of information in Redis to ensure the overall operation of the system; the efficient storage in S3 provides fast data support for subsequent device information management and retrieval; the information processing and verification in S4 ensure the accuracy and consistency of device information and provide reliable data for other processes; the information reporting and processing in S5 ensure the timely synchronization of data between platforms and provide guarantee for the collaboration of the whole system, jointly realizing the comprehensive improvement of the national standard upper-level service in disaster recovery and recovery, device information management and platform collaboration.

[0061] The present invention adopts the above technical solutions and has at least the following beneficial effects:

[0062] 1. In terms of service restart and recovery, different from the traditional technical solution that relies on caching to record the link status and needs to wait for three minutes for the lower-level timeout registration to restore the normal link status, the present invention uses Redis to record and read the link status information. When the national standard cascaded upper-level service encounters an abnormal restart, it can quickly make multi-dimensional status judgments based on the information stored in Redis, and make intelligent decisions to restore the cascaded link status or wait for the lower-level to register and go online according to the judgment results. After successfully restoring the link, a continuous and stable information update mechanism is established. This method greatly shortens the service recovery time, ensures the real-time and accurate reflection of the service status, effectively reduces the impact of abnormal restart on the overall business, significantly improves the business interruption situation, has a more efficient and stable disaster recovery and recovery means, and ensures the quick restoration of the normal link status of the platform after restart.

[0063] 2. For the first retrieval of shared device information, the old technology uses a single-threaded one-by-one warehousing method to perform the first full retrieval and warehousing, which consumes a lot of time and server resources. The present invention implements an optimized multi-stage device information warehousing process, covering accurate classification and processing of information, efficient queue management of organization and device information, and single-threaded and multi-threaded collaborative warehousing operations. Through reasonable task allocation and thread collaboration, especially after the single-threaded warehousing of organization information is completed, different thread pools are intelligently allocated according to the device type for multi-threaded device warehousing, and the number of threads in each thread pool is dynamically optimized and adjusted according to the number of devices and the real-time performance of the system, which significantly improves the efficiency of device information warehousing, greatly shortens the time spent on the first retrieval and warehousing, makes the first retrieval and warehousing more efficient and fast, and effectively saves time and server resources.

[0064] 3. In the task of secondary retrieval of shared device information, the old technical solution performs full warehousing and updates regardless of whether the device resource information has changed during the secondary full retrieval, resulting in a waste of resources. The present invention uses a comparison algorithm to distinguish the difference between the information reported this time and the previously recorded information in Redis, and implements a differentiated processing strategy based on the comparison results. Only the changed or first reported information is entered into the database and recorded in Redis, and the unchanged information is discarded. A timed verification mechanism is established to ensure that the device information in Redis is highly consistent with that in the database. This method avoids repeated processing of unchanged information, effectively improves the efficiency of secondary retrieval, does not occupy database resources by repeated entry, makes secondary retrieval and entry more efficient and fast, and at the same time ensures the accuracy and consistency of the data.

[0065] 4. In terms of the process of synchronously retrieving resource information from the lower level to the upper level, compared with the old technology that reports the lower level retrieval change notification through SIP messages, which is prone to packet loss and missing synchronization information due to network fluctuations or service restarts, the present invention uses Kafka components to build information reporting and sequential consumption channels, and performs accurate database operations and Redis information updates according to the reported information content. Kafka's sequential consumption characteristics ensure that even in the case of network fluctuations or abnormal service restarts, the upper level service can continue to accurately consume the data in the Kafka message queue from the offset position of the last interruption, ensuring the timeliness, continuity and integrity of data synchronization between platforms, and successfully solving the problem of frequent and time-consuming searches by the upper level for synchronization when the lower level shared resources are changed, so that important resource sharing is not affected by network fluctuations or abnormal service restarts, greatly improving the data reliability and stability of the entire national standard cascade service system.

[0066] In summary, the present invention comprehensively improves the overall performance, disaster tolerance ability, and data accuracy of the national standard cascading service, effectively meeting the requirements of modern video surveillance systems for efficient, stable, reliable operation, and accurate resource sharing, and having significant progressive significance and broad application prospects in the relevant technical fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0068] Figure 1 is the flowchart of the rapid recovery process of the cascading platform of the present invention;

[0069] Figure 2 is the flowchart of the multi-threaded storage of the cascading platform retrieval of the present invention;

[0070] Figure 3 is the flowchart of the rapid secondary retrieval of the cascading platform of the present invention;

[0071] Figure 4 is the flowchart of the cascading platform reporting retrieval and storage through Kafka of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0072] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. On the contrary, they are only examples of the devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0073] Embodiment 1

[0074] Please refer to Figure 1 , this embodiment provides a method for realizing rapid disaster recovery of the upper-level service of the national standard by using Redis to record cascading and device information. Through the recording, management of information, and collaborative operations between various processes, the upper-level service of the national standard cascading can quickly resume normal operation in the face of various emergencies, specifically including the following steps:

[0075] S1: During the operation of the national standard cascaded upper-level service, build a Redis storage management system for platform cascading and device information, classify and record the key information closely related to the service operation status and device management, establish a storage structure for quick query and precise management, and update the information in Redis in a timely manner according to the dynamic changes during service operation, providing basic data support for subsequent service recovery, device retrieval, and information synchronization;

[0076] S2: When the national standard cascaded upper-level service encounters an abnormal restart, quickly make multi-dimensional status judgments based on the information stored in Redis, make intelligent decisions to restore the cascaded link status or wait for the lower-level to register and go online according to the judgment results, and establish a continuous and stable information update mechanism after successfully restoring the link to ensure the real-time and accurate reflection of the service status and reduce the impact of abnormal restart on the overall business;

[0077] S3: When performing the first retrieval of shared device information operation, implement an optimized multi-stage device information warehousing process, covering precise classification processing of information, efficient queue management of organization and device information, and warehousing operations with single-thread and multi-thread cooperation, improving the efficiency of device information warehousing and saving time and server resources;

[0078] S4: When carrying out the second retrieval of shared device information task, use a comparison algorithm to distinguish the differences between the information reported this time and the information recorded in the past in Redis, implement a differential processing strategy according to the comparison results, and establish a timing verification mechanism to ensure the high consistency of device information in Redis and the database, avoiding repeated processing of unchanged information and improving the efficiency of the second retrieval;

[0079] S5: In the process of the lower-level synchronizing and retrieving resource information to the upper-level, build an information reporting and sequential consumption channel with the help of Kafka components, perform precise database operations and Redis information updates according to the reported information content, ensure the timeliness, continuity, and integrity of data synchronization between platforms, and solve the problem of information loss caused by network fluctuations or service exceptions.

[0080] As an implementation manner, in S1 of this embodiment, the key information closely related to the service running status and device management includes the basic platform cascading information and the shared device resource information. The key value of the basic platform cascading information is "AS: cascading port", and the internal record includes the cascading link status, the last heartbeat time, the last registration information, the registration validity period, and the basic information of the upper and lower platforms; the shared device resource information includes the device ID, the device network status, and the organization information to which the device belongs. Using the cascaded service ID, that is, spid, as the primary key of the Redis hash device table and the device ID as the internal key value to construct the storage structure, by using binary encoding to represent the cascading link status, recording the heartbeat timestamp as the cascading heartbeat information, recording the registration time and the registration validity period, and recording the basic information of the upper and lower platforms for the management of the lower-level registration and online status, the efficient storage and fast query of information are realized; the cascading status information in Redis is updated according to the heartbeat timing keep-alive, which means that every time the heartbeat signaling arrives, the heartbeat timestamp is immediately updated. If the heartbeat signaling is not received continuously for multiple times, it is automatically determined that the cascade may be abnormal and the corresponding processing measures are triggered in a timely manner to ensure the real-time monitoring and effective management of the cascade status.

[0081] As an implementation manner, in S2 of this embodiment, in the recovery process of the abnormal restart of the national standard cascading upper-level service, based on the Redis stored information, key status and condition judgments are made, and according to the judgment results, the recovery link is determined or waiting for registration and online. After the recovery link is established, a stable information update mechanism is established to ensure the continuous and stable operation of the service. The judgment bases include using specific status flag bits to judge the cascade status, comparing information conditions with the configuration file parameters, and determining the keep-alive time period according to the set time range. The information update mechanism is to update the cascade service status information to Redis in real time according to the heartbeat.

[0082] As Figure 1 shown, as an implementation manner, the specific implementation method of the rapid recovery process of this embodiment's cascading platform at least includes the following steps:

[0083] S11: After the national standard cascading upper-level service is successfully started for the first time, when the lower level is successfully registered and online for the first time, the cascading service is successfully linked;

[0084] S12: Record the cascading online status into Redis, and update the cascading service status information in Redis according to the heartbeat timing keep-alive mechanism;

[0085] S13: The service is abnormally restarted;

[0086] S14: Immediately read the cascade service status information from Redis and the cascade basic information from the database. If the following conditions are met: (1) The status of the cascade recorded in the current Redis is "linked"; (2) The conditions of the cascade basic information read from the database are compared with the cascade basic information in Redis and meet the requirements of the current cascade configuration, that is, the cascade information in Redis is consistent with the cascade configuration information in the database; (3) The heartbeat timestamp recorded in Redis is within the keep-alive time period of the cascade heartbeat. If the conditions are met, the service directly resumes the linked state. Otherwise, wait for the next registration signal from the lower level to register and go online;

[0087] S15. If the conditions in step S14 are met, the service resumes the normal linked state, continues to process the cascade service normally, and updates the cascade service status information to Redis in real time according to the heartbeat.

[0088] As Figure 2 shown, as an implementation, in S3 of this embodiment, when retrieving the shared device information for the first time, through the classification processing and multi-threaded operation of the information reported by the lower level, efficient information storage in the database is achieved;

[0089] The process of storing the retrieved shared device information in the database for the first time includes the following steps:

[0090] S31. After the national standard cascade upper-level service is started, it sends a retrieval signal to the lower-level platform.

[0091] S32. The lower-level cascade platform sequentially reports the retrieved organization information and retrieved device information in hierarchical levels.

[0092] S33. The national standard cascade upper-level service receives and parses the retrieved information reported by the lower-level cascade platform, and then determines whether the reported information is an organization. If it is an organization, it writes it into the organization queue to be stored in the database. If it is a device, it writes it into the device queue to be stored in the database.

[0093] S34. Repeat steps S32 and S33 until the lower-level cascade platform has finished reporting the retrieved information.

[0094] S35. While the lower-level cascade platform is reporting the retrieved information, there is also a distribution thread for processing the retrieved tasks to be processed running simultaneously: including

[0095] (1) First, process the retrieved information. The organization and the device respectively enter the organization queue and the device queue to be stored in the database.

[0096] (2) First, through a single-organization storage thread, sequentially process all the organization information in the organization queue to be stored in the database.

[0097] (3) Synchronously wait until all the organizations are stored in the database, and then start to process the device information in the device queue to be stored in the database through multi-threading.

[0098] In the overall process, except for the inbound organization information which is a single-threaded operation, after the organization inbound is completed, multi-threaded operation for equipment inbound starts;

[0099] According to the equipment type, different thread pools are intelligently allocated, and the number of threads in each thread pool is dynamically optimized and adjusted according to the equipment quantity and the system's real-time performance.

[0100] As an implementation manner, in the secondary retrieval of S4 in this embodiment, an algorithm is used to compare the information reported this time with the information recorded in Redis in the past. The information with changes or the information reported for the first time is stored in the database and recorded in Redis, while the information without changes is discarded, and the consistency between the Redis and the equipment information in the database is periodically verified.

[0101] As Figure 3 shown, as an implementation manner, the optimization process of the secondary retrieval of shared equipment information in this embodiment includes the following steps:

[0102] The optimization process of the secondary retrieval of shared equipment information includes step S41 and / or step S42, where

[0103] Step S41 is as follows:

[0104] S411: After the national standard cascaded upper-level service starts successfully, the retrieval signaling is sent for the first time, and the lower-level cascaded platform reports the retrieval response.

[0105] S412: Process the first retrieval response, store the information in the database, and after the storage is completed, record the information in the equipment information hash table of the Redis service.

[0106] S413: The national standard cascaded upper-level service sends the retrieval signaling for the second time, and the lower-level cascaded platform reports the retrieval response.

[0107] S414: After receiving the retrieval equipment response information, compare the information reported this time with the past reported information recorded in Redis according to the equipment ID to check if there is any change or if it is the first report;

[0108] S415: If the information has changed or is the first report, store the information in the database and record the information in the equipment information hash table of the Redis service in a timely manner after the storage is completed; if the information has not changed, directly discard this message and continue to process the next message.

[0109] Step S42 is as follows:

[0110] S421: After the full secondary retrieval is completed, start to periodically verify the consistency between the local cascaded equipment information recorded in Redis and the local cascaded equipment information recorded in the database. The verification period can be flexibly adjusted according to the system's real-time load and data importance;

[0111] S422: When performing verification, the device data in the database shall be used as the standard. If data inconsistency is found, the information in Redis shall be accurately adjusted in a timely manner based on the accurate device data in the database to ensure the accuracy and integrity of the data.

[0112] As Figure 4 shown, as an implementation manner, the Kafka retrieval information reporting process of the national standard cascaded upper-level service in this embodiment includes step S51 and step S52;

[0113] Step S51 is as follows:

[0114] S511: The national standard cascaded upper-level service is successfully started.

[0115] S512: The lower-level cascaded platform actively produces and reports device retrieval information, sets the number of partitions of the Kafka message queue according to the number of lower-level devices and data traffic, and sends the device retrieval information into the Kafka message queue. The messages within each partition are stored in chronological order.

[0116] S513: The national standard cascaded upper-level service consumes device retrieval message data from Kafka in the order of offsets. If the upper-level service restarts abnormally, relying on the sequential consumption feature of Kafka, it continues to consume the data in the Kafka message queue from the offset position where it was interrupted last time;

[0117] Step S52 is as follows:

[0118] S521: The national standard cascaded upper-level service judges the received retrieval device response information; compares whether there is a change in this report with the previous report information recorded in Redis according to the device ID; or it is the first report;

[0119] S522: If the information has changed or it is the first report, the information shall be stored in the database. After the database storage is completed, the information shall be recorded in the device information hash table of the Redis service; if the information has not changed, the message shall be directly discarded and the next message shall be processed continuously to ensure the timeliness, continuity and integrity of data synchronization between platforms.

[0120] As an implementation manner, the steps in this embodiment cooperate with each other. The information recorded in S1 provides a key judgment basis for the service restart and recovery in S2, a basic data query source for the device retrieval and synchronization in S3 and S4, and a device information reference for the change notification in S5; after restoring the link state in S2, it interacts with the subordinate services to ensure the normal operation of device operations and data transmission, and continuously updates information with Redis to jointly ensure the overall operation of the system; the efficient warehousing in S3 provides fast data support for subsequent device information management and retrieval; the information processing and verification in S4 ensure the accuracy and consistency of device information and provide reliable data for other processes; the information reporting and processing in S5 ensure the timely synchronization of data between platforms and provide guarantee for the collaboration of the entire system, jointly achieving a comprehensive improvement in the disaster recovery, device information management, and platform collaboration of the national standard superior service.

[0121] The present invention has at least the following innovative highlights:

[0122] 1. Efficient service disaster tolerance and fast recovery based on Redis;

[0123] Innovatively, the online status backup of the superior cascading service link is stored in Redis, and at the same time, the superior restart registration and online logic is improved. With the storage advantages of Redis and the optimized logic, when the service encounters an abnormal restart, it can break through the traditional limitations and quickly restore the normal link state of the superior within just 5 seconds, greatly shortening the service interruption duration, effectively guaranteeing the continuity of the business, significantly enhancing the disaster tolerance ability of the entire national standard cascading service and the system stability, ensuring that the business can resume normal operation in a short time, and avoiding business disruptions and adverse effects caused by restarts.

[0124] 2. Multithreading helps to efficiently speed up the first retrieval and warehousing;

[0125] In the superior retrieval and warehousing link, the traditional single-threaded method of warehousing one by one is abandoned, and the key innovative measure of using multithreading to achieve device warehousing is introduced. By reasonably planning thread allocation, intelligently configuring different thread pools according to device types, and dynamically optimizing the number of threads in each thread pool in combination with the real-time performance of the system, the speed of the first retrieval and warehousing is greatly improved. For example, in the scenario of retrieving and warehousing up to tens of thousands of devices, compared with the traditional method, the scheme of the present invention significantly speeds up the warehousing process, effectively saves a large amount of time and server resources, greatly improves the first retrieval efficiency, and lays a good data foundation for subsequent device information management and related business operations.

[0126] 3. Redis national standard cascading superior service storage structure optimizes the secondary retrieval process;

[0127] A unique storage format is designed, using the cascaded service ID, i.e., spid, as the primary key of the Redis hash device table and the device ID as the internal key value, to accurately and efficiently back up the device information shared by the lower-level platform to this platform in Redis (cooperative database). This innovative storage structure brings significant optimization to the secondary retrieval process. Especially when the device information shared by the lower level changes and the upper-level platform conducts a secondary full-scale retrieval, relying on this storage structure and the corresponding data processing mechanism, the system can accurately identify the information that has been stored in the database and has not changed, avoid duplicate storage, and only process the incrementally changed information, thus greatly shortening the time-consuming of the secondary retrieval, saving the time consumed by the duplicate storage process, improving the efficiency of the secondary retrieval and the overall accuracy of device information management, and ensuring more efficient and accurate processing of device information.

[0128] IV. Kafka ensures the integrity and timeliness of the synchronization of national standard cascaded upper-level service guarantee resource information;

[0129] Innovatively transfer the way for the lower level to synchronize and retrieve resource information to the upper level to the Kafka component, and construct a reporting method adapted to it. Through the powerful sequential consumption feature of Kafka, when the device information shared by the lower level changes and is reported, it can effectively avoid the problem of missing changed information that was prone to occur in the past, ensure that the upper-level platform can receive all change notifications comprehensively and in a timely manner, and achieve a more complete and timely update of the shared device resource information. This method ensures the timeliness, continuity, and integrity of data synchronization between platforms, makes the entire system more reliable and efficient in resource information sharing, effectively improves the overall efficiency of resource management, and meets the strict requirements of modern video surveillance systems for accurate resource sharing.

[0130] In summary, the technical solution provided in this embodiment, with its innovative highlights, has comprehensively and deeply optimized and upgraded the national standard cascaded service from multiple core dimensions such as service disaster recovery and restoration, retrieval and storage efficiency, secondary retrieval accuracy, and resource information synchronization. These highlights cooperate with each other and synergistically enhance the overall performance, disaster tolerance ability, and data accuracy of the national standard cascaded service, showing excellent advantages in the same field and better meeting the high standards of modern video surveillance systems for efficient, stable, and reliable operation and accurate resource sharing.

[0131] Embodiment 2

[0132] Note: No new drawings are provided separately in this Embodiment 2. The relevant drawings provided in Embodiment 1 can be referred to in this Embodiment 2.

[0133] This embodiment provides a specific implementation method for using Redis to record cascaded and device information to achieve rapid disaster recovery of the national standard upper-level service

[0134] I. Overall System Architecture

[0135] The system involved in the present invention mainly includes a national standard cascading upper-level server, a lower-level cascading platform, and related databases and Redis storage systems, and integrates Kafka components for information reporting and consumption. Before the system starts, the following basic configurations need to be completed:

[0136] Configure Redis connection parameters to ensure that the national standard cascading upper-level service can establish a stable connection with Redis. Set appropriate memory allocation strategies to meet the storage requirements of platform cascading and device information. For example, according to the estimated number of cascades and devices, allocate sufficient memory space for Redis, and at the same time enable the persistence function to prevent data loss.

[0137] For the database, design and create relevant data table structures for storing platform cascading information and device resource information, ensuring that data fields can completely record key data such as basic platform cascading information (cascading link status, cascading heartbeat information, lower-level registration and online status) and shared device resource information (device ID, device network status, device organization information).

[0138] Configure the Kafka cluster, and set key parameters such as the number of partitions of the Kafka message queue according to the estimated number of lower-level devices and data traffic. For example, if it is estimated that there are 5000 lower-level devices and the data traffic is large, the number of partitions can be set to 5 (according to the rule of rounding up the number of lower-level devices divided by 1000), and ensure that each partition has enough disk space for storing message data.

[0139] II. Specific Implementation Process of Each Step

[0140] (1) S1: Construct a Redis storage management system;

[0141] When the national standard cascading upper-level service is running, start the Redis storage management module. Use the cascading service ID, that is, spid, as the primary key of the Redis hash device table, and the device ID as the internal key value to construct the storage structure. For example, when a cascading service ID is "002101001732069540" and receives device information with a device ID of "Device_123", create a record item with "Port_001" as the hash table name in Redis,

[0142] "Device_123" as the key.

[0143] The key-value of the platform cascade basic information is "AS: Cascade Port". The internal record includes the cascade link status, the last heartbeat time, the last registration information, the registration validity period, and the basic information of the upper and lower platforms. For the cascade link status, binary encoding is used. For example, "1" represents a normal link, and "0" represents a disconnected link, etc. Whenever the cascade link status changes, the corresponding binary encoding value is updated in a timely manner.

[0144] Record the heartbeat timestamp as the cascade heartbeat information. Set up a heartbeat detection thread to receive the heartbeat signaling from the lower level every 60 seconds (the national standard default), and update the heartbeat timestamp after receiving the heartbeat signaling. If the heartbeat signaling is not received continuously for 3 times (configurable), it is automatically determined that the cascade may be abnormal and the corresponding alarm and processing mechanism is triggered.

[0145] Record the registration time and the registration validity period for the management of the online status of the lower-level registration. When a lower-level cascade platform registers and goes online, record the current time as the registration time, and perform validity period management according to the preset registration validity period (such as 2 hours). Within the validity period, the lower-level platform is regarded as being in a normal registration status. If the renewal operation is not performed after the validity period expires, it is marked as an expired registration status.

[0146] Record other basic information of the cascade upper and lower platforms, such as the SIP server code (upper-level platform code), the lower-level platform code, the registration transport layer protocol, etc. If the service is restarted, the basic information stored in Redis and the basic information in the database can be quickly compared. If the comparison passes, the link can be restored immediately;

[0147] Update the information in Redis in a timely manner according to the dynamic changes during the service operation. For example, when a new device is connected or the network status of the device changes, update the record of the shared device resource information in Redis in a timely manner to ensure the timeliness and accuracy of the information, and provide reliable basic data support for subsequent service recovery, device retrieval, and information synchronization.

[0148] (2) S2: National standard cascade upper-level service abnormal restart recovery process;

[0149] When the national standard cascade upper-level service encounters an abnormal restart, during the startup process, start the abnormal restart recovery. First, read the cascade service status information from Redis. For example, read the cascade service status information in the hash table corresponding to a specific "AS: Cascade Port", and obtain data such as the cascade status, information conditions, and the last heartbeat timestamp.

[0150] Perform multi-dimensional status judgment. Determine whether the current cascade status is a link by reading the previously stored cascade status code. If the status code read is "1", it means that the current cascade is in a linked state.

[0151] Information condition judgment. Compare the cascading information read from Redis with the cascading configuration parameters read from the database. For example, compare whether the online status of the subordinate registration meets the current configuration requirements, and check the integrity and accuracy of the cascading platform information between the superior and the subordinate. If the information such as the SIP server code (superior platform code), subordinate platform code, and registration transport layer protocol in the configuration file is inconsistent with the predefined parameters, it is determined that the information condition is not met.

[0152] Keep-alive time period judgment. Calculate based on the preset keep-alive time period (such as 5 minutes) and the timestamp of the last heartbeat. If the difference between the current time and the timestamp of the last heartbeat is less than the keep-alive time period, it is determined that it is within the keep-alive time period.

[0153] If the above three conditions (the current cascading status is linked, the information condition meets the requirements of the current cascading configuration, and it is within the keep-alive time period) are all met, the service directly resumes the linked state. After resuming the link, start the information update thread, and update the cascading service status information to Redis in real time according to the heartbeat. For example, update information such as the cascading link status and heartbeat timestamp every 60 seconds (consistent with the heartbeat period) to ensure that the information in Redis is synchronized with the actual service status. If any one of the conditions is not met, wait for the next registration signaling from the subordinate to register and go online. During the waiting process, continuously monitor the registration signaling port. Once a registration signaling is received, re-perform the above judgment process until the link is successfully restored or the preset waiting timeout (such as 3 minutes) is reached.

[0154] (3) S3: The process of the first retrieval of shared device information into the database;

[0155] After the national standard cascading superior service is started, it sends a retrieval signaling to the subordinate platform. The signaling content includes the instruction to retrieve device information and related retrieval scopes and conditions (such as retrieving devices under a specific organization or devices of a specific type, etc.).

[0156] After the subordinate cascading platform receives the retrieval signaling, it reports the retrieval organization information and retrieval device information hierarchically in sequence. For example, first report the organizational structure information, including the organization name, organizational hierarchical relationship, etc., and then report the device information under this organization. Each device information includes detailed content such as device ID, device network status, and device affiliated organization information.

[0157] The national standard cascading superior service receives and parses the retrieval information reported by the subordinate cascading platform. Start the parsing thread to parse the received information line by line, and judge whether the reported information is an organization. If the information contains organization characteristic fields such as organization name and organizational level, it is determined as organization information and written into the organization queue to be stored in the database; if the information contains device characteristic fields such as device ID, it is determined as device information and written into the device queue to be stored in the database.

[0158] Repeat the above steps S2 and S3 until the subordinate cascading platform finishes reporting the retrieval information. While the subordinate cascading platform is reporting the retrieval information, start the distribution thread for processing the retrieval pending tasks.

[0159] The distribution thread first preprocesses the received retrieval information and puts the organizations and devices into the corresponding organization queue and device queue to be stored in the database respectively. For example, using the queue data structure, add the organization information to the end of the organization queue in sequence and add the device information to the end of the device queue.

[0160] Start a single-organization storage thread to sequentially process all the organization information in the organization queue to be stored in the database. The single-organization storage thread reads the organization information from the head of the queue and inserts it into the organization information table in the database according to the requirements of the database table structure. For example, insert the organization name, organization level and other information into the corresponding fields of the organization information table. During the insertion process, perform data integrity and legality checks, such as checking whether the organization name is empty and whether the organization level conforms to the preset rules. If data anomalies are found, record error logs and perform corresponding error handling (such as notifying the administrator or performing data repair operations).

[0161] After synchronously waiting for all organizations to be stored in the database (judged by the return result of the database insertion operation or a specific completion flag), start processing the device information in the device queue to be stored in the database using multiple threads. Intelligently allocate different thread pools according to the device type. For example, allocate one thread pool for network camera devices and another thread pool for storage devices. The number of threads in each thread pool is dynamically optimized and adjusted according to the number of devices and the real-time performance of the system. If the current number of devices is large and the system performance permits, increase the number of threads in the thread pool; if the system resources are tight, appropriately reduce the number of threads to avoid system overload. Process the device information storage in parallel using multiple threads. Each thread obtains the device information from the device queue and inserts it into the device information table in the database, and at the same time updates the device information record in Redis to ensure the synchronization of the device information in the database and Redis.

[0162] (4) S4: Process for secondary retrieval of shared device information;

[0163] After the national standard cascading upper-level service is successfully started, send the retrieval signaling for the first time, and the subordinate cascading platform reports the retrieval response. Process the first retrieval response, store the information in the database, and after the storage is completed, record the information in the device information hash table of the Redis service. The operation process is similar to the first retrieval and storage process in S3.

[0164] The national standard cascade upper-level service reissues the retrieval signaling, and the lower-level cascade platform reports the retrieval response. After receiving the retrieval device response information, the comparison module is started. According to the device ID, the current report is compared with the previous report information recorded in Redis to determine whether there is a change or it is the first report. For example, use the CRC32 check algorithm (other efficient algorithms can be selected according to the actual situation) to calculate the hash values of the current report information and the previous record information in Redis. If the hash values are the same, it is determined that the information has not changed; if the hash values are different, it is determined that the information has changed or it is the first report.

[0165] If the information has changed or it is the first report, the information is stored in the database, and after the storage is completed, the information is promptly recorded in the device information hash table of Redis. During the storage process, follow the principles of database transaction processing to ensure the consistency and integrity of the data. If the information has not changed, the message is directly discarded and the next message is continued to be processed. In this way, the repeated processing of unchanged information is avoided, the efficiency of the secondary retrieval is effectively improved, and the database resources are saved.

[0166] After the secondary retrieval is fully completed, the timed verification module is started to begin the timed verification of the consistency between the local cascade device information recorded in Redis and the local cascade device information recorded in the database. The verification period is initially set to 30 minutes and can be flexibly adjusted according to the real-time load of the system and the importance of the data. For example, when the system load is light and the data importance is high, the verification period can be shortened to 15 minutes; when the system load is heavy, the verification period can be appropriately extended to 1 hour. When verifying, the device data in the database is used as the standard. If data inconsistency is found, the information in Redis is accurately adjusted in a timely manner according to the accurate device data in the database to ensure the accuracy and integrity of the data. By comparing the device information records in the database and Redis, and comparing field by field, if differences are found, such as the device network status is "online" in the database but "offline" in Redis, the information in the database is used as the standard to update the device network status information in Redis.

[0167] (V) S5: Kafka national standard cascade upper-level service retrieval information reporting process;

[0168] After the national standard cascade upper-level service is successfully started, it enters the Kafka information processing module. The lower-level cascade platform actively produces and reports device retrieval information, sets the number of partitions of the Kafka message queue according to the number of lower-level devices and data traffic, and sends the device retrieval information into the Kafka message queue. The messages in each partition are stored in chronological order. For example, when a lower-level cascade platform reports the retrieval information of a device, according to the previously configured partition rules, the information is sent to the corresponding partition. During the sending process, timestamp information is added to ensure that the messages are arranged in chronological order within the partition.

[0169] The national standard cascaded upper-level service consumes device retrieval message data from Kafka in the order of offsets. The Kafka consumption thread is started to read message data from the offset position where the last consumption ended. If the upper-level service restarts abnormally, relying on the sequential consumption feature of Kafka, it continues to consume the data in the Kafka message queue from the offset position where the interruption occurred last time. For example, if the last consumed message has an offset of 100, after restarting, the consumption thread will automatically start consuming from the message with offset 101, ensuring that data is not lost and the consumption order is correct.

[0170] The national standard cascaded upper-level service judges the retrieved device response information received. According to the device ID, it compares the current report with the previous report information recorded in Redis to check if there are any changes; or if it is the first report, the judgment logic is similar to the secondary retrieval information judgment in S4. If the information has changed or it is the first report, the information is stored in the database, and after the database storage is completed, the information is recorded in the device information hash table of the Redis service; if the information has not changed, the message is directly discarded, and the next message is processed, so as to ensure the timeliness, continuity, and integrity of data synchronization between platforms. In the whole process, through the high reliability and sequential consumption feature of Kafka, the problem of information loss caused by network fluctuations or service exceptions is effectively solved, ensuring that the change notifications of subordinate shared resources can be accurately received and processed by the upper-level service.

[0171] III. Cooperative working mechanism of each step;

[0172] During the operation of the entire system, the Redis storage management system constructed in S1 provides core data support for other steps. For example, the abnormal restart recovery process in S2 depends on the cascaded status information, heartbeat information, and registration online information recorded in S1. By reading this information, key status and condition judgments are made to determine the recovery link or the strategy for waiting for registration online.

[0173] The device retrieval and synchronization operations in S3 and S4 are both based on the device information stored in S1 as the basic data query source. When retrieving and storing for the first time (S3), the device information reported by the subordinate can be quickly located and classified according to the organization information in S1, improving the storage efficiency; when performing secondary retrieval (S4), by comparing the previous device information recorded in S1 with the current reported information, differential processing is achieved, avoiding duplicate work.

[0174] In the Kafka retrieval information reporting process of S5, the device information in S1 is also referred to for information comparison and processing. At the same time, the device information change notifications reported in S5 will timely update the Redis storage information in S1 to ensure the real-time and consistency of data.

[0175] After restoring the link state in S2, perform normal interaction operations with the subordinate services, such as receiving device information reported by the subordinates, sending control instructions, etc., to ensure normal device operations and data transmission. And during the interaction process, continuously interact with Redis, update the information in Redis in a timely manner according to the changes in the device state, and cooperate with the continuous update of information in Redis to ensure the stability and reliability of the overall operation of the system.

[0176] The efficient warehousing in S3 provides fast data support for subsequent device information management and retrieval (secondary retrieval in S4). For example, during the secondary retrieval, since the first retrieval and warehousing are completed efficiently, relatively complete and accurate device information has been stored in the database, enabling the secondary retrieval to quickly perform information comparison and processing, improving the overall retrieval efficiency.

[0177] The information processing and verification in S4 ensure the accuracy and consistency of the device information, providing a reliable data basis for other processes. For example, accurate device information helps the abnormal restart recovery process in S2 make correct decisions, and also provides the correct data basis for information warehousing in S3 and information reporting and processing in S5, avoiding system failures or information synchronization errors caused by data errors.

[0178] The information reporting and processing in S5 ensure the timely synchronization of data between platforms, providing a guarantee for the collaboration of the entire system. For example, when the information of subordinate devices changes, it is reported to the superior service in a timely manner through Kafka, and the superior service can quickly respond, update the information in the database and Redis, and notify other relevant modules to perform corresponding processing, thereby realizing the efficient collaboration of the entire national standard cascading service system and jointly achieving an overall improvement in the disaster recovery and recovery, device information management, and platform - to - platform collaboration of the national standard superior service.

[0179] Through the above - mentioned specific implementation manners, the present invention can effectively overcome many problems existing in the existing national standard cascading service, realize efficient, stable, and reliable service operation and resource sharing, and has high feasibility and practicality in actual application scenarios.

[0180] Although the embodiments of the present invention have been shown and described above, it can be understood that the above - mentioned embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above - mentioned embodiments within the scope of the present invention.

Claims

1. Use Redis to record cascade and device information to implement the national standard upper service disaster recovery rapid recovery system, which is characterized by: It includes the national standard cascade upper server, lower cascade platform, database and Kafka components, as follows: The Redis storage management system is configured as follows: when the national standard cascade upper-level service is running; classify and record the basic information of the platform cascade and the shared device resource information; build a storage structure with the cascade service ID as the primary key and the device ID as the internal key value; Record the heartbeat timestamp as cascade heartbeat information; record the registration time and registration validity period for the management of the lower-level registration and online status; update the information in Redis in a timely manner according to the dynamic changes in the service operation; The abnormal restart recovery module is configured as follows: when the service is abnormally restarted, if (1) the cascade link status recorded in the Redis database is connected, (2) the cascade information read from Redis and the cascade configuration parameters read from the database are met at the same time, and (3) the heartbeat timestamp is within the preset keep-alive time period, the service directly restores the link status; The first search and storage module is configured as follows: after the national standard cascade upper service is started, a search signal is sent to the lower platform; after the lower cascade platform receives the search signal, it reports the search organization information and the search device information in order and in layers; the national standard cascade upper service receives and parses the search information reported by the lower cascade platform; while the lower cascade platform reports the search information, it starts the distribution thread for processing the search pending tasks; The secondary search module is configured as follows: the national standard cascade upper service sends the search signaling for the second time, and the lower cascade platform reports the search response. After receiving the response information of the search device, the comparison module is started; if the information has changed or is reported for the first time, the information is stored in the database, and after the storage is completed, the information is recorded in the Redis device information hash table in a timely manner; The Kafka information processing module is configured as follows: the lower-level cascade platform actively produces and reports device retrieval information, sets the number of partitions of the Kafka message queue according to the number of lower-level devices and data flow, and sends the device retrieval information to the Kafka message queue. The messages in each partition are stored in chronological order. The national standard cascade upper-level service consumes device retrieval message data from Kafka in offset order. The national standard cascade upper-level service judges the received retrieval device response information. If the information has changed or is reported for the first time, the information will be stored in the database, and after the database storage is completed, the information will be recorded in the device information hash table of the Redis service.

2. According to claim 1, the system for realizing national standard upper-level service disaster recovery and rapid recovery by using Redis record cascade and device information is characterized in that: In the abnormal restart recovery module, after the link status is restored, the information update thread is started, and the cascade service status information is updated to Redis in real time according to the heartbeat; the specific implementation method of the cascade platform fast recovery process includes at least the following steps: S11: After the national standard cascade upper service is successfully started for the first time, when the lower level is successfully registered and launched for the first time, the cascade service is successfully linked; S12: Record the cascade online status into Redis, and update the cascade service status information in Redis according to the heartbeat timing keep-alive mechanism; S13: Service abnormal restart; S14: Immediately read the cascade service status information from Redis and the cascade basic information from the database if the following conditions are met: (1) the current cascade status recorded in Redis is linked; (2) the cascade basic information conditions read from the database and the cascade basic information in Redis meet the requirements of the current cascade configuration, that is, the cascade information in Redis is consistent with the cascade configuration information in the database; (3) the heartbeat timestamp recorded in Redis is in the keep-alive time period of the cascade heartbeat; if the conditions are met, the service directly restores the link status, otherwise waits for the next registration signaling from the subordinate to register the link online; S15. If the condition of step S14 is met, the service resumes the normal connection state, continues to process the cascaded business normally, and updates the cascaded service status information to Redis in real time according to the heartbeat.

3. According to claim 1, the system for realizing national standard upper-level service disaster recovery and rapid recovery by using Redis record cascade and device information is characterized in that: In the first retrieval and storage module, after the national standard cascade upper service receives and parses the retrieval information reported by the lower cascade platform, it determines whether the reported information is an organization. If it is an organization, it is written into the queue of organizations to be stored; if it is a device, it is written into the queue of devices to be stored; after all organizations have been stored, multi-threaded processing of device information in the queue of devices to be stored is started, and different thread pools are intelligently allocated according to the device type. The number of threads in each thread pool is dynamically optimized and adjusted according to the number of devices and the real-time performance of the system; The initial retrieval process for shared device information includes the following steps: S31, after the national standard cascade upper service is started, a search signal is sent to the lower platform; S32, the lower-level cascade platform reports the retrieval organization information and retrieval device information in a hierarchical order; S33, the national standard cascade upper service receives and parses the search information reported by the lower cascade platform, and then determines whether the reported information is an organization. If it is an organization, it is written into the waiting-to-enter organization queue; if it is a device, it is written into the waiting-to-enter device queue; S34, repeating steps S32 and S33 until the lower-level cascade platform completes reporting the search information; S35. While the search information is being reported to the lower-level cascade platform, a distribution thread for processing the search pending tasks is also running at the same time: including (1) First, the retrieval information is processed, and the organization and equipment are respectively put into the organization queue and equipment queue to be stored; (2) First, use the single-organization warehousing thread to sequentially process all the organization information in the queue of organizations to be stored; (3) After all the tissues are stored in the warehouse, multi-threaded processing of the equipment information in the equipment queue to be stored begins; In the whole process, except for the single-thread operation of the organization information entering the warehouse, after the organization is entered into the warehouse, the multi-thread operation of the equipment entering the warehouse begins; Different thread pools are intelligently allocated according to device types, and the number of threads in each thread pool is dynamically optimized and adjusted based on the number of devices and the real-time performance of the system.

4. According to claim 1, the system for realizing national standard upper-level service disaster recovery and rapid recovery by using Redis record cascade and device information is characterized in that: In the secondary search module, the hash value of the reported information and the previous record information in Redis is calculated, and whether the information has been changed is determined according to the hash value; the optimization process of secondary search for shared device information includes step S41 and / or step S42, wherein Step S41 is as follows: S411: After the national standard cascade upper service is successfully started, the search signaling is sent for the first time, and the lower-level cascade platform reports the search response. S412: Process the first search response and store the information in the database. After the storage is completed, the information is recorded in the device information hash table of the Redis service. S413: The national standard cascade upper service sends a search signaling for the second time, and the lower cascade platform reports a search response. S414: After receiving the response information of the retrieval device, compare the current report with the previous report information recorded in Redis according to the device ID to see whether it has changed or is the first report; S415: If the information has changed or is reported for the first time, the information is stored in the database, and after the storage is completed, the information is promptly recorded in the device information hash table of the Redis service; if the information has not changed, the message is directly discarded and the next message is processed; Step S42 is as follows: S421: After the second retrieval is completed, the consistency of the cascade device information recorded in Redis and the cascade device information recorded in the database is checked regularly. The checking period can be flexibly adjusted according to the real-time load of the system and the importance of the data. S422: During verification, the device data in the database shall prevail. If data inconsistency is found, the information in Redis shall be accurately adjusted in a timely manner based on the accurate device data in the database to ensure the accuracy and completeness of the data.

5. According to claim 1, the system for realizing national standard upper-level service disaster recovery and rapid recovery by using Redis record cascade and device information is characterized in that: In the Kafka information processing module, the Kafka retrieval information reporting process of the national standard cascade upper service includes steps S51 and S52; Step S51 is as follows: S511: The national standard cascade upper service is started successfully; S512: The lower-level cascade platform actively produces and reports device retrieval information, sets the number of partitions of the Kafka message queue according to the number of lower-level devices and data traffic, sends the device retrieval information to the Kafka message queue, and stores the messages in each partition in chronological order; S513: The national standard cascade upper service retrieves message data from the consumer device in Kafka in the order of offset. If the upper service restarts abnormally, it continues to consume the data in the Kafka message queue from the offset position where it was interrupted last time, relying on the sequential consumption characteristics of Kafka. Step S52 is as follows: S521: The national standard cascade upper service judges the received retrieval device response information; compares the current report with the past report information recorded in Redis according to the device ID to see if there is a change; or if it is the first report; S522: If the information has changed or is reported for the first time, the information will be stored in the database. After the database storage is completed, the information will be recorded in the device information hash table of the Redis service; if the information has not changed, the message will be discarded directly and the next message will be processed to ensure the timeliness, continuity and integrity of data synchronization between platforms.

Images