Message processing method and device based on broadcast control system, and computer storage medium

Abstract

The invention discloses a message processing method and device based on a broadcast control system and a computer storage medium, and relates to the technical field of network communication, and the method comprises the steps: analyzing a provider identifier in service information, and distributing the service information to a corresponding message queue based on the provider identifier; monitoring the message backlog amount, the production rate and the consumption rate of each message queue through a production counter and a consumption counter; when it is monitored that the production rate of the message queue is higher than the consumption rate and the message backlog amount reaches a first preset threshold value, creating an extended queue, and redistributing a preset migration amount of unprocessed service information in the backlog queue to the extended queue; new consumers are respectively allocated to the backlog queue and the extended queue, so that the parallel processing of the service information is realized, and the message processing efficiency is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of network communication technology, and in particular to a message processing method, device and computer storage medium based on a broadcast control system. Background Art

[0002] In the IPTV (Internet Protocol Television) broadcast control system, the processing of business information usually adopts a single queue processing model. When the system faces a high concurrency environment, the current message queue processing mechanism is prone to message accumulation due to the influx of a large number of business requests. Under the single queue processing model, all business information is waiting to be processed in the same queue in turn, and its processing capacity is relatively fixed. When multiple content providers upload a large amount of business information at the same time, the message processing speed of the single queue cannot keep up with the message production speed, which will cause a large number of messages to be accumulated, resulting in processing delays. Summary of the invention

[0003] The main purpose of this application is to provide a message processing method, device and computer storage medium based on a broadcast control system, aiming to solve the technical problem of poor message processing in an IPTV broadcast control system under a high concurrency environment.

[0004] To achieve the above purpose, an embodiment of the present application provides a message processing method based on a broadcast control system, and the message processing method based on a broadcast control system includes: Parsing the provider identifier in the business information, and assigning the business information to a corresponding message queue based on the provider identifier; Monitoring the message backlog, production rate and consumption rate of each message queue through a production counter and a consumption counter; When it is monitored that the production rate of the message queue is higher than the consumption rate and the message backlog reaches a first preset threshold, an extended queue is created, and a preset migration amount of unprocessed business information in the backlog queue is reallocated to the extended queue; New consumers are allocated to the backlog queue and the expansion queue respectively, so as to realize parallel processing of the business information.

[0005] In one embodiment, when it is monitored that the production rate of the message queue is higher than the consumption rate and the message backlog reaches a first preset threshold, creating an extended queue and reallocating a preset migration amount of unprocessed business information in the backlog queue to the extended queue includes: Migrating a preset number of unprocessed business information in the backlog queue to the extended queue in batches, wherein the number of the unprocessed business information migrated in each batch does not exceed a single migration amount threshold; Recording the original order of the unprocessed business information that needs to be migrated in the backlog queue; Extracting the unprocessed service information from the backlog queue in sequence according to the original order, and migrating the unprocessed service information to the extended queue; The unprocessed service information migrated to the extended queue is arranged based on the original sequence.

[0006] In one embodiment, the step of arranging the unprocessed service information migrated to the extended queue based on the original order further includes: Determine a starting point for migration, and initialize a pointer or index to point to the unprocessed business information that enters the queue earliest in the backlog queue; Migrating the unprocessed service information pointed to by the pointer or the index to the extended queue, and updating the pointer or the index to point to the next unprocessed service information; Continue to migrate the unprocessed service information, and update the pointer or the index each time the unprocessed service information is migrated, until a preset migration quantity is reached.

[0007] In one embodiment, when it is monitored that the production rate of the message queue is higher than the consumption rate and the message backlog reaches a first preset threshold, after the step of creating an extended queue and reallocating a preset migration amount of unprocessed business information in the backlog queue to the extended queue, the process further includes: Distribute the newly generated service information evenly to the message queue and the extension queue based on the provider identifier; Monitoring the message backlog, the production rate, and the consumption rate of the message queue and the extended queue; If all the unprocessed business information after migration in the extended queue has been processed, and the message backlog of the message queue is reduced to a second preset threshold, the remaining unprocessed information in the extended queue is merged into the original message queue.

[0008] In one embodiment, the step of monitoring the message backlog, production rate, and consumption rate of each message queue through a production counter and a consumption counter includes: Whenever the business information enters the message queue, the production counter corresponding to the message queue increases; When the business information is consumed, the message counter corresponding to the message queue increases; At regular intervals, read the current values ​​of the production counter and the consumption counter of each message queue to obtain the message backlog of the corresponding message queue; The current value is compared with the value of the corresponding message queue in the previous cycle, and the production rate and the consumption rate of each message queue are calculated.

[0009] In one embodiment, after the step of monitoring the message backlog, production rate and consumption rate of each message queue through the production counter and the consumption counter, the method further includes: When the message backlog of the message queue reaches a first preset threshold, if the production rate of the backlog queue is less than or equal to the consumption rate, calculate the ratio of the production rate to the consumption rate, and determine the number of consumers that need to be added to the backlog queue based on the ratio; Start a corresponding number of new consumer instances and establish connections between the new consumer instances and the backlog queue, or expand a corresponding number of threads on the existing consumer instances in the backlog queue.

[0010] In one embodiment, after the step of parsing the provider identifier in the service information and allocating the service information to the corresponding message queue based on the provider identifier, the following steps are included: Extracting metadata from the business information; Writing the metadata into a storage target according to a preset rule, and marking the processing status of the metadata as unprocessed; When the business information is processed in the corresponding queue, the processing status of the metadata in the storage target is updated in real time.

[0011] In one embodiment, the message processing method based on the broadcast control system further includes: receiving an export instruction including export parameters, wherein the export parameters include a time range, a processing status of the business information, and a name of the message queue; Based on the export instruction, filtering out metadata of the business information that meets the conditions from the storage target; Exporting metadata of the business information that meets the conditions into a standard format file; When performing fault recovery or data backtracking, the metadata in the standard format file is read, and based on the processing status of the metadata and the provider identifier, the metadata in the unprocessed state is imported into the corresponding message queue to resume processing.

[0012] An embodiment of the present application also provides a message processing device based on a broadcast control system, and the message processing device based on the broadcast control system includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the message processing method based on the broadcast control system as described above.

[0013] An embodiment of the present application further provides a computer storage medium, which is a computer-readable storage medium. A computer program is stored on the computer storage medium. When the computer program is executed by a processor, the steps of the message processing method based on the broadcast control system as described above are implemented.

[0014] The embodiment of the present application discloses a message processing method based on a broadcast control system, which parses the provider identifier in the business information and allocates the business information to the corresponding message queue based on the provider identifier; monitors the message backlog, production rate and consumption rate of each message queue through the production counter and consumption counter; when it is monitored that the production rate of the message queue is higher than the consumption rate and the message backlog reaches a first preset threshold, creates an extended queue, and reallocates the unprocessed business information of a preset migration quantity in the backlog queue to the extended queue; allocates new consumers to each of the backlog queue and the extended queue to realize parallel processing of the business information. The present application effectively improves the message processing efficiency and ensures system stability and business continuity by diverting the business information and new business information in the backlog queue. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flow chart of a first embodiment of a message processing method based on a broadcast control system according to an embodiment of the present application; Figure 2 This is a schematic diagram of a message queue partition processing flow of a first embodiment of a message processing method based on a broadcast control system according to an embodiment of the present application; Figure 3 This is a flow chart of a second embodiment of a message processing method based on a broadcast control system according to an embodiment of the present application; Figure 4 This is a flow chart of a third embodiment of a message processing method based on a broadcast control system according to an embodiment of the present application; Figure 5 This is a flow chart of a fourth embodiment of a message processing method based on a broadcast control system according to an embodiment of the present application; Figure 6 This is a schematic diagram of a process of dynamically adjusting a message queue in a fourth embodiment of a message processing method based on a broadcast control system according to an embodiment of the present application; Figure 7 This is a flowchart of a fifth embodiment of a message processing method based on a broadcast control system according to an embodiment of the present application; Figure 8 A flowchart of the export and import operations of the fifth embodiment of the message processing method based on the broadcast control system involved in the embodiment of the present application; Fig. 9 This is a flow chart of a sixth embodiment of a message processing method based on a broadcast control system according to an embodiment of the present application; Fig.10 This is a schematic diagram of a message persistence process of a sixth embodiment of a message processing method based on a broadcast control system according to an embodiment of the present application; Fig.11 This is a flow chart of a seventh embodiment of a message processing method based on a broadcast control system according to an embodiment of the present application; Fig.12 This is a flow chart of an automatic compensation mechanism for service information processing failure in a seventh embodiment of a message processing method based on a broadcast control system according to an embodiment of the present application; Fig.13 This is a structural diagram of the message processing device based on the broadcast control system of this application.

[0016] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0017] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0018] In the IPTV (Internet Protocol Television) broadcast control system, the processing of business information usually adopts a single queue processing model or a fixed queue and fixed number of consumer models. When the system faces a high concurrency environment, the current message queue processing mechanism is prone to message accumulation due to the influx of a large number of business requests. Due to the lack of an effective dynamic load balancing mechanism, both the single queue processing model and the consumer model cannot dynamically adjust the queue resource allocation according to the message type and business volume, which limits the overall message throughput of the system and causes processing delays.

[0019] In order to solve the above-mentioned defects existing in the related art, an embodiment of the present application proposes a message processing method, which parses the provider identifier in the business information and allocates the business information to the corresponding message queue based on the provider identifier; monitors the message backlog, production rate and consumption rate of each message queue through the production counter and consumption counter; when it is monitored that the production rate of the message queue is higher than the consumption rate and the message backlog reaches a first preset threshold, creates an extended queue and reallocates the unprocessed business information of a preset migration number in the backlog queue to the extended queue; assigns new consumers to each of the backlog queue and the extended queue to realize parallel processing of the business information. The present application effectively improves the message processing efficiency and ensures system stability and business continuity by diverting the business information and new business information in the backlog queue.

[0020] It should be noted that the execution subject of this embodiment may be a message processing system, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a message processing device based on a broadcast control system that can realize the above functions, etc. The following takes the message processing system as an example (hereinafter referred to as the "system") to illustrate this embodiment and the following embodiments.

[0021] The message processing method based on the broadcast control system of the first embodiment proposed in this application, please refer to Figure 1 The method comprises steps S10 to S40: Step S10: parsing the provider identifier in the service information, and allocating the service information to a corresponding message queue based on the provider identifier.

[0022] The IPTV (Internet Protocol Television) broadcast control system needs to process service information from different providers to ensure that the content of each provider can be correctly scheduled and played. In this process, vspCode (Vendor Specific Code) is the key identifier to distinguish different sources of service information. The provider identifier uniquely identifies the source provider of the service information.

[0023] Please refer to Figure 2 , Figure 2This is a diagram of the message queue partitioning process. After the system receives the business information, it will read and parse the data packet of each business information and extract the vspCode from it. Next, the message queue partitioning module in the system classifies the business information according to the vspCode and assigns different types of business information to corresponding independent message queues. If no classification is performed, it is assigned to the default queue. Each message queue is responsible for processing at least one business information with the same vspCode. When the business information enters the message queue, the queue manager in the message queue partitioning module will distribute the business information to the corresponding consumers for processing.

[0024] Specifically, the system uses preset parsing rules, such as regular expressions or XML (Extensible Markup Language) parsers, to extract the provider ID from the business information. After extracting the provider ID, the message queue partition module allocates the business information to the corresponding message queue based on the mapping relationship between the provider ID and the pre-configured message queue. The allocation process can be based on a hash table search or implemented through mapping rules defined in the configuration file.

[0025] Step S20: Monitor the message backlog, production rate and consumption rate of each message queue through a production counter and a consumption counter.

[0026] It should be noted that the production counter is used to record the number of business messages sent to the message queue within a certain period of time, reflecting the production rate of the message queue. The consumption counter is used to record the number of business messages removed from the message queue and processed by the consumer within a certain period of time, reflecting the consumption rate of the message queue. The message backlog refers to the number of business messages in the current message queue that have not been consumed, that is, the number of business messages that have not been processed, which can measure the load status of the message queue.

[0027] In this step, the system can periodically obtain the current status data of the message queue from the queue manager by using timed polling or a time-driven mechanism. The production counter and consumption counter can also be variables in memory, which are used to perform increment operations when business information is produced or consumed.

[0028] Specifically, step S20 may include steps S21 to S24: Step S21: Whenever the business information enters the message queue, the production counter corresponding to the message queue increases.

[0029] Step S22: When the service information is consumed, the message counter corresponding to the message queue increases.

[0030] Step S23: every fixed period, read the current values ​​of the production counter and the consumption counter of each message queue to obtain the message backlog of the corresponding message queue.

[0031] Step S24: Compare the current value with the value of the corresponding message queue in the previous cycle, and calculate the production rate and the consumption rate of each message queue.

[0032] In this embodiment, a production counter and a consumption counter are defined for each message queue. When the event of business information entering a message queue is triggered, the value in the production counter of the corresponding message queue is increased by 1 to count the input flow of the business information. When the business information is sent to the consumer for processing, that is, when the business information is consumed, the value of the consumption counter of the corresponding message queue is also increased by 1.

[0033] Set a scheduled task to execute once every fixed period. Each time the scheduled task is executed, the system will read the current values ​​of the production counter and consumption counter of the message queue, and compare the current values ​​with the values ​​of the previous cycle to calculate the production rate and consumption rate of each message queue. The message backlog of the message queue is obtained by calculating the difference between the current production counter and consumption counter of the message queue.

[0034] Specifically, the message backlog, production rate, and consumption rate of the message queue are calculated as follows: Message backlog = current production count value - current consumption count value; Production rate = current production count value - last production count value; Consumption rate = current consumption count value - previous consumption count value.

[0035] For example, a 5-second timer task is set. When the timer task is executed, the current production count value of a message queue is 55, and the current consumption count value is 40. The last production count value is 35, and the last consumption count value is 25. The current count value is compared with the count value of the last 5-second cycle, and the message backlog, production rate, and consumption rate of the message queue are calculated to be 20, 15, and 10, respectively.

[0036] Step S30: When it is monitored that the production rate of the message queue is higher than the consumption rate and the message backlog reaches a first preset threshold, an extended queue is created to reallocate a preset migration quantity of unprocessed business information in the backlog queue to the extended queue.

[0037] In this embodiment, when the production rate of business information in the message queue is higher than the consumption rate, it means that the speed at which business information enters the message queue exceeds the speed at which it is processed. As time goes by, the amount of business information in the message queue will inevitably increase, and the new business information entering the message queue will need to wait longer before being processed, which will cause the processing delay of the entire system to increase.

[0038] If not processed in a timely manner, long-term message backlogs will consume more system resources, such as memory and storage space, which will affect the performance of the message queue server, resulting in longer service response time, thus affecting the stability of the system.

[0039] The first preset threshold is a critical value set according to factors such as historical business information processing conditions and system resource carrying capacity, etc. The backlog queue refers to the target message queue when the message backlog in the message queue reaches the first preset threshold.

[0040] Therefore, when the production rate of the message queue is higher than the consumption rate and the message backlog reaches the first preset threshold, the system will create a new queue, namely the extended queue, and then reallocate the preset migration quantity of unprocessed business information in the backlog queue to the new queue to disperse the load of each message queue.

[0041] It should be noted that the preset migration quantity can be determined according to the historical business processing capacity of the system or the current system resource carrying capacity.

[0042] In this embodiment, a duration may be set. When the production rate of business information in the message queue is higher than the consumption rate for a period of time, it is determined whether an extended queue needs to be created based on the current message backlog in the message queue.

[0043] For example, when the production rate of business information in the message queue is continuously higher than the consumption rate within 3 minutes, and the message backlog in the message queue reaches 5,000, the system immediately creates a new message queue for the message queue as an extended queue. At the same time, the system extracts 2,000 business information from the backlog queue and migrates them to the extended queue based on the current resource carrying capacity or the preset migration quantity, so that the load in the original backlog queue is effectively alleviated, and the new extended queue shares part of the business information processing task.

[0044] It should be noted that, during the migration of the service information, the service information with the same provider ID is also migrated to the extended queue with the same provider ID according to the provider ID, and there is at least one extended queue.

[0045] In this embodiment, during the migration of the service information of the backlog queue, the newly generated service information can be diverted to the original message queue or the new extended queue through a hash algorithm or a polling algorithm to balance the load of each message queue.

[0046] It should be noted that when the newly generated business information is diverted to the original message queue or the new extended queue, it is also classified according to the provider identifier and allocated to the queue accordingly.

[0047] Optionally, during the migration of unprocessed business information in the backlog queue, or when the production rate of the message queue is higher than the consumption rate and the message backlog reaches a first preset threshold, or when creating an extended queue, the sending of new business information to the original backlog queue can be suspended, and the newly generated business information can be sent to the extended queue instead. This ensures that the business information in the backlog queue can be effectively processed, and the new business information can be properly allocated and processed in the extended queue, thereby avoiding further message backlogs in the original backlog queue and ensuring the rational use of system resources and the balance of queue loads.

[0048] Optionally, after the migration of the unprocessed business information in the backlog queue is completed, the migrated business information records originally occupy a certain amount of memory space in the backlog queue to store their own data and corresponding index resources for rapid positioning and management. Therefore, the migrated business information records in the backlog queue can be traversed to release the memory and index resources occupied by the business information records.

[0049] Specifically, the system uses an internal traversal algorithm or loop mechanism to search and locate the storage area and index structure corresponding to each migrated business information, and then uses a memory release function or resource recovery mechanism to clear and recycle the memory space occupied by it. At the same time, it destroys or resets the corresponding index resources so that these resources can be reused by the system, avoiding resource waste and improving the overall resource utilization efficiency of the system.

[0050] Furthermore, when the situation of the backlog queue is alleviated, that is, the production rate no longer exceeds the consumption rate and the message backlog reaches the first preset threshold, the restriction on the suspension of sending new business information to the backlog queue is lifted, and the inflow rate of new business information into the original backlog queue can be set to half of the preset rate. Continue to monitor for a period of time. If the production rate of the backlog queue does not exceed the consumption rate again during this period, and the message backlog does not reach the first preset threshold, the system can restore the inflow rate of new business information to the original preset rate to ensure the stability of the message queue.

[0051] It should be noted that the monitoring time refers to the length of time the system continues to observe the backlog queue status after lifting the restriction on suspending the sending of new business information to the backlog queue and adjusting the inflow rate of new business information. This time can be preset or estimated based on the relief time of the system's historical backlog queue. The preset rate is the standard rate at which new business information flows into the message queue that is preset by the system during normal operation.

[0052] Step S40: allocating new consumers to the backlog queue and the expansion queue respectively, so as to realize parallel processing of the business information.

[0053] It should be noted that consumers refer to components in the system that can obtain business information from the message queue and process the business information. These components can be in the form of services or processes.

[0054] By assigning consumers to each of the two queues, the system can process the business information in the backlog queue and the business information in the expansion queue at the same time, thereby improving the processing efficiency of business information, shortening the residence time of business information in the message queue, and improving the performance of the entire system.

[0055] Please refer to Figure 3 The message processing method based on the broadcast control system of the second embodiment proposed in the present application includes steps S21-S22 after step S20: Step S21: when the message backlog of the message queue reaches a first preset threshold, if the production rate of the backlog queue is less than or equal to the consumption rate, calculate the ratio of the production rate to the consumption rate, and determine the number of consumers that need to be added to the backlog queue based on the ratio; Step S22: Start a corresponding number of new consumer instances, and establish connections between the new consumer instances and the backlog queue, or expand a corresponding number of threads on the existing consumer instances in the backlog queue.

[0056] In this embodiment, when the accumulated message backlog of a message queue reaches a first preset threshold, if the production rate of the backlog queue (when the message backlog of a message queue reaches the first preset threshold, it is called a backlog queue) is less than or equal to the consumption rate, it means that the current message queue has a certain message processing capability, but the current backlog situation of the backlog queue needs to be optimized and adjusted.

[0057] At this time, the system calculates the ratio of production rate to consumption rate, which can reflect the relative speed between production and consumption and the balance between the two. Then, the system uses the pre-set rules or algorithms to determine the number of consumers that need to be added to the backlog queue based on the ratio, so as to reasonably supplement the message processing resources, speed up the processing of business information, and effectively alleviate and improve the message backlog in the backlog queue.

[0058] After determining the number of new consumers, the system starts a corresponding number of new consumer instances, which are program units with independent processing functions. Through the communication mechanism and configuration operations, the new consumer instances are connected to the backlog queue, so that the new consumer instances can obtain business information from the backlog queue and process it.

[0059] Alternatively, based on the existing consumer instances in the backlog queue, expand the corresponding number of threads according to the required number of new ones, and use the thread parallel processing mechanism to improve the processing capabilities of the existing consumer instances, and then collaboratively process the business information in the backlog queue to ensure the efficiency of the message processing process and the rationality of resource allocation.

[0060] Please refer to Figure 4 In the message processing method based on the broadcast control system of the third embodiment proposed in the present application, step S30 includes steps S31 to S34: Step S31: Migrating a preset number of unprocessed business information in the backlog queue to the extended queue in batches, wherein the number of the unprocessed business information in each batch does not exceed a single migration amount threshold.

[0061] There is a lot of unprocessed business information in the backlog queue. If all the unprocessed business information is migrated to the extended queue at one time, it may bring great resource pressure to the system. For example, it will instantly occupy a large amount of memory, network bandwidth and other resources, and may even cause the system to freeze or fail.

[0062] Therefore, the batch migration method is adopted. The single migration volume threshold is the pre-set upper limit of the number of unprocessed business information allowed to be migrated in each batch. The system divides the unprocessed business information in the backlog queue into multiple batches based on the single migration volume threshold.

[0063] During the actual migration, through the data transmission mechanism, such as calling the system's internal message passing interface or data replication function, unprocessed business information not exceeding the single migration threshold is taken out from the backlog queue each time and gradually migrated to the expansion queue, so as to ensure the rational use of system resources during the migration process and the smoothness of the entire migration operation.

[0064] Step S32: Record the original order of the unprocessed business information that needs to be migrated in the backlog queue.

[0065] Business information originally exists in a certain order in the backlog queue. This order is related to important factors such as the initiation time and priority of the business information. The original order is the order formed when the business information first enters the backlog queue based on time, business priority number or other pre-set sorting rules.

[0066] The system utilizes a sequential recording mechanism, for example, creating a sequential recording table, using the unique identifier of the business information (such as the business information ID, etc.) as an index, and correspondingly recording the sequential number of the business information in the backlog queue.

[0067] Alternatively, a sequence mark field may be added to the data structure of the business information to save its original sequence information, etc., to ensure that the original sequence of each piece of business information to be migrated can be accurately recorded.

[0068] Step S33: extracting the unprocessed service information from the backlog queue in sequence according to the original order, and migrating the unprocessed service information to the extended queue.

[0069] Step S34: Arrange the unprocessed service information migrated to the extended queue based on the original order.

[0070] In this embodiment, when performing the migration operation of unprocessed business information, the system first identifies and locks the original order of the unprocessed business information in the backlog queue. Then, according to the original order, the system starts from the head or starting position of the backlog queue, extracts the unprocessed business information one by one, and accurately migrates the unprocessed business information to be migrated to the extended queue with the help of the queue interaction mechanism, ensuring the orderliness and integrity of the business information during the transfer process and avoiding information confusion or loss.

[0071] During the migration process, there may be slight deviations in the order in which information arrives at the extended queue or it may not be arranged in the original order due to factors such as network transmission and system resource allocation. The system will re-arrange the unprocessed business information that has been migrated to the extended queue. For example, the system will readjust the business information in the extended queue to a sequence that is completely consistent with the original order based on the sequence number in the business information.

[0072] Furthermore, the system may construct an index structure for the unprocessed business information in the backlog queue, such as an index based on a hash linked list, wherein the hash value is generated by a key identifier and a sequential number of the business information.

[0073] When migrating unfinished business information, the unfinished business information is extracted in sequence according to the index structure and migrated to the extended queue. At the same time, at the receiving end of the extended queue, the pre-set synchronization mechanism and sequence verification logic can be used to ensure that the business information is accurately inserted into the corresponding position of the extended queue according to the original order corresponding to the index structure, maintain the consistency and correctness of message processing in the entire system, ensure that the system's business rules based on sequential processing can be effectively executed, and avoid business errors or data inconsistencies caused by disordered order.

[0074] Furthermore, step S34 also includes steps S341 to S343: Step S341: determine the starting point of migration, and initialize a pointer or index to point to the unprocessed business information that enters the queue earliest in the backlog queue.

[0075] Step S342: Migrate the unprocessed service information pointed to by the pointer or the index to the extended queue, and update the pointer or the index to point to the next unprocessed service information.

[0076] Step S343: Continue to migrate the unprocessed service information, and update the pointer or the index each time the unprocessed service information is migrated, until a preset migration quantity is reached.

[0077] In this embodiment, to ensure that the service information is migrated to the extended queue in the original order, the system first determines the starting point of the migration operation and initializes a pointer or index that points to the earliest unprocessed service information in the backlog queue.

[0078] Next, the system migrates the unprocessed business information pointed to by the pointer or index to the extended queue, and immediately updates the pointer or index after the migration to point to the next unprocessed business information in the extended queue, ensuring that the business information is processed in the order in which it is in the original message queue.

[0079] Finally, the system continues to migrate unprocessed business information, and updates the pointer or index after each migration operation until the preset migration number is reached.

[0080] Through the above method, the system ensures the continuity and integrity of the migration process, and ensures that the order of business information in the extended queue remains consistent with the original message queue.

[0081] Please refer to Figure 5 The message processing method based on the broadcast control system of the fourth embodiment proposed in the present application further includes steps S310 to S330 after step S30: Step S310: Distribute the newly generated service information evenly to the message queue and the extension queue based on the provider identifier.

[0082] In this embodiment, an even distribution strategy is adopted to distribute the newly generated business information to the message queue and the extension queue respectively. This method aims to balance the load pressure of the two queues and avoid excessive pressure on a single queue, which will lead to message backlog again. In the process of allocating new business information, the system ensures that business information from the same source has relatively balanced processing opportunities in different queues based on the provider identifier.

[0083] It should be noted that, in this embodiment, the allocation of business information from the same source to different queues refers to the situation where an extended queue is created when a message queue corresponding to the source implements an over-diversion strategy due to message backlog.

[0084] If there is no message backlog in the message queue corresponding to the provider identifier in the business information, that is, no corresponding extended queue is created for the message queue, the business information will continue to be assigned to its original corresponding message queue for processing without performing a diversion operation.

[0085] Step S320: monitoring the message backlog, the production rate, and the consumption rate of the message queue and the extended queue.

[0086] The system uses production counters and consumption counters to periodically obtain the message backlog, production rate, and consumption rate in the message queue and extended queue, ensuring that the system can grasp the status of the queue in real time, respond and adjust the processing strategy in a timely manner according to actual conditions, and maintain the operation and efficient processing capabilities of the message queue.

[0087] Step S330: If all the unprocessed business information after migration in the extended queue has been processed and the message backlog of the message queue is reduced to a second preset threshold, the remaining unprocessed information in the extended queue is merged into the original message queue.

[0088] When all the unprocessed business information from the backlog queue in the extended queue is successfully processed, and the message backlog of the message queue is reduced to the second preset threshold, it indicates that the processing pressure of the original message queue has been effectively relieved, and the task of the extended queue is basically completed. The system will migrate the remaining unprocessed business information in the extended queue to the original message queue, optimize system resource allocation, reduce unnecessary queue management overhead, and ensure the consistency and integrity of business information processing.

[0089] For example, in order to help understand the implementation process of the dynamic adjustment of the message queue of the message processing method based on the broadcast control system obtained by combining the present embodiment with the above embodiments, please refer to Figure 6 , Figure 6 A flowchart for dynamically adjusting the message queue is provided, specifically: The system monitors the production rate and consumption rate of business information in the message queue, and counts the message backlog in each message queue. When the message backlog exceeds the first preset threshold, it further determines whether the production rate is greater than the consumption rate. If the production rate is greater than the consumption rate, an extended queue is created, and the unprocessed business information of the preset migration amount in the backlog queue is reallocated to the extended queue, and new consumers are assigned to the extended queue. If the production rate is less than or equal to the consumption rate, the number of consumers is added to the backlog queue to speed up the processing of business information.

[0090] During the business information diversion process, if the message backlog of the message queue is reduced to below the second preset threshold, it means that the previous process of creating an extended queue or increasing the number of consumers of the backlog queue has a certain effect.

[0091] After the message backlog of the message queue is reduced to below the second preset threshold, the number of consumers in the extended queue or the backlog queue can be reduced to release system resources. When reducing the extended queue, the remaining unprocessed business information in the extended queue is migrated to the original message queue. If the message backlog of the message queue has not been reduced to below the second preset threshold, the current consumer queue (including the extended queue) and the number of consumers are maintained.

[0092] Please refer to Figure 7 The message processing method based on the broadcast control system of the fifth embodiment proposed in the present application further includes steps S510 to S540: Step S510: receiving an export instruction including export parameters, wherein the export parameters include a time range, a processing status of the business information, and a name of the message queue.

[0093] Step S520: Based on the export instruction, the metadata of the business information that meets the conditions is filtered out from the storage target.

[0094] In order to enhance the disaster recovery capability of the system, the system in this embodiment supports exporting and importing the business information in the message queue. The user can perform export operations through the console to back up the business information in the message queue. When the system fails or needs to be restored, the previously backed up business information can be quickly imported through the console to continue processing the unprocessed business information.

[0095] In this embodiment, the user can specify the name of the message queue, time range or processing status and other screening conditions to be exported through the console, system interface or API interface.

[0096] Business information can be filtered and backed up based on conditions such as time range, name of message queue, and processing status of business information to ensure the integrity of business information.

[0097] Specifically, the time range refers to filtering out business information within a specific time period based on the generation time or processing time of the business information, ensuring that only the required business information is exported.

[0098] The queue name refers to filtering business information by the name of the specified message queue, and supports backing up business information in a specific message queue.

[0099] Processing status refers to filtering business information according to the processing status of the business, such as "unprocessed", "processed successfully", "processed failed", etc., so that unprocessed or failed messages can be exported first when the system fails to recover.

[0100] Step S530: exporting metadata of the business information that meets the conditions into a standard format file.

[0101] Standard format files refer to files that follow the data formats that are commonly used or uniformly specified within the system, such as the common CSV format, JSON format, etc.

[0102] The screening and exporting process specifically includes: The system reads metadata of business information that meets the filtering criteria from a storage target, such as a database or Elasticsearch.

[0103] Convert the filtered metadata into a specified format, such as JSON or CSV, and then write the filtered metadata in batches into the export file to obtain a standard format file and improve export efficiency.

[0104] Finally, save the exported standard format file to a specified storage location, such as a file system or cloud storage, for use in scenarios such as system failure recovery or system migration.

[0105] Specifically, when a standard format file is saved to a specified storage location, a unique file name is generated for locating the standard format file during subsequent import operations. In addition, the system will also record the export operation in the database or system log, including the export file name, export conditions, and timestamp, etc., for subsequent tracing or recovery.

[0106] Step S540: when performing fault recovery or data backtracking, the metadata in the standard format file is read, and based on the processing status of the metadata and the provider identifier, the metadata in the unprocessed state is imported into the corresponding message queue for recovery processing.

[0107] When the system needs to perform fault recovery or data backtracking, you can import the business information of the previously exported standard format file into the corresponding message queue through the console. The system will automatically identify the processing status of the business information and continue with subsequent processing.

[0108] Specifically, when the system performs a recovery operation, the user selects the standard format file to be imported and uploads it to the system, or directly selects the latest backup file to import through the system recovery operation. Then, the system reads and parses the metadata of the business information in the standard format file or backup file one by one, such as the business information ID, processing status, and the generation time of the business information.

[0109] During the reading process, the system checks the processing status of each business information. If the processing status is "unprocessed" or "processing failed", the business information is rewritten into the corresponding message queue for reprocessing according to the provider ID. If the processing status is "processing successful", the system can decide whether to skip the business information or reprocess it according to business needs.

[0110] After the business information is rewritten into the corresponding message queue, the system will update the processing status of the rewritten business information in the database, for example, marking it as "recovered" or "reprocessing".

[0111] After the import operation is completed, the system will record the import operation log, including the number of imported business information, import status and other related content, so that the administrator can verify the import results.

[0112] For example, assuming that the business information of a certain period of time in the system needs to be backed up, the specific steps are as follows: Step 1, selection of export parameters: The user chooses to export the messages in the Queue_A message queue, with the time range from 00:00:00 on November 1, 2024 to 23:59:59 on November 2, 2024, and only exports the business information with the processing status of "unprocessed" and "processing failed".

[0113] Step 2, filter and save: The system filters out the qualified business information from the database, converts the business information into JSON format, and generates the file backup_Queue_A_20241101.json.

[0114] Step 3, Export record: The system records the export operation in the system log, including the export file name, filter conditions, and timestamp.

[0115] For example, assuming that the business information of a certain period of time in the system needs to be restored, the specific steps are as follows: Step 4: Select a standard format file or backup file: When restoring, the system selects the backup_Queue_A_20241101.json file and starts importing.

[0116] Step 5, read and parse the file content: the system reads each piece of business information in the file and determines the processing status of the business information based on the metadata.

[0117] Step 6, write to the message queue: For business information with a processing status of "unprocessed" and "processing failed", the system rewrites it to the Queue_A message queue for processing by the consumer.

[0118] Step 7, update processing status: update the processing status of the corresponding business message in the database to "recovered" or "reprocessing".

[0119] Step 8, Feedback and Records: The system records the import log, including the amount of imported business information and the import status, such as "Import Successful", for the administrator to review.

[0120] For example, to help understand the implementation process of the message processing method based on the broadcast control system of this embodiment, please refer to Figure 8 , Figure 8 The figure is a flowchart of export and import operations.

[0121] When the import operation is performed, the system receives the export instruction and filters the metadata in the database according to the export parameters of the export instruction, including the time range, message queue name, and processing status of the business information. Then, the system exports the screening results to a standard format file such as JSON / CSV and saves it in the specified storage location. While exporting, the system also records the log of the export operation.

[0122] When exporting, the system will select the backup file or standard format file to be imported, and then read the metadata in the file one by one. If the processing status of the business information corresponding to the metadata is unprocessed or failed, the system will rewrite the business information in these two processing states into the corresponding message queue for processing. If the processing status is other processing status, the business information will be skipped or marked as successfully processed.

[0123] In addition, the system will update the processing status corresponding to the imported business information in the storage target and record the log of the import operation to ensure the effective processing of messages and the consistency of data.

[0124] Please refer to Fig. 9 The message processing method based on the broadcast control system of the sixth embodiment proposed in the present application includes steps S11 to S13 after step S10: Step S11: extracting metadata from the business information.

[0125] Step S12: writing the metadata into a storage target according to a preset rule, and marking the processing status of the metadata as unprocessed.

[0126] Step S13: When the business information is processed in the corresponding queue, the processing status of the metadata in the storage target is updated in real time.

[0127] In order to ensure that business information is not lost when a system failure occurs, the system implements the persistence of business information metadata through AOP (Aspect Oriented Programming) technology. Whenever business information enters the corresponding message queue, the system immediately writes the metadata of the business information to the storage target, such as a database or Elasticsearch (a distributed search and analysis engine), and then marks the processing status of the corresponding metadata as "unprocessed". Through the AOP aspect, the system can ensure that all business information entering the message queue will automatically record metadata to avoid message loss or repeated processing.

[0128] It should be noted that metadata refers to information describing the attributes of business information itself, including business information ID, creation time, provider ID, business type, etc. The storage target can be a database, file system or any other form of persistent storage medium.

[0129] While the system is processing business information, it will also update the processing status of business data in real time in the storage target, such as "processing successful", "processing failed", etc. In addition, the system will also regularly clean up expired business information according to the configuration.

[0130] If the system fails during processing, the status of the business information can be restored through the database or Elasticsearch to continue processing the unfinished or unprocessed business information. Fig.10 , Fig.10 This is a schematic diagram of the message persistence process.

[0131] The producer starts to generate business information, and then the queue manager receives the business information and distributes it to the corresponding message queue. During the processing of business information, the queue manager determines whether there is processing delay or message accumulation in the message queue. If so, the message smooth migration mechanism is triggered. If not, the business information is normally distributed to consumers.

[0132] Consumers receive business information and process it. After processing, they update the processing status of the business information to the storage target, such as a database / Elasticsearch. During the processing of business information, the system persists the metadata of the business information in real time. When the system fails, the business information can be automatically restored based on the persistent metadata, ensuring the stability of the entire process and the integrity of the data.

[0133] AOP technology defines behaviors across multiple points (such as logging, transaction management, etc.), and weaves this behavior into the execution process of the program through "aspects" without modifying the actual business logic code. The specific implementation of AOP aspects is as follows: Define an aspect that contains a pointcut that specifies the behavior that triggers metadata recording, usually when business information is put into a message queue.

[0134] Define a notification in the aspect, which contains specific persistence logic, that is, extracting metadata from business information and writing it to the storage target.

[0135] In this notification, the system performs the following calculations: a. Determine that the business information has successfully entered the message queue. b. Extract metadata from the business information. c. Format the metadata according to preset rules, such as database table structure, file format, or API specification. d. Write the formatted metadata to the storage target, for example, insert it into a database table through an SQL statement, or write it to the file system through a file I / O (Input / Output, read and write) operation. e. Update the processing status of the metadata to "unprocessed", which involves setting a status field in the storage target.

[0136] For example, when the "aspect" of business information entering the message queue is triggered, the system will automatically organize SQL statements, and use INSERT statements to insert the extracted metadata values ​​into the table in the order of the corresponding fields through the database connection object, such as "INSERT INTO metadata_table (message_id, create_time, provider_identifier, business_type) VALUES ('business information ID value', 'creation time value', 'provider identifier value', 'business type value')", where the value is the actual metadata content extracted from the business information.

[0137] At the same time, after the metadata insertion operation is completed, an additional field for marking the processing status (such as the "status" field) is set in the database table, and its corresponding value is updated to a specific identifier indicating unprocessed (such as 0 represents unprocessed).

[0138] In this implementation, metadata in the business information is extracted and written into a storage target, and AOP technology is used to implement metadata persistence operations, thereby ensuring that business information data is not lost when a system failure occurs.

[0139] Please refer to Fig.11 In the seventh embodiment of the message processing method based on the broadcast control system proposed in this application, when the service information processing fails, the system can compensate or retry through the persistent service information record to ensure the integrity of the service process. Specifically, the message processing method based on the broadcast control system can also include steps S610~S630: Step S610: monitoring the processing status of the business information in the message queue.

[0140] The processing status of business information includes the different stages from when the business information enters the message queue to when the system attempts to process the business information, such as "unprocessed", "processing", "successful processing", "failed processing", "recovered", "reprocessing" and other processing statuses.

[0141] The system regularly queries the processing status of business information to promptly detect any abnormalities in business information.

[0142] It should be noted that when business information processing fails, in addition to marking the processing status as "processing failed", it also includes information such as the business information ID, failure reason, and number of retries.

[0143] The number of retries is used to record the number of retries for processing the business information to avoid infinite loop retries.

[0144] The metadata of failed business information is also stored in the storage target so that it can be called during retry and compensation.

[0145] Step S620: When the processing status of the business information is processing failure, the failure reason is recorded and the failed business information is automatically moved into a dead letter queue.

[0146] When business information processing fails or the system fails, the system will mark the processing status of the failed business information as "processing failed" and automatically move the failed business information to the Dead Letter Queue (DLQ). System administrators can regularly check the Dead Letter Queue and re-trigger the business information processing flow manually or through automated scripts.

[0147] Specifically, the system will first automatically retry the "processing failed" business information. The specific steps include: Step 1, check the number of retries: Each failed business message will have a maximum number of retries, for example, 3 times. When the number of retries does not reach the upper limit, the system will try to process the business message again. If the number of retries exceeds the upper limit, the business message will be transferred to the dead letter queue for subsequent manual or custom compensation strategy processing.

[0148] Step 2: Retry delay mechanism: To avoid waste of resources and excessive system load caused by immediate retry, the system will set a delayed retry time for failed business information. For example, the first retry delay is 5 seconds, the second retry delay is 10 seconds, and the delay time is gradually increased until the retry limit is reached.

[0149] Step 3, retry logic: Each time the system retries, it will obtain the metadata of the business information and redistribute it to the corresponding message queue or consumer for processing. At the same time, the system will record the results of each retry, update the number of retries and error information.

[0150] Step S630: In the dead letter queue, a customized compensation strategy is implemented for the failed service information.

[0151] For messages that have failed to retry multiple times, the system supports custom compensation strategies, which process messages in the dead letter queue by setting specific business rules. Common custom compensation strategies include: Manual processing: The administrator regularly checks the dead letter queue, manually triggers the compensation process, resends the business information to the message queue, or repairs the content of the business information.

[0152] Automatic transfer compensation: The system regularly transfers business information in the dead letter queue to a dedicated compensation queue and hands it over to specific consumers for compensation processing according to customized logic.

[0153] Specific compensation logic: For example, for failed messages caused by network timeout, you can increase the delay or change the network request parameters when compensating; for messages with incorrect data formats, you can set format conversion rules and reprocess them.

[0154] For example, when the system attempts to download a video file, the download fails due to a network timeout. The system will immediately mark the service information as "processing failure" and record information such as the service information ID, the cause of the error (network timeout), and the current number of retries (1). Then, the system enters the retry process. The first retry will attempt to download again after 5 seconds. If it still fails, the number of retries will be increased by 1, and the second retry will be delayed by 10 seconds. When the number of retries reaches the maximum value, such as 3 times, the system will no longer retry automatically, but will transfer the service information to the dead letter queue.

[0155] For business information that fails to be downloaded and enters the dead letter queue, the system adopts an automatic transfer compensation strategy to put it into the compensation queue, and a dedicated consumer will try to download it again during the low traffic period of the next day.

[0156] In addition, the customized compensation strategy may also include adjusting download parameters or changing download paths, etc., to ensure that the message processing is completed to the maximum extent possible and to avoid wasting resources.

[0157] For example, to help understand the implementation process of the message processing method based on the broadcast control system of this embodiment, please refer to Fig.12 , Fig.12 This is a flowchart of the automatic compensation mechanism for business information processing failure.

[0158] When the business information processing fails, the system records the ID of the business information that failed to be processed, the reason for failure, and the number of retries. Next, the system will check the number of retries to determine whether the number of retries for the business information that failed to be processed has exceeded the maximum number of retries set. If it does not exceed the maximum number of retries, the business information will wait for the set retry delay time and increase the number of retries. Then the business information will be reallocated to the corresponding message queue for another retry.

[0159] If the maximum number of retries is exceeded, the system will transfer the business information to the dead letter queue. Then it will determine whether to use the automatic compensation strategy. If not, the system will wait for the administrator to manually check and reprocess the business information. If it is an automatic compensation strategy, the business information will be transferred to a specific compensation queue and processed by the compensation consumer.

[0160] Finally, regardless of whether the business information is processed through an automatic compensation strategy or a manual compensation strategy, the system will update the processing status of the business information to ensure that the processing records of all business information are always kept up to date and accurate.

[0161] The present application provides a message processing device based on a broadcast control system, and the message processing device based on the broadcast control system includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the message processing method based on the broadcast control system in the above-mentioned embodiment 1.

[0162] Reference below Fig.13 , which shows a schematic diagram of the structure of a message processing device based on a broadcast control system suitable for implementing an embodiment of the present application. The message processing device based on a broadcast control system in the embodiment of the present application may include various hardware and software components for implementing a message processing method based on a broadcast control system. Fig.13The message processing device based on the broadcast control system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0163] like Fig.13 As shown, the message processing device based on the broadcast control system may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 to the random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the message processing device based on the broadcast control system are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the message processing device based on the broadcast control system to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a message processing device based on the broadcast control system with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0164] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0165] The message processing device based on the broadcast control system provided by the present application adopts the message processing device method based on the broadcast control system in the above embodiment, which can solve the technical problem of low message processing of the IPTV broadcast control system in a high-concurrency environment. Compared with the prior art, the beneficial effects of the message processing device based on the broadcast control system provided by the present application are the same as the beneficial effects of the message processing method based on the broadcast control system provided by the above embodiment, and other technical features in the message processing device based on the broadcast control system are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0166] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0167] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0168] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the message processing method based on the broadcast control system in the above-mentioned embodiment.

[0169] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0170] The computer-readable storage medium may be included in the message processing device based on the broadcast control system; or may exist independently without being assembled into the message processing device based on the broadcast control system.

[0171] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by a message processing device based on a broadcast control system, the message processing device based on the broadcast control system: parses the provider identifier in the business information, and allocates the business information to the corresponding message queue based on the provider identifier; monitors the message backlog, production rate and consumption rate of each message queue through a production counter and a consumption counter; when it is monitored that the production rate of the message queue is higher than the consumption rate and the message backlog reaches a first preset threshold, creates an extended queue and reallocates a preset migration number of unprocessed business information in the backlog queue to the extended queue; and allocates new consumers to each of the backlog queue and the extended queue to realize parallel processing of the business information.

[0172] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0173] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0174] The modules involved in the embodiments described in the present application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0175] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned message processing method based on the broadcast control system, and can solve the technical problem of low message processing of the IPTV broadcast control system in a high-concurrency environment. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the message processing method based on the broadcast control system provided in the above-mentioned embodiment, and will not be repeated here.

[0176] An embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the message processing method based on the broadcast control system as described above when the computer program is executed by a processor.

[0177] The computer program product provided by the present application can solve the technical problem of low message processing of IPTV broadcast control system in a high concurrency environment. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present application are the same as the beneficial effects of the message processing method based on the broadcast control system provided by the above embodiment, which will not be repeated here.

[0178] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

[0179] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0180] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0181] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A message processing method based on a broadcast control system, characterized in that: The message processing method based on the broadcast control system includes: Parsing the provider identifier in the business information, and assigning the business information to a corresponding message queue based on the provider identifier; Monitoring the message backlog, production rate and consumption rate of each message queue through a production counter and a consumption counter; When it is monitored that the production rate of the message queue is higher than the consumption rate and the message backlog reaches a first preset threshold, an extended queue is created, and a preset migration amount of unprocessed business information in the backlog queue is reallocated to the extended queue; New consumers are allocated to the backlog queue and the expansion queue respectively, so as to realize parallel processing of the business information.

2. The message processing method based on the broadcast control system according to claim 1, characterized in that: When it is monitored that the production rate of the message queue is higher than the consumption rate and the message backlog reaches a first preset threshold, creating an extended queue and reallocating a preset migration amount of unprocessed business information in the backlog queue to the extended queue comprises: Migrating a preset number of unprocessed business information in the backlog queue to the extended queue in batches, wherein the number of the unprocessed business information migrated in each batch does not exceed a single migration amount threshold; Recording the original order of the unprocessed business information that needs to be migrated in the backlog queue; Extracting the unprocessed service information from the backlog queue in sequence according to the original order, and migrating the unprocessed service information to the extended queue; The unprocessed service information migrated to the extended queue is arranged based on the original sequence.

3. The message processing method based on the broadcast control system according to claim 2, characterized in that: The step of arranging the unprocessed service information migrated to the extended queue based on the original order further includes: Determine a starting point for migration, and initialize a pointer or index to point to the unprocessed business information that enters the queue earliest in the backlog queue; Migrating the unprocessed service information pointed to by the pointer or the index to the extended queue, and updating the pointer or the index to point to the next unprocessed service information; Continue to migrate the unprocessed service information, and update the pointer or the index each time the unprocessed service information is migrated, until a preset migration quantity is reached.

4. The message processing method based on the broadcast control system according to claim 1, characterized in that: After the step of creating an extended queue and reallocating a preset migration amount of unprocessed business information in the backlog queue to the extended queue when it is monitored that the production rate of the message queue is higher than the consumption rate and the message backlog reaches a first preset threshold, the method further includes: Distribute the newly generated service information evenly to the message queue and the extension queue based on the provider identifier; Monitoring the message backlog, the production rate, and the consumption rate of the message queue and the extended queue; If all the unprocessed business information after migration in the extended queue has been processed, and the message backlog of the message queue is reduced to a second preset threshold, the remaining unprocessed information in the extended queue is merged into the original message queue.

5. The message processing method based on the broadcast control system according to claim 1, characterized in that: The step of monitoring the message backlog, production rate and consumption rate of each message queue through the production counter and the consumption counter includes: Whenever the business information enters the message queue, the production counter corresponding to the message queue increases; When the business information is consumed, the message counter corresponding to the message queue increases; At regular intervals, read the current values ​​of the production counter and the consumption counter of each message queue to obtain the message backlog of the corresponding message queue; The current value is compared with the value of the corresponding message queue in the previous cycle, and the production rate and the consumption rate of each message queue are calculated.

6. The message processing method based on the broadcast control system according to claim 1, characterized in that: After the step of monitoring the message backlog, production rate and consumption rate of each message queue by using the production counter and the consumption counter, the method further includes: When the message backlog of the message queue reaches a first preset threshold, if the production rate of the backlog queue is less than or equal to the consumption rate, calculate the ratio of the production rate to the consumption rate, and determine the number of consumers that need to be added to the backlog queue based on the ratio; Start a corresponding number of new consumer instances and establish connections between the new consumer instances and the backlog queue, or expand a corresponding number of threads on the existing consumer instances in the backlog queue.

7. The message processing method based on the broadcast control system according to claim 1, characterized in that: After the step of parsing the provider identifier in the business information and allocating the business information to the corresponding message queue based on the provider identifier, the method further includes: Extracting metadata from the business information; Writing the metadata into a storage target according to a preset rule, and marking the processing status of the metadata as unprocessed; When the business information is processed in the corresponding queue, the processing status of the metadata in the storage target is updated in real time.

8. The message processing method based on the broadcast control system according to claim 1, characterized in that: The message processing method based on the broadcast control system also includes: receiving an export instruction including export parameters, wherein the export parameters include a time range, a processing status of the business information, and a name of the message queue; Based on the export instruction, filtering out metadata of the business information that meets the conditions from the storage target; Exporting metadata of the business information that meets the conditions into a standard format file; When performing fault recovery or data backtracking, the metadata in the standard format file is read, and based on the processing status of the metadata and the provider identifier, the metadata in the unprocessed state is imported into the corresponding message queue to resume processing.

9. A message processing device based on a broadcast control system, characterized in that: The message processing device based on the broadcast control system includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the message processing method based on the broadcast control system as described in any one of claims 1 to 8.

10. A computer storage medium, characterized in that: The computer storage medium is a computer-readable storage medium, and a computer program is stored on the computer storage medium. When the computer program is executed by a processor, the steps of the message processing method based on the broadcast control system as described in any one of claims 1 to 8 are implemented.

Images