Information flow processing method, device, equipment, storage medium and program product

Abstract

The present application provides a method, apparatus, device, storage medium and program product for processing information flow; the method includes: receiving information flow; based on the service identification of the information in the information flow, performing first-level aggregation on the information in the information flow to obtain multiple first information sets, wherein different first information sets are associated with different service identifications; for the information in each first information set, based on the device identification in the information, performing second-level aggregation on the information in the first information set to obtain multiple second information sets, wherein different second information sets are associated with different device identifications; for the information in the second information set, sending the information to the device corresponding to the device identification in the information. Through the present application, the processing efficiency of information can be improved.

Description

Technical Field

[0001] The present application relates to data processing technology, and in particular to an information flow processing method, device, equipment, storage medium and program product. Background Art

[0002] When processing information, the push system often processes it one by one. That is, each time the upstream server receives a piece of information to be pushed, it stores the information in the channel corresponding to the device identifier of the information. When the downstream server sends the information to the corresponding terminal, it classifies the information according to the business identifier corresponding to the information, and sends the classified information to terminals of different manufacturers, resulting in a large delay. In a scenario with high information concurrency, processing one by one can easily lead to information blocking, resulting in too low information processing efficiency. Summary of the invention

[0003] The embodiments of the present application provide a method, apparatus, device, storage medium and program product for processing an information flow, which can improve the efficiency of information processing.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides a method for processing an information flow, the method comprising:

[0006] Receive information streams;

[0007] Based on the service identifier of the information in the information flow, performing first-level aggregation on the information in the information flow to obtain multiple first information sets, wherein different first information sets are associated with different service identifiers;

[0008] For each piece of information in the first information set, based on the device identifier in the information, perform a second-level aggregation on the information in the first information set to obtain multiple second information sets, wherein different second information sets are associated with different device identifiers;

[0009] For the information in the second information set, the information is sent to a device corresponding to the device identifier in the information.

[0010] The embodiment of the present application provides a device for processing an information flow, the device comprising:

[0011] An information receiving module, used for receiving information flow;

[0012] A first aggregation module, configured to perform first-level aggregation on the information in the information flow based on the service identifier of the information in the information flow to obtain multiple first information sets, wherein different first information sets are associated with different service identifiers;

[0013] A second aggregation module is used to perform a second-level aggregation on the information in each of the first information sets based on the device identifier in the information to obtain multiple second information sets, wherein different second information sets are associated with different device identifiers;

[0014] An information sending module is used to send the information in the second information set to a device corresponding to the device identifier in the information.

[0015] An embodiment of the present application provides an electronic device, the electronic device comprising:

[0016] A memory for storing computer executable instructions or computer programs;

[0017] The processor is used to implement the information flow processing method provided in the embodiment of the present application when executing the computer executable instructions or computer program stored in the memory.

[0018] An embodiment of the present application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the information flow processing method provided in the embodiment of the present application when executed by a processor.

[0019] An embodiment of the present application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, the information flow processing method provided by the embodiment of the present application is implemented.

[0020] The embodiments of the present application have the following beneficial effects:

[0021] First, the information in the information flow is aggregated at the first level through the business identifier, and then the information in each first information set obtained by the first aggregation is aggregated at the second level through the device identifier. Before sending the information to the corresponding terminal, the information is classified and processed at the business and device levels, so that the classified information can be sent directly to the corresponding terminal, reducing the information delay and complexity caused by classifying the information before sending, saving processing time, and improving the convenience of information processing. In high-concurrency scenarios, batch processing of information through multi-level aggregation also avoids the situation where information processing is blocked due to slow processing of one piece of information, thereby improving the efficiency of information processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the architecture of the information flow processing system 100 provided in an embodiment of the present application;

[0023] Figure 2is a schematic diagram of the structure of the server 200 provided in an embodiment of the present application;

[0024] Figure 3A It is a first flow chart of the information flow processing method provided in the embodiment of the present application;

[0025] Figure 3B It is a second flow chart of the information flow processing method provided in the embodiment of the present application;

[0026] Figure 3C It is a third flow chart of the information flow processing method provided in the embodiment of the present application;

[0027] Figure 3D It is a fourth flow chart of the information flow processing method provided in the embodiment of the present application;

[0028] Figure 3E It is a fifth flow chart of the information flow processing method provided in the embodiment of the present application;

[0029] Figure 3F It is a sixth flow chart of the information flow processing method provided in the embodiment of the present application;

[0030] Figure 3G It is a seventh flow chart of the information flow processing method provided in the embodiment of the present application;

[0031] Figure 3H This is an eighth flow chart of the information flow processing method provided in the embodiment of the present application;

[0032] Fig. 3I This is a ninth flow chart of the information flow processing method provided in the embodiment of the present application;

[0033] Figure 3J is a tenth flow chart of the information flow processing method provided in an embodiment of the present application;

[0034] Figure 4A It is a schematic diagram of the division of the channel cache provided in an embodiment of the present application;

[0035] Figure 4B It is a training principle diagram of the machine learning model provided in the embodiment of the present application;

[0036] Figure 5A It is an information diagram of various product types provided in the embodiments of the present application;

[0037] Figure 5B This is a schematic diagram of the style of push information provided in the embodiment of the present application;

[0038] Figure 6 It is a schematic diagram of the principle of information push of the recommendation system provided in the embodiment of the present application;

[0039] Figure 7 It is a schematic diagram of the principle of channel caching provided in an embodiment of the present application;

[0040] Figure 8 It is a first principle schematic diagram of the secondary channel cache provided by an embodiment of the present application;

[0041] Fig. 9 It is a second principle schematic diagram of the secondary channel cache provided by an embodiment of the present application;

[0042] Fig.10 It is a schematic diagram of the principle of dynamically adjusting channel parameters provided in an embodiment of the present application.

[0043] It should be pointed out that the above-mentioned "first" and "second" are only used to distinguish different solutions, and do not represent the degree of superiority or inferiority of the solutions or the priority in the implementation process. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0045] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0046] In the following description, the terms "first / second / third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0047] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0048] Unless otherwise specified, at least one mentioned below refers to one or more situations, and "plurality" may refer to two or more situations.

[0049] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0050] The relevant data collection and processing in the embodiments of this application should be strictly in accordance with the requirements of relevant laws and regulations when applied in examples, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the personal information subject.

[0051] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0052] 1) Information flow refers to the collection of ordered information generated from information sources, processed and aggregated at multiple levels, and finally sent to specific devices in the process of information recommendation according to certain rules and order. In information flow management, it usually includes the generation, processing, transmission, storage and consumption of information.

[0053] 2) Business ID is an identifier used to uniquely identify and distinguish different business areas in information. In information management systems, business IDs are used to distinguish which specific business area or business line the information belongs to. For example, in a company with multiple businesses, business IDs can be used to distinguish which business the information belongs to, such as different business areas such as video, news or music. Business IDs can be business numbers, business types, business status or business keywords, etc.

[0054] 3) Device identification is an identifier used in information to uniquely identify and distinguish devices produced by different manufacturers. In the information management system, the device manufacturer identification is used to indicate that the information will eventually be sent to a device produced by a specific manufacturer. The device manufacturer identification is usually related to the manufacturer, brand, model, operating system, and hardware characteristics of the device. By using the device manufacturer identification, the communication and data flow with the devices of a specific manufacturer can be better managed and optimized, and the recognition efficiency of the device and the accuracy of information transmission can be improved. The device identification can be a device number, device type, device status, or device keyword.

[0055] 4) Channel cache is a storage structure used to store and aggregate information, which is usually associated with a specific business identifier or device identifier. In an information management system, the channel cache is used to temporarily store information and aggregate this information into a higher-level cache or storage structure as needed. The channel cache can be physical memory or disk storage, or it can be part of a cloud storage or distributed cache system.

[0056] 5) Equipment manufacturers refer to commercial entities that manufacture, produce and sell various electronic devices. Equipment manufacturers are not only responsible for designing, developing and producing equipment, but also responsible for providing corresponding software and hardware support to ensure the reliability of the equipment and user experience. Each equipment manufacturer corresponds to multiple equipment identifiers.

[0057] 6) Information template identification refers to the identifier used to uniquely identify and distinguish information templates in information. It is a classification mechanism used to ensure that information can be accurately classified into a specific template or pattern based on its content or structure. Information template identification is usually associated with the key characteristics or structure of the information, such as data format, business logic or display method. By using information template identification, the processing and aggregation process of information can be managed and optimized to ensure that the same or similar information can be aggregated together for further processing or distribution. For example, when sending notification information, different notification types, such as meeting notifications, order confirmations or system updates, can use different information template identifications so that the system can aggregate similar notification information together and process and display it in a unified manner.

[0058] 7) Device parameters refer to parameters used to describe and distinguish the characteristic information of a device. Device parameters may include the manufacturer name, device size, model, system version, production date, serial number, and status information of the device. Device parameters can effectively identify and distinguish different devices for accurate management, configuration, and maintenance. Device identification includes multiple device parameters to determine the location and role of the device in the system. In addition, device parameters can also be used to guide the distribution and processing of information, such as filtering and processing information based on specific attributes of the device (such as system version or production date).

[0059] 8) Buffer refers to the memory area in the L2 cache channel used to temporarily store data. In the L2 channel cache, the buffer is used to store information that is about to be processed or is being processed. The size of the buffer is usually limited, which determines the upper limit of the amount of data that can be stored in the L2 channel cache. The buffer is designed to reduce the number of accesses to the underlying storage system and improve the speed and efficiency of data processing.

[0060] 9) Channel parameters refer to the parameters used to characterize the information processing performance of the secondary channel cache. Channel parameters include parallelism, maximum number of batches processed, maximum duration of batch aggregation, and buffer size. Channel parameters are used to guide how the secondary channel cache processes and stores information efficiently.

[0061] Parallelism: Indicates the number of concurrent working threads in each L2 channel cache. A higher parallelism can improve the processing power of the channel cache.

[0062] Maximum number of batches: This parameter indicates the maximum number of messages that can be processed in one batch by each secondary channel cache. This parameter determines the amount of messages that can be processed by the channel cache in one processing operation.

[0063] Maximum duration of a batch aggregation: represents the maximum duration of the second-level aggregation of each batch in each secondary channel cache. This parameter is used to control the time limit of information aggregation to prevent long-term processing operations from occupying resources.

[0064] Buffer size: represents the upper limit of the amount of data that can be stored in each L2 channel cache. This parameter determines the size of the buffer, which directly affects the amount of data that the channel cache can handle.

[0065] 10) Object group identification refers to an identifier used to uniquely identify and distinguish different object groups in information. Object group identification includes a combination of information such as object characteristics, preferences, region, education level, object level and consumption capacity. By using object group identification, the processing and aggregation process of information can be better managed and optimized, ensuring that the same or similar object group information can be aggregated together for further processing or distribution.

[0066] 11) Message middleware is a distributed message queue that stores information in the order in which it is received, providing asynchronous message delivery and distributed message queue functions. Message middleware allows applications to communicate by publishing messages without knowing the location or status of other applications. Messages are placed in queues, from which applications can read messages or persist them in storage systems. Message middleware can be an open source distributed stream processing platform (Kafka) for building real-time data pipelines and streaming applications. Kafka processes messages by maintaining an ordered byte stream in its distributed system, supports high throughput and scalability, and is suitable for processing large-scale data streams.

[0067] 12) Operation code refers to the code used to indicate the result of a specific operation in a software application. It is usually returned by the server to the client after executing the request. The operation code usually contains the result status of the operation, such as success or failure, and possible reasons for the status or error code. By checking the operation code, you can determine whether the request is successfully processed and take appropriate actions as needed, such as retrying, logging, or notifying the user.

[0068] When sending information to the corresponding device, the relevant technology usually adopts a one-by-one processing method to process push messages. It is impossible to classify and process messages from different businesses or manufacturers during the information writing stage of the upstream server, resulting in additional classification processing when the downstream server sends the information to the corresponding terminal, causing a large delay. In the scenario of high information concurrency, the one-by-one processing method is prone to cause information blocking, resulting in too low information processing efficiency.

[0069] Based on the above analysis, the applicant found that the information flow processing method of the related technology cannot aggregate information before it is consumed, nor can it process information under high concurrency. To address the above problems, the embodiments of the present application provide a method for processing information flow that can improve the efficiency of information processing.

[0070] The following describes an exemplary application of the electronic device provided by the embodiment of the present application. The electronic device provided by the embodiment of the present application can be implemented as various types of terminals such as laptops, tablet computers, desktop computers, set-top boxes, smart phones, smart speakers, smart watches, smart TVs, and vehicle-mounted terminals. For example, in a multi-terminal local area network environment, a part of the terminals sends the information stream to another part of the terminals through the information stream processing method of the embodiment of the present application. It can also be implemented as a server. For example, in a recommendation system, the server sends the information stream to the terminal by implementing the information stream processing method of the embodiment of the present application. Below, an exemplary application when the electronic device is implemented as a server will be described.

[0071] See also Figure 1 , Figure 1 It is a schematic diagram of the architecture of the information flow processing system 100 provided in an embodiment of the present application. To implement a processing application that supports an information flow, a terminal (terminal 400-1 and terminal 400-2 are shown as examples) is connected to a server 200 via a network 300. The network 300 may be a wide area network or a local area network, or a combination of the two.

[0072] The server 200 is used to perform multi-level aggregation on the information in the information flow, and send the aggregated result to the device corresponding to the device identifier in the information. The terminal 400-1 or the terminal 400-2 is used to receive the information sent by the server 200 and display it on a graphical interface (graphic interface 410-1 and graphical interface 410-2 are shown as examples).

[0073] Taking the network media scenario as an example, the server 200 sends information of different media applications to the device corresponding to the device identifier in the information. If the terminal 400-1 and the terminal 400-2 correspond to the same device identifier and service identifier, that is, the terminal 400-1 and the terminal 400-2 are the same type of devices and use the same media application, then the terminal 400-1 or the terminal 400-2 receives the same information sent by the server 200 and displays it on the graphical interface (graphic interface 410-1 and graphical interface 410-2 are shown as examples).

[0074] In some embodiments, the server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0075] See also Figure 2 , Figure 2 is a schematic diagram of the structure of the server 200 provided in an embodiment of the present application, Figure 2 The server 200 shown includes: at least one processor 210, a memory 230 and at least one network interface 220. The various components in the server 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 240 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 240 is not described in detail. Figure 2 Various buses are labeled as bus system 240 .

[0076] The processor 210 may be an integrated circuit chip having signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor may be a microprocessor or any conventional processor, etc.

[0077] The memory 230 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 230 may optionally include one or more storage devices that are physically remote from the processor 210.

[0078] The memory 230 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 230 described in the embodiment of the present application is intended to include any suitable type of memory.

[0079] In some embodiments, memory 230 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0080] Operating system 231, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0081] A network communication module 232, used to reach other electronic devices via one or more (wired or wireless) network interfaces 220, exemplary network interfaces 220 include: Bluetooth, Wireless LAN (WiFi), and Universal Serial Bus (USB), etc.;

[0082] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2 The information flow processing device 233 stored in the memory 230 is shown, which can be software in the form of a program and a plug-in, etc., including the following software modules: an information receiving module 2331, a first aggregation module 2332, a second aggregation module 2333 and an information sending module 2334. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be explained below.

[0083] In some embodiments, the terminal or server can implement the information flow processing method provided in the embodiment of the present application by running various computer executable instructions or computer programs. For example, the computer executable instructions can be microprogram-level commands, machine instructions or software instructions. The computer program can be a native program or software module in the operating system; it can be a program that needs to be installed in the operating system to run. In short, the above-mentioned computer executable instructions can be instructions in any form, and the above-mentioned computer program can be an application, module or plug-in in any form.

[0084] See also Figure 3A , Figure 3Ais a first flow chart of the information flow processing method provided in the embodiment of the present application. The execution subject may be the server or terminal mentioned above. The following takes the server as an example. Figure 3A The steps shown are explained.

[0085] In step 101, an information stream is received.

[0086] In some embodiments, the information flow includes multiple information, wherein the information includes a service identifier, a device identifier, an information template identifier, an object group identifier, and a payload, and the payload is the actual content of the information. The information flow may include multiple types of information, such as at least one of text, pictures, videos, and audio.

[0087] In some embodiments, see Figure 3B , Figure 3B 1 is a second flow chart of the information flow processing method provided in the embodiment of the present application. Before step 101, execute Figure 3B Steps 201 to 203 are described in detail below.

[0088] In step 201, the information in the information flow to be pushed is detected to obtain a detection result.

[0089] In some embodiments, identity detection, flow detection, integrity detection, compliance detection, etc. are performed on the information in the information flow to be pushed, wherein identity detection refers to the verification of the identity of the information sending platform to ensure the credibility of the information source, and identity detection can be performed through encryption authentication, digital signatures or access lists; flow detection refers to detecting whether the push flow of the information flow exceeds the flow threshold, and the size, frequency or sending time of the flow can be used as the basis for flow detection; integrity detection refers to detecting whether the information has been tampered with or damaged during transmission, which is usually achieved through data verification and digital signature technology, such as hash functions or message authentication codes; compliance detection refers to detecting whether the information complies with regulations, involving content review, encryption requirements, and compliance checks on data storage and transmission.

[0090] In step 202, in response to the detection result being passed, information is written into the message middleware.

[0091] In some embodiments, the test results include pass and fail. If any test fails, the test result is fail. If all tests pass, the test result is pass. First, ensure that the message middleware (Kafka) is correctly configured and running, including configuring topics, partitions, and replicas. Then, determine the format in which the information should be written to the message middleware, such as open standard file formats and data exchange formats (JavaScript Object Notation, JSON), Extensible Markup Language (Extensible Markup Language, XML), or Data Description Language (Protocol Buffers, ProtoBuf), etc., select a format suitable for storage and transmission; construct the message to be sent according to the selected message format, and the message usually includes a header and a body; if the message contains complex data types, such as objects or arrays, serialize the message to convert it into a byte sequence suitable for transmission; select a suitable message producer class of the message middleware and send the message to the message middleware. The producer class provides a variety of sending options, such as asynchronous sending or synchronous sending; configure the producer parameters, such as the number of retries, request timeout or compression options, to ensure the reliability and performance of the message; use the configured producer to send the message to the message middleware, and you can choose to send the message to a specific topic; detect the message production and sending process, and record key log information for troubleshooting and performance analysis; in some cases, it is necessary to implement a message confirmation mechanism to ensure that the message has been successfully sent to the message middleware, which can be achieved through manual confirmation or timed automatic confirmation.

[0092] In step 203, a preset amount of information is read from the message middleware in a storage order from first to last, so as to be combined into an information flow to be pushed.

[0093] In some embodiments, the information in the message middleware is in the form of a queue and stored in sequence from the first to the last time it is received. That is, the information that arrives first is stored at the front of the message middleware, and the information that arrives later is stored at the back of the message middleware. A preset number of information is read from the message middleware in sequence from the first to the last time the information is received, and the preset number of information is combined into an information stream to be pushed.

[0094] The embodiment of the present application detects information and stores the detected information in order in the message middleware, avoiding the problem of traffic overload caused by a surge in messages, so that the messages are always executed in order in the message middleware, reducing instantaneous traffic and saving resource usage.

[0095] Continue to see Figure 3AIn step 102, based on the service identifier of the information in the information flow, the information in the information flow is first aggregated to obtain multiple first information sets, wherein different first information sets are associated with different service identifiers.

[0096] In some embodiments, the first information set and the business identifier are in a one-to-one correspondence, for example, there are two first information sets, namely, the first information set A and the first information set B, and two business identifiers, namely, business identifier A and business identifier B, wherein the first information set A corresponds to business identifier A, and the first information set B corresponds to business identifier B. Business identifiers related to the business are extracted from the information stream, such as business ID (Identity Document, ID), business type, business status or business keywords, etc.; multiple first information sets are created based on the extracted business identifiers, and the information in each first information set will be associated with the same business identifier; the information in the information stream is classified into the corresponding first information set based on the corresponding business identifier, which can be achieved through a mapping table, a hash table or an index.

[0097] For example, the information flow includes information A, information B, information C and information D, wherein information A and information B both include business identification A, and information C and information D both include business identification B. Information A and information B are placed in a first information set A, and information C and information D are placed in a first information set B, wherein the first information set A corresponds to business identification A, and the first information set B corresponds to business identification B.

[0098] In some embodiments, a plurality of storage spaces corresponding to a plurality of service identifiers are pre-allocated in the primary channel cache. Figure 3C , Figure 3C This is a third flow chart of the information flow processing method provided in an embodiment of the present application. Figure 3A Step 102 of "performing a first-level aggregation on the information in the information flow based on the service identifier of the information in the information flow to obtain multiple first information sets" can be performed by Figure 3C Steps 1021 to 1023 are implemented as described in detail below.

[0099] For example, see Figure 4A , Figure 4A The first-level channel cache is pre-allocated with multiple storage spaces corresponding to multiple service identifiers. For example, in the first-level channel cache, service identifier A, service identifier B and service identifier C each correspond to a storage space.

[0100] In step 1021, based on the service identifier of each information in the information flow, a target storage space corresponding to the information is searched in multiple storage spaces.

[0101] In some embodiments, in the primary channel cache, each service identifier corresponds to a storage space. A mapping mechanism is established to map the service identifier to the corresponding storage space, which can be implemented through a hash table, an index, or a database table. Mapping is performed based on the service identifier of each information in the information flow, and the target storage space corresponding to the information is queried in multiple storage spaces.

[0102] In step 1022, each information is written into the target storage space corresponding to each information.

[0103] In some embodiments, a lock mechanism can be introduced to control access to storage space. The lock mechanism can be lock-free, such as optimistic concurrency control (OCC); timestamp-based, such as timestamp lock (TSL); row-based, such as optimistic concurrency control or pessimistic concurrency control (PCC). Concurrent access strategies such as read-write separation, write-write conflict handling, or read-write conflict handling are defined to ensure the correctness of data in a concurrent environment.

[0104] In step 1023, the set of information in each storage space is taken as a first information set.

[0105] In some embodiments, the information in each first information set corresponds to the same service identifier.

[0106] For example, there are three storage spaces in total, each storage space contains 4 information, for example, the first storage space includes information A, information B, information C and information D, the second storage space includes information E, information F, information G and information H, and the third storage space includes information I, information J, information K and information L. Then the set of information A, information B, information C and information D is regarded as a first information set, and information A, information B, information C and information D correspond to the same business identifier; the set of information E, information F, information G and information H is regarded as a first information set, and information E, information F, information G and information H correspond to the same business identifier; the set of information I, information J, information K and information L is regarded as a first information set, and information I, information J, information K and information L correspond to the same business identifier.

[0107] The embodiment of the present application divides the first-level channel cache into multiple storage spaces through business identifiers, and uses the information in each storage space as a first information set, so that the information can be quickly divided at the business level according to different business identifiers. When querying the information of a specific business identifier, the query time is reduced, the query efficiency is improved, and data management is convenient.

[0108] In some embodiments, the first level aggregation is performed on the information flow in batches. Before step 102 "performing the first level aggregation on the information in the information flow based on the business identification of the information in the information flow", the following processing can also be performed: when the amount of information in the received information flow is equal to the quantity threshold, the information of the quantity threshold is treated as a batch; or, when the received duration of the information flow is equal to the receiving duration threshold, the information received within the receiving duration threshold is treated as a batch.

[0109] For example, the quantity threshold is 5, and the reception duration threshold is 2 minutes. When the number of messages in the received information flow is equal to 5, the 5 messages are aggregated as a batch at the first level. If the number of messages in the received information flow is less than 5, for example 3, but the reception duration of the information flow is equal to 2 minutes, the 3 messages are aggregated as a batch at the first level.

[0110] In some embodiments, each time a message is received, a target storage space corresponding to the message can be determined based on the business identifier of the information in the information flow, the information can be written into the target storage space, and in response to the amount of information stored in the target storage space reaching a space threshold, the information in the target storage space can be used as a first information set.

[0111] The embodiment of the present application ensures that each piece of information is clustered at the first level by processing the received unprocessed information one by one, thereby improving the real-time performance of information processing.

[0112] Continue to see Figure 3A In step 103, for each information in the first information set, based on the device identifier in the information, the information in the first information set is aggregated at the second level to obtain multiple second information sets, wherein different second information sets are associated with different device identifiers.

[0113] In some embodiments, the second information set and the device identification are in a one-to-one correspondence, for example, there are two second information sets, namely, second information set A and second information set B, and two device identifications, namely, device identification A and device identification B, wherein the second information set A corresponds to device identification A, and the second information set B corresponds to device identification B. Extract the device identification related to the device from the information, such as the device number (Identity Document, ID), device type, device status or device keyword, etc.; create multiple second information sets based on the extracted device identification, and the information in each second information set will be associated with the same device identification; classify the information in each first information set into the corresponding second information set according to the corresponding device identification, which can be achieved through a mapping table, a hash table or an index.

[0114] In some embodiments, each device manufacturer corresponds to multiple device identifiers, and each device manufacturer pre-allocates a corresponding secondary channel cache. Figure 3D , Figure 3D This is a fourth flow chart of the information flow processing method provided in an embodiment of the present application. Figure 3A Step 103 of “performing a second-level aggregation on the information in the first information set based on the device identifier in the information” can be performed by executing Figure 3D Steps 1031 to 1033 are implemented as described in detail below.

[0115] For example, see Figure 4A Each device manufacturer is pre-allocated with a corresponding secondary channel cache. For example, device manufacturer 1 and device manufacturer 2 each correspond to a secondary channel cache.

[0116] In step 1031, based on the device identification in the information, the target device manufacturer corresponding to the device identification is queried.

[0117] In some embodiments, each device manufacturer corresponds to multiple device identifiers, and a mapping mechanism is established to map the device identifiers to the corresponding device manufacturers, which can be implemented through a hash table, index or database table. Mapping is performed based on the device identifier in the information, and the device manufacturer corresponding to the information is queried among multiple device manufacturers.

[0118] In some embodiments, see Figure 3E , Figure 3E This is the fifth flow chart of the information flow processing method provided in the embodiment of the present application. Figure 3D Step 1031 can be Figure 3E Steps 10311 to 10312 are implemented as described in detail below.

[0119] In step 10311, based on the device identifier in the information, the target device manufacturer corresponding to the device identifier is queried from the distributed cache, wherein the distributed cache is stored in the form of key-value pairs, the key in the key-value pair is the device identifier, and the value in the key-value pair is the device manufacturer corresponding to the device identifier.

[0120] In some embodiments, in the distributed cache, multiple device identifiers and corresponding device manufacturers are pre-stored in the form of key-value pairs. Different device identifiers can correspond to the same device manufacturer. The distributed cache allows multiple information to query the corresponding target device manufacturer based on the device identifier at the same time.

[0121] For example, in a distributed cache, if the key is device ID A, the value corresponding to device ID A is device manufacturer A, and device ID A-device manufacturer A is used as a key-value pair; if the key is device ID B, the value corresponding to device ID B is device manufacturer B, and device ID B-device manufacturer B is used as a key-value pair.

[0122] In step 10312, in response to not finding the target device manufacturer corresponding to the device identifier in the distributed cache, the target device manufacturer corresponding to the device identifier is queried from the database.

[0123] In some embodiments, the database includes multiple types, each type supports different query languages ​​and query methods, such as relational database management system (RDBMS), which supports structured query language (SQL), and uses a table to store device identifiers and corresponding target device manufacturers; non-relational database (NoSQL), which supports multiple query methods, including key-value query, document query, column family query and graph query; key-value storage database, which usually uses key-value pairs to store data, and the query is based on the key to obtain the corresponding value, the key can be the device identifier, and the value is the target device manufacturer corresponding to the device identifier; document database, data is stored in the form of documents, and the query is usually based on the document structure and content; column family database, data is stored in columns, and the query usually involves a large number of data aggregation operations; graph database, which uses graphs (nodes and edges) to represent data, and the query is based on the path and pattern of the graph; object database, data is stored in the form of objects, and the query usually involves object-relationship mapping; in-memory database, data is stored in memory, providing extremely fast access speed.

[0124] The embodiment of the present application first queries the target device manufacturer corresponding to the device identifier in the distributed cache, allowing multiple information to be queried at the same time, and the cache has a shorter response time than the memory, which reduces the query time and improves the query efficiency. When the target device manufacturer is not found in the distributed cache, the target device manufacturer corresponding to the device identifier is queried in the database, which provides a query backup for the distributed cache query, improves the query fault tolerance rate, and ensures that the corresponding target device manufacturer can be queried through the device identifier.

[0125] Continue to see Figure 3D In step 1032, based on the target device manufacturer, a target L2 channel cache corresponding to the information is determined from the L2 channel cache pre-allocated by each device manufacturer.

[0126] In some embodiments, different device manufacturers correspond to different secondary channel caches. A mapping mechanism is established to map device manufacturers to corresponding secondary channel caches, which can be implemented through a hash table, an index, or a database table. Mapping is performed based on the target device manufacturer corresponding to the device identifier, and the target secondary channel cache corresponding to the information is queried in multiple secondary channel caches.

[0127] In step 1033, the information is written into the target L2 channel cache.

[0128] In some embodiments, a suitable cache strategy is selected according to application requirements and characteristics of the target secondary channel cache, such as Least Recently Used (LRU), Least Frequently Used (LFU), or First Input First Output (FIFO). If the information is a complex data type, such as an object or an array, it needs to be serialized into a data format acceptable to the target secondary channel cache, usually a byte stream or a string. Use the cache application programming interface (API) to write the information to the target secondary channel cache. You can set an expiration time (Time To Live, TTL) to control the timeliness of the information in the target secondary channel cache. Process the results of the write to the target secondary channel cache, such as checking whether the write is successful or handling possible exceptions.

[0129] After the first-level aggregation based on the business identifier is performed to obtain multiple first information sets, the embodiment of the present application performs a second-level aggregation on the information in each first information set based on the device identifier. In the information writing stage, the information is differentiated and aggregated from the business level and the device level, avoiding additional classification processing during subsequent information consumption, reducing the complexity and time of information processing, and improving the efficiency of information classification processing. By splitting the information corresponding to different equipment manufacturers into different secondary aggregation channels, the information corresponding to each equipment manufacturer is independently circulated in its respective secondary channels, and the messages are consumed according to their respective processing capabilities and speeds, avoiding the problem of overall channel blocking due to the slow processing speed of a certain equipment manufacturer's server.

[0130] In the embodiment of the present application, after performing a first-level aggregation of information based on a small number of business identifiers, a second-level aggregation is performed on the information after the first aggregation based on a large number of device identifiers. By adopting an aggregation method of first coarse classification and then fine classification, compared with an aggregation method of first performing a first-level aggregation of information based on a large number of device identifiers and then performing a second aggregation on the information after the first-level aggregation based on a small number of business identifiers, the time of the two aggregations is reduced and the efficiency of information aggregation is improved.

[0131] In some embodiments, the secondary channel cache includes multiple third-level channel caches, and one third-level channel cache corresponds to one information template identifier, see Figure 3F , Figure 3F This is the sixth flow chart of the information flow processing method provided in the embodiment of the present application. Figure 3D Step 1033 can be achieved by Figure 3F Steps 10331 to 10332 are implemented as described in detail below.

[0132] For example, see Figure 4A The secondary channel cache includes multiple third-level channel caches, and one third-level channel cache corresponds to one information template identifier. For example, in the secondary channel cache corresponding to device manufacturer 1, information template identifier 1 and information template identifier 2 each correspond to one third-level channel cache; in the secondary channel cache corresponding to device manufacturer 2, information template identifier 1, information template identifier 3, and information template identifier 5 each correspond to one third-level channel cache.

[0133] In step 10331, based on the information template identifier in the information, a target third-level channel cache corresponding to the information is determined in multiple third-level channels.

[0134] In some embodiments, an information template identifier is extracted from the information, and the information template identifier is an identifier used to distinguish different information types. A mapping table of the information template identifier and the corresponding three-level channel cache is established, and the mapping table uses the information template identifier as a key and the three-level channel cache corresponding to the information template identifier as a value. The extracted information template identifier is used as a keyword to query the mapping table to determine the target three-level channel cache corresponding to the information.

[0135] In step 10332, the information is written to the target L3 channel cache.

[0136] In some embodiments, writing information to the queried target third-level channel cache involves the use of cache APIs, such as setting key-value pairs or updating cache data. Apply cache policies, such as cache expiration time, cache size management, or cache update logic, to ensure the timeliness and accuracy of cache data. Record detailed information about cache write operations, such as time, key-value pairs, or cache names, to facilitate subsequent detection and troubleshooting. If the consistency of the cache needs to be maintained, mechanisms such as cache refresh or data synchronization can be implemented to ensure that the data in the target third-level channel cache is consistent with the original data source. Implement exception handling mechanisms, such as handling of abnormal situations such as cache unavailability or data inconsistency. Before writing information to the target third-level channel cache, check whether there are sufficient permissions for write operations and implement appropriate security measures, such as data encryption or access control.

[0137] The embodiment of the present application further subdivides the information in the second information set through the information template identifier in the information, so that information of the same service, the same device and using the same information template is aggregated together, forming a multi-level aggregation structure of service-manufacturer-template, so as to facilitate batch sending of specific types of information and improve the efficiency of information sending.

[0138] In some embodiments, the device identification includes multiple device parameters, the third-level channel cache includes multiple fourth-level channel caches, and one fourth-level channel cache corresponds to a combination of multiple device parameters, see Figure 3G , Figure 3G This is the seventh flow chart of the information flow processing method provided in the embodiment of the present application. Figure 3F Step 10332 can be Figure 3G Steps 301 to 302 are implemented as described below.

[0139] For example, see Figure 4A , the device identification includes multiple device parameters, the third-level channel cache includes multiple fourth-level channel caches, and one fourth-level channel cache corresponds to a combination of multiple device parameters. For example, for the second-level channel cache corresponding to device manufacturer 1, in the third-level channel cache corresponding to information template identification 1, a combination of multiple device parameters 1 and a combination of multiple device parameters 2 each correspond to a fourth-level channel cache; in the third-level channel cache corresponding to information template identification 2, a combination of multiple device parameters 3 and a combination of multiple device parameters 4 each correspond to a fourth-level channel cache. For the second-level channel cache corresponding to device manufacturer 2, in the third-level channel cache corresponding to information template identification 1, a combination of multiple device parameters 1 and a combination of multiple device parameters 2 each correspond to a fourth-level channel cache; in the third-level channel cache corresponding to information template identification 3, a combination of multiple device parameters 3 and a combination of multiple device parameters 4 each correspond to a fourth-level channel cache; in the third-level channel cache corresponding to information template identification 5, a combination of multiple device parameters 5 and a combination of multiple device parameters 6 each correspond to a fourth-level channel cache.

[0140] In step 301, based on a combination of multiple device parameters of the information, a target L4 channel cache corresponding to the information is searched in multiple L4 channel caches.

[0141] In some embodiments, the plurality of device parameters include device manufacturer name, device size, device model, device system version, production date, serial number or status information, etc. A four-level channel buffer corresponds to a combination of the plurality of device parameters.

[0142] For example, the first four-level channel cache corresponds to a combination of device manufacturer name A, device size A, device model A, device system version A, production date A, serial number A or status information A, and the second four-level channel cache corresponds to a combination of device manufacturer name B, device size B, device model B, device system version B, production date B, serial number B or status information B.

[0143] In some embodiments, relevant device parameters are extracted from the information, and the extracted multiple device parameters are combined to obtain multiple device parameter combinations. A mapping table of device parameter combinations to their corresponding L4 channel caches is established, with the device parameter combination as a key and the corresponding L4 channel cache as a value. The mapping table is queried using the device parameter combination as a keyword to determine the target L4 channel cache corresponding to the information.

[0144] In step 302, information is written into the target L4 channel cache.

[0145] In some embodiments, information is written to the queried target L4 channel cache through a cache API, such as setting a key-value pair or updating cache data. Cache policies are applied, such as cache expiration time, cache size management, or cache update logic, to ensure the timeliness and accuracy of cache data in the target L4 channel. Detailed information of cache write operations, such as time, key-value pairs, or cache name, is recorded to facilitate subsequent detection and troubleshooting. If the consistency of the cache needs to be maintained, mechanisms such as cache refresh or data synchronization can be implemented to ensure that the data in the cache is consistent with the original data source. Exception handling mechanisms are implemented, such as handling of abnormal situations such as cache unavailability or data inconsistency. Before writing to the target L4 channel cache, check whether there are sufficient permissions for write operations and implement appropriate security measures, such as data encryption or access control.

[0146] The embodiment of the present application divides the information into a finer level through the device parameters of the information, so that the information can be sent more specifically and the device to which the information corresponds can be quickly determined. When searching for a certain type of information, the specific type of information can be accurately located, thereby reducing the scope of the queried information and improving query efficiency and performance.

[0147] In some embodiments, each secondary channel cache includes a buffer and a plurality of channel parameters. The buffer is used to store information in the secondary channel cache. The channel parameters represent the processing performance of the secondary channel cache for information. Figure 3H , Figure 3H 1033, execute Figure 3H Steps 401 to 404 are described in detail below.

[0148] In some embodiments, the buffer is a memory area for storing information in the secondary channel cache. The multiple channel parameters include parallelism, a maximum number of batches processed, a maximum duration of a batch aggregation, and a buffer size, wherein the parallelism represents the number of simultaneous working threads in each secondary channel cache, the maximum number of batches processed represents the maximum number of information that can be processed in a batch by each secondary channel cache, the maximum duration of a batch aggregation represents the maximum duration of each batch of each secondary channel cache that can perform second-level aggregation, and the buffer size represents the upper limit of the amount of data that can be stored in each secondary channel cache.

[0149] For example, the degree of parallelism is 20, indicating that 20 working threads are executed simultaneously in each secondary channel cache; the maximum number of batches processed is 200 / batch, indicating that each secondary channel cache can process a maximum of 200 messages in one batch; the maximum duration of a batch aggregation is 200 milliseconds / batch, indicating that the maximum duration of the second-level aggregation for each batch of each secondary channel cache is 200 milliseconds; the buffer size is 10,000, indicating that the upper limit of the amount of data that can be stored in each secondary channel cache is 10,000 messages.

[0150] In step 401, based on the capacity and usage of the buffer in the target L2 channel cache, the utilization of the buffer in the target L2 channel cache is determined.

[0151] In some embodiments, a ratio of the usage of the buffer in the target L2 channel cache to the capacity of the buffer in the target L2 channel cache is determined, and the ratio is used as the utilization rate of the buffer in the target L2 channel cache.

[0152] For example, the capacity of the buffer in the target secondary channel cache is M, and the usage of the buffer in the target secondary channel cache is N. Then, the ratio of the usage of the buffer in the target secondary channel cache to the capacity of the buffer in the target secondary channel cache is N / M, and N / M is used as the utilization rate of the buffer in the target secondary channel cache.

[0153] In step 402, in response to a utilization rate of a buffer in a target L2 channel cache being greater than a first utilization rate threshold, a plurality of channel parameters are increased.

[0154] In some embodiments, the first utilization threshold represents a threshold at which the utilization may be adjusted upward.

[0155] Continuing with the example of step 401 above, if the first utilization threshold is 80%, the utilization of the buffer in the target secondary channel cache is N / M, and N / M=90%, then the utilization of the buffer in the target secondary channel cache is greater than the first utilization threshold, and multiple channel parameters are increased. For example, the degree of parallelism is increased from 20 to 30, indicating that 30 working threads are simultaneously executed in each secondary channel cache; the maximum number of batch processing is increased from 200 / batch to 300 / batch, indicating that each secondary channel cache can process a maximum of 300 messages in a batch; the maximum duration of a batch aggregation is increased from 200 milliseconds / batch to 100 milliseconds / batch, indicating that the maximum duration of the second-level aggregation per batch of each secondary channel cache is 100 milliseconds; the buffer size is increased from 10,000 to 11,000, indicating that the upper limit of the amount of data that can be stored in each secondary channel cache is 11,000 messages.

[0156] In step 403, in response to the utilization of the buffer in the target L2 channel cache being less than a second utilization threshold, a plurality of channel parameters are reduced.

[0157] In some embodiments, the second utilization threshold represents a threshold at which the utilization may be adjusted downward.

[0158] Continuing with the example of step 401 above, if the second utilization threshold is 20%, the utilization of the buffer in the target secondary channel cache is N / M, and N / M=10%, then the utilization of the buffer in the target secondary channel cache is less than the second utilization threshold, and multiple channel parameters are reduced. For example, the degree of parallelism is reduced from 20 to 10, indicating that 10 working threads are simultaneously executed in each secondary channel cache; the maximum number of batch processing is reduced from 200 / batch to 100 / batch, indicating that each secondary channel cache can process a maximum of 100 messages in a batch; the maximum duration of a batch aggregation is reduced from 200 milliseconds / batch to 500 milliseconds / batch, indicating that the maximum duration of the second-level aggregation per batch of each secondary channel cache is 500 milliseconds; the buffer size is reduced from 10,000 to 5,000, indicating that the upper limit of the amount of data that can be stored in each secondary channel cache is 5,000 messages.

[0159] In step 404, based on the updated multiple channel parameters, the maximum amount of information in the target secondary channel buffer is updated.

[0160] In some embodiments, the information in the current target secondary channel cache is analyzed, including information type, access mode, and update frequency, to determine the information processing requirements of the cache. Based on the updated channel parameters and cache information analysis results, the maximum amount of information that the target secondary channel cache can accommodate is calculated. Using the cache management API or configuration interface, the configuration of the target secondary channel cache is updated to set the maximum amount of information to the calculated value. Verify whether the updated cache configuration meets the performance requirements and adjust it according to the system operation status. Detect the performance of the cache system and collect feedback information, such as cache hit rate, processing delay, and resource utilization. Based on the detection results, further optimize the channel parameters and the maximum amount of information to achieve optimal cache performance.

[0161] The embodiment of the present application dynamically adjusts the channel parameters to avoid the situation where information and resource allocation is unbalanced due to different processing capabilities of the secondary channel cache, thereby improving the utilization rate of the channel cache, reducing resource occupancy, increasing the channel parameters for the channel cache with higher processing capabilities, and reducing the channel parameters for the channel cache with lower processing capabilities, thereby improving the rationality of resource allocation between different channel caches and improving the information processing efficiency.

[0162] In some embodiments, the channel parameter is positively correlated with the maximum amount of information. The larger the channel parameter is, the larger the maximum amount of information is, and the smaller the channel parameter is, the smaller the maximum amount of information is.

[0163] In some embodiments, the maximum amount of information in the secondary channel cache can be obtained through a pre-trained machine learning model, which can be achieved by performing the following processing: obtaining an initialized machine learning model; obtaining information samples and true labels, wherein the information samples include multiple channel parameters, and the true labels represent the maximum amount of information in the true secondary channel cache; calling the initialized machine learning model based on the information samples to obtain predicted labels, wherein the predicted labels represent the maximum amount of information in the secondary channel cache predicted by the initialized machine learning model; determining a loss value based on the true labels and the predicted labels, and updating the parameters of the initialized machine learning model based on the loss value to obtain a pre-trained machine learning model; calling the pre-trained machine learning model based on the information to obtain the maximum amount of information in the target secondary channel cache corresponding to the information, and writing the information with the maximum amount of information into the target secondary channel cache.

[0164] For example, see Figure 4B , Figure 4B This is a training principle diagram of the machine learning model provided in the embodiment of the present application. Based on the information sample, the convolution layer and the fully connected layer of the initialized machine learning model can be called to obtain the predicted label, the loss value can be determined based on the true label and the predicted label, and the parameters of the initialized machine learning model can be updated based on the loss value to obtain a pre-trained machine learning model.

[0165] In some embodiments, the initialization representation randomly assigns values ​​to the parameters of the machine learning model, for example, all the parameters of the machine learning model are assigned values ​​of 0 or all of 1.

[0166] In some embodiments, the difference between the true label and the predicted label is calculated as the loss value. The loss value is back-propagated to update the parameters of the initialized machine learning model, and the process of calculating the loss value and updating the parameters is iterated multiple times until the loss value no longer increases or decreases, and the iterative process is stopped to form a pre-trained machine learning model.

[0167] For example, back propagation is implemented through the back propagation algorithm, which calculates the gradient of each neuron from the output layer to the input layer, and updates the weight and bias of the neuron according to the gradient, and continuously updates the parameters by gradient descent to reduce the loss value. Gradient descent can use various gradient descent algorithms, such as batch gradient descent algorithm, stochastic gradient descent algorithm, adaptive gradient descent algorithm and momentum gradient descent algorithm.

[0168] The embodiment of the present application obtains the maximum amount of information cached in the secondary channel through the pre-trained machine learning model obtained through training, thereby improving the calculation efficiency and accuracy of the maximum amount of information.

[0169] In some embodiments, step 1033 may also be implemented by executing the following process: writing the updated maximum information quantity information into the target L2 channel cache.

[0170] Continue to see Figure 3A In step 104, for the information in the second information set, the information is sent to the device corresponding to the device identifier in the information.

[0171] In some embodiments, the device corresponding to the device identification in the information may be the terminal 400-1 or the terminal 400-2 described above. If the terminal 400-1 and the terminal 400-2 correspond to the same device identification, the same information is sent to the terminal 400-1 or the terminal 400-2. If the information A includes the device identification A, the information B includes the device identification B, the terminal 400-1 corresponds to the device identification A, and the terminal 400-2 corresponds to the device identification B, the information A is sent to the terminal 400-1, and the information B is sent to the terminal 400-2.

[0172] In some embodiments, the extracted device identifier is used to query device information, including information such as the network address, IP address or port of the device. Prepare the network connection and protocol settings required to communicate with the device to ensure effective data transmission with the device through the network. Encapsulate the information to be sent into a format suitable for the device to receive, including operations such as data packaging, serialization or encryption. Send the encapsulated information to the corresponding device through the established network connection and protocol settings. After the transmission is completed, confirm whether the information has been successfully received by the device. If the transmission fails, decide whether to retry according to the situation.

[0173] In some embodiments, in step 102 "Based on the service identifier of the information in the information flow, the information in the information flow is first aggregated to obtain multiple first information sets", the following processing can be performed: based on the information template identifier of the information, the recommended content in the information is read from the database. Step 104 "Sending the information to the device corresponding to the device identifier in the information" can be implemented by performing the following processing: Sending the recommended content in the information to the device corresponding to the device identifier in the information.

[0174] In some embodiments, the recommended content includes the title of the information, specific content text, jump links and picture links, etc. The recommended content of the information is packaged into a format that can be received by the device, and the recommended content in the packaged information is sent to the device corresponding to the device identifier in the information.

[0175] When performing the first-level aggregation, the embodiment of the present application synchronously executes the reading of recommended content in the information from the database based on the information template identifier of the information. Since the number of business identifiers is smaller than that of device identifiers, the system is prevented from being overloaded by querying the recommended content again during the second-level aggregation. The system resources can be effectively utilized to maximize resource utilization and save subsequent query time. If some information does not need to distinguish between devices when it is sent, that is, it is directly sent after the first-level aggregation through the business identifier, then the recommended content of the information is synchronously queried during the first-level aggregation process of the information, which also improves the speed and convenience of information recommendation.

[0176] In some embodiments, the information includes an object group identifier, each object group identifier is pre-allocated with a corresponding five-level channel cache, see Fig. 3I , Fig. 3I 1 is a ninth flow chart of the information flow processing method provided in the embodiment of the present application. Before step 103, execute Fig. 3I Steps 501 to 503 are described in detail below.

[0177] For example, see Figure 4AEach object group identifier is pre-allocated with a corresponding five-level channel cache. For example, for the second-level channel cache corresponding to device manufacturer 1, object group identifier 1 and object group identifier 2 correspond to different five-level channel caches; for the second-level channel cache corresponding to device manufacturer 2, object group identifier 1 and object group identifier 2 correspond to different five-level channel caches.

[0178] In step 501, for each piece of information in the second information set, based on the object group identifier in the information, a target level 5 channel cache corresponding to the information is determined in a plurality of level 5 channel caches.

[0179] In some embodiments, the object group identifier includes the characteristics, preferences, region, education level, object level and consumption ability of the object. Different object group identifiers correspond to different five-level channel caches. A mapping mechanism is established to map the object group identifier with the corresponding five-level channel cache, which can be achieved through a hash table, an index or a database table. Mapping is performed based on the target five-level channel cache corresponding to the object group identifier, and the target five-level channel cache corresponding to the information is queried in multiple five-level channel caches.

[0180] In step 502, the information is written into the target level 5 channel cache.

[0181] In some embodiments, a suitable cache strategy is selected according to application requirements and characteristics of the target five-level channel cache, such as Least Recently Used (LRU), Least Frequently Used (LFU), or First Input First Output (FIFO). If the information is a complex data type, such as an object or an array, it needs to be serialized into a data format acceptable to the target five-level channel cache, usually a byte stream or a string. Use the cache application programming interface (API) to write the information to the target five-level channel cache. You can set an expiration time (Time To Live, TTL) to control the timeliness of the information in the target five-level channel cache. Process the results of the write to the target five-level channel cache, such as checking whether the write is successful or handling possible exceptions.

[0182] In step 503, the set of information in each level 5 channel cache is taken as a third information set.

[0183] In some embodiments, the information in each third information set corresponds to the same object group identifier.

[0184] For example, there are three five-level channel caches in total, each of which contains four pieces of information. For example, the first five-level channel cache includes information M, information N, information O, and information P, the second five-level channel cache includes information Q, information R, information S, and information T, and the third five-level channel cache includes information U, information V, information W, and information X. Then, the set of information M, information N, information O, and information P is regarded as a third information set, and information M, information N, information O, and information P correspond to the same object group identifier; the set of information Q, information R, information S, and information T is regarded as a third information set, and information Q, information R, information S, and information T correspond to the same object group identifier; the set of information U, information V, information W, and information X is regarded as a third information set, and information U, information V, information W, and information X correspond to the same object group identifier.

[0185] In some embodiments, step 104 may be implemented by executing the following process: for each information in the third information set, sending the information to the device corresponding to the device identifier in the information and the device corresponding to the object group identifier.

[0186] In some embodiments, the extracted device identifier and object group identifier are used to query device information, including information such as the network address, IP address or port of the device. Prepare the network connection and protocol settings required to communicate with the device to ensure effective data transmission with the device through the network. Encapsulate the information to be sent into a format suitable for the device to receive, including operations such as data packaging, serialization or encryption. Send the encapsulated information to the corresponding device through the established network connection and protocol settings. After the transmission is completed, confirm whether the information has been successfully received by the device. If the transmission fails, decide whether to retry according to the situation.

[0187] The embodiment of the present application further aggregates information through object group identification, so that based on the segmentation of the information at the business level and the device level, it can be sent in a targeted manner to the device where the object group corresponding to the information is located, thereby refining the grouping structure of the information and improving the query efficiency and processing efficiency of the information.

[0188] In some embodiments, see Figure 3J , Figure 3J is a tenth flow chart of the information flow processing method provided in the embodiment of the present application. After step 104, execute Figure 3J Steps 105 to 107 are described in detail below.

[0189] In step 105, an operation code corresponding to the information is detected.

[0190] In some embodiments, the received information is parsed to extract an opcode, which is a code indicating the intent and processing of the information. According to a predefined protocol or format, the binary or text information is converted into an opcode identifier through a mapping table or a parser.

[0191] In step 106, in response to the operation code indicating that the information transmission fails, the information is re-sent to the device corresponding to the device identifier in the information.

[0192] In some embodiments, the operation code is 0, indicating that the information is sent successfully, and the operation code is 1, indicating that the information is sent unsuccessfully. When the information fails to be sent, the information is directly re-sent locally to the device corresponding to the device identifier in the information.

[0193] In step 107, in response to failure in information resending, the information is written into the message middleware, and the process proceeds to the step of reading a preset amount of information from the message middleware in a storage order from first to last to combine into an information flow to be pushed.

[0194] In some embodiments, if local retransmission of the information still fails, the information is rewritten back to the message middleware. The information in the message middleware is in the form of a queue and stored in sequence from the time of receipt to the last received. That is, the first arriving information is placed in front of the message middleware, and the later arriving information is placed behind the message middleware. A preset number of information is read from the message middleware in sequence from the time of receipt, and the preset number of information is combined into an information stream to be pushed.

[0195] The embodiment of the present application ensures that the information can successfully reach the device corresponding to the device identifier in the information through the information retransmission mechanism, improves the reliability of information transmission, and reduces the risk of data loss or delay. Writing information into the message middleware can enhance the fault tolerance of the system. The message middleware supports asynchronous information processing. Information can be transmitted asynchronously between the sender and the receiver, improving the response speed and throughput of the system. The sender can continue to perform other tasks before the information is actually processed without waiting for confirmation, thereby improving processing efficiency.

[0196] The following describes an exemplary application of the embodiments of the present application in an application scenario of network media.

[0197] The embodiments of the present application are not limited to information processing on network media, but are also applicable in other scenarios, such as e-commerce scenarios, educational APP scenarios, financial APP scenarios, or other scenarios where information recommendation can be performed.

[0198] When processing information, the related technology often processes it one by one, that is, each time the upstream server receives a piece of information to be pushed, it stores the information in the channel corresponding to the device identifier of the information. When the downstream server sends the information to the corresponding terminal, it classifies the information according to the business identifier corresponding to the information, and sends the classified information to terminals of different manufacturers, resulting in a large delay. In a scenario with high information concurrency, the one-by-one processing method is likely to cause information blocking, resulting in too low information processing efficiency.

[0199] The embodiment of the present application first performs the first-level aggregation of the information in the information flow through the service identifier, and then performs the second-level aggregation of the information in each first information set obtained by the first aggregation through the device identifier. Before sending the information to the corresponding terminal, the information classification processing is completed from the service and device level, so that the classified information can be directly sent to the corresponding terminal, reducing the information delay and complexity caused by classifying the information before sending, saving processing time, and improving the convenience of information processing. In the high-concurrency scenario of information, the method of batch processing of information through multi-level aggregation also avoids the situation where information processing is blocked due to slow processing of one information, thereby improving the processing efficiency of information.

[0200] See also Figure 5A , Figure 5A It is a schematic diagram of information of various product types provided in the embodiments of the present application. Figure 5A In the figure, the information of multiple product types may be information 501 of video software, information 502 of novel reading software, information 503 of browser, and information 504 of application store.

[0201] join Figure 5B , Figure 5B Schematic diagram of the push information provided by the embodiment of the present application. The push information style can be a combination of text and picture 505, a new message reminder but without displaying specific information content 506 and specific text content 507.

[0202] First, push messages from different applications (i.e., information in the information flow) are initially aggregated in real time, so that push messages from the same application can be aggregated into a batch for post-processing. Then, the system splits messages from different manufacturers (i.e., device manufacturers) into secondary aggregation channels (i.e., secondary channel caches), and performs secondary aggregation (i.e., second-level aggregation) on messages based on application and template IDs (i.e., information template identifiers), forming a multi-level aggregation structure of application-manufacturer-template. In this way, messages from different manufacturers and different applications can flow independently, avoiding classification processing during subsequent consumption and significantly improving overall processing efficiency.

[0203] The streaming multi-level aggregation distribution push system can fully utilize the batch push capabilities provided by manufacturers to speed up push speed and improve push efficiency. At the same time, it solves the problem of overall channel blockage caused by the slow processing speed of a certain manufacturer in the traditional push system, as well as the inefficiency caused by the need for additional classification processing during consumption. By optimizing the asynchronous processing capabilities of the push system, the business growth needs can be accelerated as a whole to empower the business.

[0204] The resources required for the streaming multi-level aggregation distribution push system include: cloud server, which uses Linux system, 8-core central processing unit (CPU), 8 gigabytes (GB) of memory and 100 containers; cloud storage remote dictionary service (Redis), where the cloud storage includes 8GB*128 shards (cluster version), each shard includes 1 primary copy and 2 secondary copies; hardware environment topology: containerized deployment.

[0205] Take the above-mentioned information flow processing system 100 as a recommendation system as an example, see Figure 6 , Figure 6 Schematic diagram of the principle of information push of the recommendation system provided in the embodiment of the present application. Figure 6 It includes the following parts:

[0206] Push middle station: It can be an upstream server as the upstream of the channel layer, where the channel gateway is the traffic entrance. When a message is sent, it needs to go through multiple links such as interface authentication, traffic control, preprocessing, multi-tenancy and routing distribution. Interface authentication refers to the verification of the upstream identity after receiving the business. Only after the verification is passed can it be pushed; traffic control refers to obtaining the push traffic threshold for any business, such as 30w / second. If the current traffic exceeds the push traffic threshold, the information is intercepted and processed; preprocessing refers to the unique ID tagging of the message and pre-processing of whether the content is complete and compliant; multi-tenancy refers to reading different business configurations, such as video APP, news APP, sports APP or animation APP, etc.; routing distribution refers to routing distribution based on the configuration information of multiple tenants, and sending messages to different clusters for processing.

[0207] Message queue: Perform traffic peak reduction for the messages sent. Based on Kafka, the message is first put into the message queue, and then the downstream "Push channel logic" service performs consumption processing.

[0208] Push channel logic: It is the core system of the channel, including the following parts: processing message content. One is that the content sent is a complete message structure, including the main title, subtitle, picture or jump address, etc. The second is to use the template ID to exchange the complete message content in real time for push; device mapping refers to querying the device service through the business ID to obtain the manufacturer's push token and the manufacturer's type.

[0209] Channel selection: Classify the devices according to their manufacturers, and send the devices with device ID A through the push channel of device ID A; send the devices with device ID B through the push channel of device ID B.

[0210] System A, System B and System C can be operating system A, operating system B and operating system C. The channel encapsulation of system A corresponds to the downstream server of system A; the channel encapsulation of system B corresponds to the downstream server of system B; the channel encapsulation of system C corresponds to the downstream server of system C; the long link channel encapsulation is a self-built channel, which is responsible for the push channel for the server and the mobile phone to maintain a long link.

[0211] Terminal of system A: On the one hand, it integrates the software development kits (SDKs) of multiple manufacturers using system A, is responsible for receiving messages sent by the downstream servers of system A, and finally displays the messages on the mobile terminal using system A. On the other hand, it receives messages through the self-developed long link channel and also displays them on the mobile terminal using system A.

[0212] Terminals of System B: On the one hand, it integrates the information push service (Apple Push Notificationservice, APNs) capability of System B, is responsible for receiving messages sent by the downstream server of System B, and displays the messages on the mobile terminals using System B. On the other hand, it receives messages through the self-developed long link channel and also displays them on the mobile terminals using System B.

[0213] Terminal of System C: On the one hand, it integrates the SDKs of multiple manufacturers using System C, is responsible for receiving messages sent by the downstream servers of System C, and finally displays the messages on the mobile terminal using System C. On the other hand, it receives messages through the self-developed long link channel and also displays them on the mobile terminal using System C.

[0214] Device services include device management services and device read-only services.

[0215] Device management service: responsible for collecting terminal SDKs, registering and reporting device information, such as push tokens, manufacturer types, notification switches, and version types.

[0216] Device read-only service: Different from the device management service, it is responsible for providing high-performance queries and adopts a read / write separation architecture.

[0217] Push data funnel: responsible for digitizing the black box information processing process.

[0218] Channel delivery: The business calls the channel and counts the number of message calls, including success and failure.

[0219] Channel arrival: whether the push message successfully reaches the mobile phone.

[0220] Click: The terminal reports the click behavior, that is, calculates the click rate of the information.

[0221] See also Figure 7 , Figure 7 It is a schematic diagram of the principle of the channel cache provided in the embodiment of the present application. When a push message (including the message content of the service identifier, device identifier and template identifier (i.e., information template identifier)) enters the system, the system will perform streaming real-time aggregation (i.e., first-level aggregation) on the message according to the service ID when writing the message into the first-level channel cache. Specifically, push messages with the same service ID, such as messages corresponding to service identifier A, will be aggregated together, messages corresponding to service identifier B will be aggregated together, and messages corresponding to service identifier C will be aggregated together; push messages with the same service ID can be aggregated into a batch during the writing phase and wait for subsequent processing.

[0222] In the configuration file, you can set the capacity and time thresholds. When a batch of accumulated messages reaches the capacity threshold (i.e., the quantity threshold) or the time threshold (i.e., the receiving duration threshold), the system will trigger batch processing. When the threshold is reached, the system will process the batch of messages. Batch processing includes vendor information reading and content processing, using parallel processing mode.

[0223] Extract a batch of device IDs from the current batch of messages. Send the extracted device IDs to the device read-only service (i.e., distributed cache) for query. Device services are divided into device management and device read-only (high-performance read), where detailed device information is stored in the database (DataBase, DB), and the performance is poor during concurrent reading; therefore, for query situations, a device read-only service is designed to store commonly used device information (such as the manufacturer corresponding to the device) in the distributed cache. The cache is in the form of key-value pairs, where the key is the device ID and the value is commonly used device information, including manufacturer information (i.e., device manufacturer). Here, the corresponding manufacturer information is actually parsed from the value. The device read-only service will return the manufacturer information corresponding to this batch of device IDs to the manufacturer information reading service.

[0224] Batch process the message contents with the same business ID. Specifically: First, the content refers to the title, specific content text, jump link and image link of a push message; there are two types of push: single push, that is, a single content corresponds to a single user, the upper-level service will directly pass in the content information in JSON format, parse the fields in JSON and store them in the content field corresponding to the current push; group push, that is, the same content will be sent to multiple users. In order to save bandwidth, a template ID will be used to correspond to a content. For example, the information corresponding to different manufacturers will obtain the content corresponding to the information according to the template ID. Figure 7 In the figure, the contents corresponding to Template 1, Template 2, Template 3, Template 4 and Template 5 are all different. This correspondence exists in Redis. The cache libraries stored in different businesses are isolated. The batch processing here is to perform content mapping query for a single business based on the different split businesses, and query the content information in Redis corresponding to the business based on the template ID. It can be understood as the process of querying the content of messages under the same business.

[0225] Classify messages according to manufacturer information: Split batch processed messages according to manufacturer information. Write classified messages into the secondary channel cache: Different from the method in related technologies where all manufacturers correspond to one channel, the system creates a corresponding channel cache, i.e., the secondary channel cache, for each manufacturer during initialization; write each manufacturer's message into its corresponding channel cache, for example, the information of manufacturer 1, manufacturer 2, manufacturer 3, and long links are stored in different secondary channel caches, respectively, to achieve asynchronous processing and improve service throughput. Through the batch push interface, the information in each secondary channel cache is sent to the downstream server of the corresponding manufacturer, for example, the information in the secondary channel cache corresponding to manufacturer 1 is sent to the downstream server of manufacturer 1, the information in the secondary channel cache corresponding to manufacturer 2 is sent to the downstream server of manufacturer 2, and the information in the secondary channel cache corresponding to the long link is sent to the downstream server of the long link, and the information is sent to the corresponding terminal through the downstream server.

[0226] When performing secondary aggregation, there are two main issues that can cause channel blockage, reduce processing efficiency, and affect overall performance:

[0227] 1. Uneven message arrival: see Figure 8 , Figure 8 This is a first principle schematic diagram of the secondary channel cache provided in an embodiment of the present application. Figure 8It shows the situation that messages arrive unevenly in different secondary channel caches during secondary aggregation. After the manufacturer's messages are split, the messages from different manufacturers may arrive unevenly at the entrance of the secondary channel cache. For example, the message volume in the secondary channel cache corresponding to manufacturer 1 is large, and the resources are insufficient, which easily leads to channel blocking; the message volume in the secondary channel cache corresponding to manufacturer 2 is small, the resources are idle, and some resources are wasted; the message volume in the secondary channel cache corresponding to the long link is medium. In other words, in the same time interval, the number of messages from some manufacturers may be significantly more than that of other manufacturers. For example, the number of messages from manufacturer 1 is higher than that of manufacturer 2 and long links, resulting in data skew. Data skew may lead to uneven resource allocation, so that the message processing capabilities of some manufacturers are not fully utilized, thereby reducing overall performance.

[0228] 2. There are differences in processing capabilities among manufacturers: see Fig. 9 , Fig. 9 This is a second principle schematic diagram of the secondary channel cache provided in an embodiment of the present application. Fig. 9 It shows that when performing secondary aggregation, different secondary channel caches have different processing capabilities for different vendors. At the message cache exit, different vendors have different batch interface processing capabilities. Some vendors may only be able to process a small number of messages at a time, and the processing speed is slow, while other vendors may have stronger processing capabilities and higher processing speeds. For example, the batch push interface processing capability of vendor 1 is strong, causing the channel buffer of the secondary channel cache to be idle for a long time; the batch push interface processing capability of vendor 2 is weak, which easily causes channel blocking; the batch push interface processing capability of vendor 3 is moderate. The difference in processing capability may cause channel blocking, that is, when a vendor's processing speed is slow, the entire channel may be blocked, resulting in reduced overall processing efficiency, and some messages cannot be processed in time, affecting user experience.

[0229] In order to solve the above problems, dynamic channel parameter adjustment can be performed to balance the differences between manufacturers, make resource allocation more reasonable, and ensure that the processing capacity of each manufacturer is fully utilized, thereby improving user experience; it can also make full use of the manufacturer's batch push capabilities to avoid channel blocking, thereby improving overall performance.

[0230] See also Fig.10 , Fig.10This is a schematic diagram of the principle of dynamically adjusting channel parameters provided by an embodiment of the present application. First, the manufacturer channel utilization (that is, the buffer utilization) is detected. The ratio adjustment service will periodically detect the current utilization of different manufacturer channels, that is, the data packet arrival and processing volume in the past period of time are counted; then, the channel utilization is analyzed, and according to the utilization of each manufacturer's channel, whether the channel parameters need to be adjusted; finally, the channel parameters are dynamically adjusted, and according to the analysis results, the channel parameters are dynamically configured. The channel parameters include concurrency, single batch aggregation capacity threshold (that is, the maximum number of batch processing), single batch aggregation time threshold (that is, the maximum duration of a batch aggregation) and the pre-allocated size of the channel cache (that is, the buffer size), etc., to improve the utilization of downstream batch capacity, wherein the parallelism represents the number of working threads in each secondary channel cache at the same time, the single batch aggregation capacity threshold represents the maximum number of information that can be processed in one batch of each secondary channel cache, the single batch aggregation time threshold represents the maximum duration of each batch of each secondary channel cache that can perform second-level aggregation, and the pre-allocated size of the channel cache represents the upper limit of the amount of data that can be stored in each secondary channel cache.

[0231] Get the total size M of the buffer (i.e., the capacity of the buffer) and the actual size N of the buffer (i.e., the usage of the buffer), and calculate the ratio N / M of the actual size N of the buffer to the total size M of the buffer as the utilization of the buffer. If the utilization of the buffer is greater than the first utilization threshold, for example, 80%, increase the concurrency or the single-batch aggregation capacity threshold, or reduce the single-batch aggregation time threshold. For example, increase the concurrency in the secondary channel cache corresponding to vendor 1 from 20 to 30, increase the single-batch aggregation capacity threshold from 200 / batch to 300 / batch, reduce the single-batch aggregation time threshold from 200 milliseconds / batch to 100 milliseconds / batch, and increase the pre-allocated size of the channel cache from 10,000 to 11,000; if the utilization of the buffer is less than the second utilization threshold, for example, 20%, reduce the concurrency or the single-batch aggregation capacity threshold, or increase the single-batch aggregation time threshold. Batch aggregation time threshold, for example, the concurrency in the secondary channel cache corresponding to vendor 2 is reduced from 20 to 10, the single batch aggregation capacity threshold is reduced from 200 / annotation to 100 / batch, the single batch aggregation time threshold remains unchanged, and the pre-allocated size of the channel cache is reduced from 10,000 to 5,000; the concurrency in the secondary channel cache corresponding to the long link is reduced from 20 to 5, the single batch aggregation capacity threshold is reduced from 200 / annotation to 50 / batch, the single batch aggregation time threshold is increased from 200 milliseconds / batch to 500 milliseconds / batch, and the pre-allocated size of the channel cache is reduced from 10,000 to 2,000. Among them, the buffer utilization can also be replaced by machine CPU utilization or machine memory utilization.

[0232] When the split messages from different manufacturers are placed in the corresponding secondary channel cache, the messages will be streamed and aggregated in real time according to the business ID and message content template ID. Specifically, push messages with the same business ID and template ID will be aggregated together and wait for subsequent processing. Through the batch push interface, the information in each secondary channel cache is sent to the downstream server of the corresponding manufacturer. For example, the information in the secondary channel cache corresponding to manufacturer 1 is sent to the downstream server of manufacturer 1, the information in the secondary channel cache corresponding to manufacturer 2 is sent to the downstream server of manufacturer 2, and the information in the secondary channel cache corresponding to the long link is sent to the downstream server of the long link, and the information is sent to the corresponding terminal through the downstream server.

[0233] In the configuration file, you can set the thresholds of capacity and time. When a batch of accumulated messages reaches the capacity threshold or the time threshold, the system will trigger batch processing. When the threshold is reached, the current batch of messages is extracted from the secondary channel cache. According to the manufacturer + business dimension, call the corresponding manufacturer's interface for batch push. Detect and collect the push results of each manufacturer, and resend if it fails.

[0234] Specific logic for failed resending: First, when sending, it will check whether the operation code returned by the manufacturer is 0 (that is, success). If it fails, it will retry sending it locally once; if it still fails, the failed push will be rewritten back to the Kafka queue (that is, the message middleware) before the channel layer of the push system, and then consumed by the channel service.

[0235] The embodiment of the present application firstly aggregates the information in the information flow through the service identifier, and then performs the second-level aggregation on the information in each first information set obtained by the first aggregation through the device identifier. Before sending the information to the corresponding terminal, the classification processing of the information is completed from the service and device level, so that the classified information can be directly sent to the corresponding terminal, reducing the information delay and complexity caused by classifying the information before sending, saving processing time, and improving the convenience of information processing. In the high concurrency scenario of information, the method of batch processing of information through multi-level aggregation also avoids the situation where the information processing is blocked due to the slow processing of one information, thereby improving the processing efficiency of information. The information is detected, and the information that passes the detection is stored in the message middleware in order to avoid the problem of traffic overload caused by the surge of messages, so that the messages are always executed in the message middleware in order, reducing the instantaneous traffic and saving resource occupation. By processing the unprocessed information received one by one, it is ensured that each information will be clustered at the first level, and the real-time performance of information processing is improved. Through the information template identifier in the information, the information in the second information set is further subdivided, so that the information of the same business, the same device and using the same information template is aggregated together, forming a multi-level aggregation structure of business-manufacturer-template, so as to facilitate batch transmission of specific types of information, thereby improving the transmission efficiency of information. By dynamically adjusting the channel parameters, the situation of uneven information and resource allocation caused by different processing capabilities of the secondary channel cache is avoided, the utilization rate of the channel cache is improved, and the resource occupation is reduced. The channel parameters are increased for the channel cache with higher processing capabilities, and the channel parameters are reduced for the channel cache with lower processing capabilities, so as to improve the rationality of resource allocation between different channel caches and improve the processing efficiency of information. Through the information retransmission mechanism, it is ensured that the information can successfully reach the device corresponding to the device identifier in the information, improve the reliability of information transmission, and reduce the risk of data loss or delay. Writing information into the message middleware can enhance the fault tolerance of the system. The message middleware supports asynchronous information processing. The information can be transmitted asynchronously between the sender and the receiver, which improves the response speed and throughput of the system. The sender can continue to perform other tasks before the information is actually processed without waiting for confirmation, thereby improving the processing efficiency.

[0236] The following continues to describe the exemplary structure of the information flow processing device 233 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software modules stored in the information flow processing device 233 of the memory 230 may include:

[0237] The information receiving module 2331 is used to receive information flow.

[0238] The first aggregation module 2332 is used to perform first-level aggregation on the information in the information flow based on the service identifier of the information in the information flow to obtain multiple first information sets, wherein different first information sets are associated with different service identifiers.

[0239] The second aggregation module 2333 is used to perform second-level aggregation on the information in each first information set based on the device identifier in the information to obtain multiple second information sets, wherein different second information sets are associated with different device identifiers.

[0240] The information sending module 2334 is used to send information in the second information set to the device corresponding to the device identifier in the information.

[0241] In some embodiments, the information receiving module 2331 is also used to detect the information in the information flow to be pushed and obtain the detection result; in response to the detection result being passed, the information is written into the message middleware; from the message middleware, a preset amount of information is read in the storage order from first to last to combine into the information flow to be pushed.

[0242] In some embodiments, multiple storage spaces corresponding to multiple business identifiers are pre-allocated in the first-level channel cache, and the first aggregation module 2332 is also used to query the target storage space corresponding to the information in the multiple storage spaces based on the business identifier of each information in the information flow; write each information into the target storage space corresponding to each information; and take the collection of information in each storage space as a first information collection.

[0243] In some embodiments, the first level aggregation is performed on the information flow in batches, and the first aggregation module 2332 is also used to treat the information of the quantity threshold as a batch when the number of information in the received information flow is equal to the quantity threshold; or, when the received duration of the information flow is equal to the reception duration threshold, treat the information received within the reception duration threshold as a batch.

[0244] In some embodiments, each device manufacturer corresponds to multiple device identifiers, and each device manufacturer is pre-allocated with a corresponding secondary channel cache. The second aggregation module 2333 is also used to perform the following processing for each information in the first information set: based on the device identifier in the information, query the target device manufacturer corresponding to the device identifier; based on the target device manufacturer, determine the target secondary channel cache corresponding to the information from the secondary channel cache pre-allocated by each device manufacturer; and write the information into the target secondary channel cache.

[0245] In some embodiments, the second aggregation module 2333 is also used to query the target device manufacturer corresponding to the device identifier from the distributed cache based on the device identifier in the information, wherein the distributed cache is stored in the form of a key-value pair, the key in the key-value pair is the device identifier, and the value in the key-value pair is the device manufacturer corresponding to the device identifier; in response to not finding the target device manufacturer corresponding to the device identifier in the distributed cache, querying the target device manufacturer corresponding to the device identifier from the database.

[0246] In some embodiments, the secondary channel cache includes multiple third-level channel caches, one third-level channel cache corresponds to one information template identifier, and the second aggregation module 2333 is also used to determine the target third-level channel cache corresponding to the information in multiple third-level channels based on the information template identifier in the information; and write the information into the target third-level channel cache.

[0247] In some embodiments, the device identification includes multiple device parameters, the third-level channel cache includes multiple fourth-level channel caches, one fourth-level channel cache corresponds to a combination of multiple device parameters, and the second aggregation module 2333 is also used to query the target fourth-level channel cache corresponding to the information in the multiple fourth-level channel caches based on the combination of multiple device parameters of the information; and write the information into the target fourth-level channel cache.

[0248] In some embodiments, each secondary channel cache includes a buffer and multiple channel parameters, the buffer is used to store information in the secondary channel cache, and the channel parameters characterize the processing performance of the secondary channel cache for information. The second aggregation module 2333 is also used to determine the utilization of the buffer in the target secondary channel cache based on the capacity and usage of the buffer in the target secondary channel cache; in response to the utilization of the buffer in the target secondary channel cache being greater than a first utilization threshold, increase multiple channel parameters; in response to the utilization of the buffer in the target secondary channel cache being less than a second utilization threshold, reduce multiple channel parameters; based on the updated multiple channel parameters, update the maximum amount of information in the target secondary channel cache.

[0249] In some embodiments, the second aggregation module 2333 is further used to write the updated maximum information quantity information into the target L2 channel cache.

[0250] In some embodiments, the first aggregation module 2332 is further used to read recommended content in the information from a database based on the information template identifier of the information.

[0251] In some embodiments, the information sending module 2334 is further used to send the recommended content in the information to the device corresponding to the device identifier in the information.

[0252] In some embodiments, the information includes an object group identifier, each object group identifier is pre-allocated with a corresponding five-level channel cache, and the second aggregation module 2333 is also used to determine, for the information in each second information set, a target five-level channel cache corresponding to the information in multiple five-level channel caches based on the object group identifier in the information; write the information into the target five-level channel cache; and use the collection of information in each five-level channel cache as a third information set.

[0253] In some embodiments, the information sending module 2334 is further used to send information in each third information set to the device corresponding to the device identifier in the information and the device corresponding to the object group identifier.

[0254] In some embodiments, the information sending module 2334 is also used to detect an operation code corresponding to the information; in response to the operation code indicating a failure in information sending, resend the information to the device corresponding to the device identifier in the information; in response to a failure in information resending, write the information into the message middleware, and proceed to the step of reading a preset number of information from the message middleware in a storage order from first to last to combine them into an information stream to be pushed.

[0255] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the electronic device reads the computer executable instruction or the computer program from the computer-readable storage medium, and the processor executes the computer executable instruction or the computer program, so that the electronic device executes the information flow processing method described in the embodiment of the present application.

[0256] The embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions or computer programs are stored. When the computer-executable instructions or computer programs are executed by a processor, the processor will execute the information flow processing method provided by the embodiment of the present application, for example, Figure 3A The information flow processing method is shown.

[0257] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0258] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0259] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0260] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0261] In summary, through the embodiment of the present application, the information in the information flow is first aggregated at the first level through the business identifier, and then the information in each first information set obtained by the first aggregation is aggregated at the second level through the device identifier. Before sending the information to the corresponding terminal, the classification processing of the information is completed from the business and device levels, so that the classified information can be directly sent to the corresponding terminal, reducing the information delay and complexity caused by classifying the information before sending, saving processing time, and improving the convenience of information processing. In the high concurrency scenario of information, the way of batch processing of information through multi-level aggregation also avoids the situation where the information processing is blocked due to the slow processing of one information, and improves the processing efficiency of information. The information is detected, and the information that passes the detection is stored in the message middleware in order to avoid the problem of traffic overload caused by the surge of messages, so that the messages are always executed in the message middleware in order, reducing the instantaneous traffic and saving resource occupation. By processing the unprocessed information received one by one, it is guaranteed that each information will be clustered at the first level, and the real-time performance of information processing is improved. Through the information template identifier in the information, the information in the second information set is further subdivided, so that the information of the same business, the same device and using the same information template is aggregated together, forming a multi-level aggregation structure of business-manufacturer-template, so as to facilitate batch transmission of specific types of information, thereby improving the transmission efficiency of information. By dynamically adjusting the channel parameters, the situation of uneven information and resource allocation caused by different processing capabilities of the secondary channel cache is avoided, the utilization rate of the channel cache is improved, and the resource occupation is reduced. The channel parameters are increased for the channel cache with higher processing capabilities, and the channel parameters are reduced for the channel cache with lower processing capabilities, so as to improve the rationality of resource allocation between different channel caches and improve the processing efficiency of information. Through the information retransmission mechanism, it is ensured that the information can successfully reach the device corresponding to the device identifier in the information, improve the reliability of information transmission, and reduce the risk of data loss or delay. Writing information into the message middleware can enhance the fault tolerance of the system. The message middleware supports asynchronous information processing. The information can be transmitted asynchronously between the sender and the receiver, which improves the response speed and throughput of the system. The sender can continue to perform other tasks before the information is actually processed without waiting for confirmation, thereby improving the processing efficiency.

[0262] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method for processing information flow, characterized in that: The method comprises: Receive information streams; Based on the service identifier of the information in the information flow, performing first-level aggregation on the information in the information flow to obtain multiple first information sets, wherein different first information sets are associated with different service identifiers; For each information in the first information set, based on the target device manufacturer corresponding to the device identifier in the information, the secondary channel cache pre-allocated by the target device manufacturer is used as the target secondary channel cache; in response to the utilization rate of the buffer in the target secondary channel cache being greater than the first utilization rate threshold, multiple channel parameters are increased; in response to the utilization rate of the buffer in the target secondary channel cache being less than the second utilization rate threshold, the multiple channel parameters are decreased; based on the updated multiple channel parameters, the maximum number of information in the target secondary channel cache is updated; Based on the updated maximum information quantity, writing the information into the target secondary channel cache to obtain multiple second information sets, wherein different second information sets are associated with different device identifiers; For the information in the second information set, the information is sent to a device corresponding to the device identifier in the information.

2. The method according to claim 1, characterized in that A plurality of storage spaces corresponding to the plurality of service identifiers are pre-allocated in the first-level channel cache; The first-level aggregation of the information in the information flow based on the service identifier of the information in the information flow to obtain multiple first information sets includes: Based on the service identifier of each of the information in the information flow, querying the target storage space corresponding to the information in the multiple storage spaces; Writing each of the information into the target storage space corresponding to each of the information; The set of information in each storage space is taken as a first information set.

3. The method according to claim 2, characterized in that The first level aggregation is performed on the information flow in batches; Before performing the first-level aggregation on the information in the information flow based on the service identifier of the information in the information flow, the method further includes: When the number of the received information in the information flow is equal to a number threshold, the information of the number threshold is regarded as a batch; or, When the received duration of the information flow is equal to the receiving duration threshold, the information received within the receiving duration threshold is regarded as a batch.

4. The method according to claim 1, characterized in that: Each device manufacturer corresponds to a plurality of the device identifiers, and each of the device manufacturers is pre-allocated with a corresponding secondary channel buffer.

5. The method according to claim 1, characterized in that The secondary channel cache includes a plurality of third-level channel caches, and one third-level channel cache corresponds to one information template identifier; The step of writing the information into the target secondary channel cache comprises: Based on the information template identifier in the information, determining a target third-level channel cache corresponding to the information in the plurality of third-level channels; The information is written into the target level 3 channel cache.

6. The method according to claim 5, characterized in that The device identification includes a plurality of device parameters, the third-level channel cache includes a plurality of fourth-level channel caches, and one of the fourth-level channel caches corresponds to a combination of the plurality of device parameters; The step of writing the information into the target level 3 channel cache comprises: Based on a combination of the multiple device parameters of the information, querying a target L4 channel cache corresponding to the information in the multiple L4 channel caches; The information is written into the target L4 channel cache.

7. The method according to claim 1, characterized in that Each of the secondary channel caches includes the buffer and the plurality of channel parameters, the buffer is used to store the information in the secondary channel cache, and the channel parameters represent the processing performance of the secondary channel cache for the information; Before increasing the plurality of channel parameters in response to the utilization of the buffer in the target L2 channel cache being greater than the first utilization threshold, the method further includes: Based on the capacity and usage of the buffer in the target L2 channel cache, a utilization rate of the buffer in the target L2 channel cache is determined.

8. The method according to claim 1, characterized in that When performing first-level aggregation on the information in the information flow based on the service identifier of the information in the information flow to obtain multiple first information sets, the method further includes: Reading recommended content in the information from a database based on the information template identifier of the information; The sending the information to the device corresponding to the device identifier in the information includes: The recommended content in the information is sent to a device corresponding to the device identifier in the information.

9. The method according to claim 1, characterized in that: Before using the secondary channel cache pre-allocated by the target device manufacturer corresponding to the device identifier in the information as the target secondary channel cache, the method further includes: Based on the device identifier in the information, query the target device manufacturer corresponding to the device identifier from the distributed cache, wherein the distributed cache is stored in the form of a key-value pair, the key in the key-value pair is the device identifier, and the value in the key-value pair is the device manufacturer corresponding to the device identifier; In response to not finding the target device manufacturer corresponding to the device identification in the distributed cache, querying the target device manufacturer corresponding to the device identification from a database.

10. The method according to any one of claims 1 to 9, characterized in that: The information includes an object group identifier, each of which is pre-allocated with a corresponding five-level channel cache; Before sending the information in the second information set to the device corresponding to the device identifier in the information, the method further includes: For each piece of information in the second information set, based on the object group identifier in the information, determining a target level-5 channel cache corresponding to the information from among the plurality of level-5 channel caches; Writing the information into the target level 5 channel cache; taking the set of information in each of the five-level channel caches as a third information set; The sending the information in the second information set to a device corresponding to the device identifier in the information includes: For each of the information in the third information set, the information is sent to a device corresponding to the device identifier in the information and a device corresponding to the object group identifier.

11. The method according to any one of claims 1 to 9, characterized in that: Before receiving the information flow, the method includes: Detecting the information in the information flow to be pushed to obtain a detection result; In response to the detection result being passed, writing the information into the message middleware; A preset number of the information is read from the message middleware in a storage order from first to last, so as to be combined into the information flow to be pushed.

12. The method according to claim 11, characterized in that After sending the information to the device corresponding to the device identifier in the information, the method further includes: Detecting an operation code corresponding to the information; In response to the operation code indicating that the information transmission fails, resending the information to the device corresponding to the device identifier in the information; In response to the failure of the information resending, the information is written into the message middleware, and the step of reading a preset number of the information from the message middleware in a storage order from first to last is entered to combine them into the information flow to be pushed.

13. An information flow processing device, characterized in that: The device comprises: An information receiving module, used for receiving information flow; A first aggregation module, configured to perform first-level aggregation on the information in the information flow based on the service identifier of the information in the information flow to obtain multiple first information sets, wherein different first information sets are associated with different service identifiers; A second aggregation module is used for, for each of the information in the first information set, based on the target device manufacturer corresponding to the device identifier in the information, using the secondary channel cache pre-allocated by the target device manufacturer as the target secondary channel cache; in response to the utilization rate of the buffer in the target secondary channel cache being greater than the first utilization rate threshold, increasing multiple channel parameters; in response to the utilization rate of the buffer in the target secondary channel cache being less than the second utilization rate threshold, reducing the multiple channel parameters; based on the updated multiple channel parameters, updating the maximum number of information in the target secondary channel cache; based on the updated maximum number of information, writing the information into the target secondary channel cache to obtain multiple second information sets, wherein different second information sets are associated with different device identifiers; An information sending module is used to send the information in the second information set to a device corresponding to the device identifier in the information.

14. An electronic device, characterized in that: The electronic device comprises: A memory for storing computer executable instructions or computer programs; A processor, configured to implement the information flow processing method according to any one of claims 1 to 12 when executing the computer executable instructions or computer programs stored in the memory.

15. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the information flow processing method according to any one of claims 1 to 12 is implemented.

16. A computer program product comprising computer executable instructions or a computer program, characterized in that When the computer executable instructions or computer programs are executed by a processor, the information flow processing method according to any one of claims 1 to 12 is implemented.

Images