Feed stream mixed diffusion method and device based on railway CMS derivative application scene

Abstract

The invention relates to a feed stream mixed diffusion method and device based on a railway CMS derivative application scenario, and the method comprises the steps: obtaining a multi-level authorization information storage structure which comprises a user table, a unit table, a feed table and a column table; generating a diffusion mode judgment rule based on the column table and the user activeness model; performing composite authorization calculation on the Feed stream source according to the multi-level authorization information storage structure and the diffusion mode judgment rule, and performing aggregation calculation on the result of the first authorization calculation to generate a Feed stream weighting table of the authorization source user; and based on request information of the Feed stream, after dynamic and static request separation of the dynamic and static separation gateway and read-write diffusion separation of the Feed stream calling module obtained by the user in sequence, entering a write diffusion acquisition path to obtain Json data of the Feed stream. According to the invention, by distinguishing the scene of read-write diffusion in the mixed diffusion mode, the authorization mode and the authentication calculation amount of read diffusion are simplified, and the requirement of actual use is conveniently met.

Description

Technical Field

[0001] The present invention relates to the field of railway CMS derivative application scenarios, and in particular to a feed stream mixing and diffusion method and device based on railway CMS derivative application scenarios. Background Art

[0002] In common railway industry CMS (content management systems) applications (such as websites, training platforms, and knowledge bases), published content is often provided through authorized access. However, with millions of categorized authorized users and a massive amount of content, using traditional relational databases for categorized, hierarchical, and authorized retrieval no longer meets practical requirements. Therefore, improvements are needed. Summary of the Invention

[0003] Based on this, it is necessary to provide a feed stream hybrid diffusion method and device based on railway CMS derivative application scenarios to address the problem that the use of traditional relational databases for classified and graded authorized retrieval can no longer meet the requirements of actual use.

[0004] The present invention provides a feed stream mixed diffusion method based on railway CMS derivative application scenarios, the method comprising: Acquire a multi-level authorization information storage structure, wherein the multi-level authorization information storage structure includes a user table, an organization table, a feed table, and a column table; Generate diffusion pattern determination rules based on the column list and user activity model; Based on the multi-level authorization information storage structure and diffusion mode judgment rules, a composite authorization calculation is performed on the feed flow source, and the results of the first authorization calculation are aggregated to generate a feed flow weighted table for the authorized source user; Based on the request information of the feed stream, it passes through the dynamic and static request separation of the dynamic and static separation gateway, the read-write diffusion separation of the user authorized feed stream call module, and then enters the write diffusion acquisition path to obtain the Json data of the feed stream.

[0005] In one embodiment, the generating of diffusion mode determination rules based on the column list and the user activity model includes: A composite calculation is performed on the resource sensitivity and user activity to generate a priority determination function. The priority determination function is as follows: , in, For user activity, is resource sensitivity, is the user activity weight, Resource sensitivity weight, is the total weight; In response to a feed stream source passing the priority determination, write diffusion processing is performed on the feed stream source, and read diffusion processing is performed on other feed stream sources. The specific expression is as follows: .

[0006] In one embodiment, the generating of diffusion mode determination rules based on the column list and the user activity model includes: Based on the column differentiation method and user activity, a priority determination function is generated. The priority determination function is as follows: , in, User activity; In response to a feed stream source passing the priority determination, write diffusion processing is performed on the feed stream source, and read diffusion processing is performed on other feed stream sources. The specific expression is as follows: .

[0007] In one embodiment, performing aggregation calculation on the result of the first authorization calculation to generate a feed flow weighted table of the authorized source user includes: Based on the authorized feed prediction categories, the corresponding rounds of full feed calculation lists are started according to the release time. The feed prediction categories include popular selections and important reminders. Determine the feeds and users involved in this round of feed flow changes; The feed flow table is persisted to the FastDFS distributed file system through write diffusion, and a feed flow weighted table for authorized source users is generated.

[0008] In one embodiment, determining the feeds and users involved in the current round of feed stream changes includes: According to the scheduled publishing time, obtain the ID list of all feeds involved in this round; Calculate user access frequency, session duration, and content interaction depth in real time to obtain the feeds and users involved in this round of feed flow changes.

[0009] In one embodiment, persisting the feed flow table to the FastDFS distributed file system by write diffusion and generating a feed flow weighted table of the authorized source user includes: Get all involved feeds and user information; Based on the aggregation calculation method, the feed stream file list of each user is read in turn, and the Json sequence in the list file is appended and saved to generate the feed stream weighted table of the authorized source user.

[0010] In one embodiment, the aggregation calculation of the result of the first authorization calculation to generate a feed flow weighted table of the authorized source user further includes: Call Purge to update the cache.

[0011] The present invention also provides a feed flow mixing and diffusion device based on a railway CMS derivative application scenario, comprising: An acquisition module, configured to acquire a multi-level authorization information storage structure, wherein the multi-level authorization information storage structure includes a user table, an organization table, a feed table, and a column table; A generation module is used to generate diffusion pattern determination rules based on the column list and user activity model; The calculation module is used to perform composite authorization calculation on the feed flow source based on the multi-level authorization information storage structure and diffusion mode determination rules, and aggregate the results of the first authorization calculation to generate a feed flow weighted table for the authorized source user; The separation module is used to process the request information based on the feed stream, which passes through the dynamic and static request separation of the dynamic and static separation gateway and the read-write diffusion separation of the user authorization feed stream call module, and then enters the write diffusion acquisition path to obtain the Json data of the feed stream.

[0012] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the feed stream mixing and diffusion method based on any of the above-mentioned railway CMS derived application scenarios.

[0013] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-described feed stream mixing and diffusion methods based on railway CMS-derived application scenarios.

[0014] The above-mentioned feed stream hybrid diffusion method and device based on railway CMS derivative application scenarios distinguishes between read and write diffusion scenarios in the hybrid diffusion mode, simplifies the authorization method and authentication calculation amount of read diffusion, and conveniently meets the requirements of actual use. At the same time, the application of the write diffusion mode in complex authorization scenarios eliminates the IO bottleneck of the read diffusion mode. In write diffusion, each user only reads his or her own data, and this data can also be deployed in a distributed manner, thereby solving the resource competition problem under high concurrency. At the same time, by enabling write diffusion to pre-place the corresponding calculation to the write operation link in the feed business, the complex authentication, weighting, and group flow calculation processes are pre-placed to directly form a result set. The acquirer directly pulls the source feed stream list in the result set, which simplifies the business process and is faster. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 The figure is a flow chart of a feed stream hybrid diffusion method based on a railway CMS derivative application scenario in one embodiment; Figure 2 A schematic diagram of a first flow chart for generating a diffusion mode determination rule in one embodiment; Figure 3 A second flow chart of generating diffusion mode determination rules in one embodiment; Figure 4 Schematic diagram of the process of performing aggregate calculation on the result of the first authorization calculation in one embodiment; Figure 5 A schematic diagram of the feed and user process involved in determining the feed flow changes in this round in one embodiment; Figure 6 This is a schematic diagram of the process of generating a feed flow weighting table for authorized source users in one embodiment; Figure 7 Schematic diagram of the structure of a feed flow mixing and diffusion device based on a railway CMS derivative application scenario in one embodiment; Figure 8 FIG. 1 is a diagram showing the internal structure of an electronic device according to an embodiment. DETAILED DESCRIPTION

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0018] The railway industry's management systems at all levels are characterized by clear hierarchies and strict authority management. However, the development of related content publishing systems also presents the problem of user-published content being unable to efficiently and accurately target its target audience within the multi-tiered management system's corporate websites. In this scenario, authorized access to relevant information is necessary. However, with millions of users and a vast amount of classified content, relying on traditional relational databases for classified and hierarchical authorized retrieval is no longer sufficient.

[0019] A feed is a web feed (or content summary), a data format that provides users with frequently updated content. Content publishers collaborate to create a web feed, allowing users to subscribe to a channel and access content from multiple publishers. In other words, a feed aggregates multiple subscribed news sources into a content aggregator, helping users continuously access the latest content from their subscriptions. Common internet feed applications include popular content platforms such as NetEase News, Weibo, Douyin, and Bilibili.

[0020] Early internet niche content service providers typically offered subscription feeds. A typical application scenario involves an RSS feed. Users subscribed to the feed, which then updated and pushed information about the user's subscriptions in chronological order, displaying it to the user. As application scenarios and needs expanded, corresponding feed sorting models evolved. Currently, they fall into three main categories:

[0021] 1. Simple Timeline Mode: Displays a list of feed content in the order of update time. Common application scenarios include WeChat Moments.

[0022] 2. Permission Weight Calculation Mode: This mode calculates the permissions and weight of content based on the corresponding user identity and weight factor, thereby determining which content to display and the order in which it is produced. Common application scenarios include Bilibili's subscription channel.

[0023] 3. Intelligent Recommendation Model: In the later stages of the development of the permission-weighted calculation model, the commercial value demands of internet applications, such as user stickiness and advertising effectiveness, emerged. This is when intelligent recommendation algorithms based on user profiles emerged. Common application scenarios include TikTok and Toutiao.

[0024] The most significant feature of the feed stream service is that the web content summaries obtained by information acquirers (audiences) are provided and authorized by publishers. Technically, there are two main types of feed stream collections: read diffusion (pull) and write diffusion (push).

[0025] In a complex multi-source feed system, multiple feeds exist, each authorizing a user group. Authorization not only authorizes the access relationship between the user entity and the feed entity, but also authorizes user-related attributes (such as gender and age), associated entities (such as the user's organization and profession), and related attributes of these associated entities (such as the user's organizational level), as well as feed-related attributes (such as feed publication time and content length), and associated entities (such as the feed's category, feed source website or system), and related attributes of these associated entities (such as the feed's category). This authorization information is stored as a cascade relationship between multiple entity relationships.

[0026] At this time, the authorization information of a single user is output as a feed stream, usually in the following two ways.

[0027] 1. Read Diffusion Working principle: During authorization, only the corresponding rules are stored. When a user access request is triggered, real-time permission aggregation calculation is performed. The total time consumption formula for permission aggregation calculation is as follows: , Where n is the number of associated authority sources, The query time of a single rule is is the aggregate calculation time, k is the total time consumed by all rule queries The proportion of .

[0028] The calculation formula for the storage space occupancy rate of its authorization information is as follows: , Where M and N are the number of authorized and authorized entities, respectively, and O represents the space complexity. In a complex multi-source feed flow system, authorized entities refer to the relevant attributes of user entities, associated entities, and their related attributes. Authorized entities refer to the relevant attributes of feeds, associated entities, and their related attributes.

[0029] Typical features: In read mode, access rights are recalculated according to the authorization rules each time a user retrieves their authorization feed. The feed results returned by the different rules are aggregated and returned to the user. This results in a high compression ratio for the storage of authorization information, but it also increases the consumption of computing resources and time during the user authentication process, especially in high-concurrency scenarios with complex authorization systems.

[0030] 2. Write Diffusion Working Principle: During authorization, permission aggregation calculation is performed, and copies of corresponding user authorization information are stored for different users. The total calculation time formula is consistent with read diffusion.

[0031] The calculation formula for the storage space occupancy rate of its authorization information is as follows: ,

[0032] Where M and N are the number of authorized objects and authorized objects respectively. The storage capacity increases with the product of the number of authorized objects and authorized objects.

[0033] Typical features: In write mode, a pre-aggregated permission calculation is performed on each user's feed stream immediately upon the feed's appearance. This generates and stores the resulting authorization feed stream data for each user. Therefore, when the user accesses the feed stream, they directly access the stored feed stream. This results in highly immediate permission acquisition (requiring only a single key-value query), but at the expense of high storage costs.

[0034] Database retrieval in read-scale mode is primarily based on binary tree operations in index mode, and the time complexity of a binary tree depends on its structure. In a balanced tree, the time complexity of searches, insertions, and deletions is O(log n). If the tree is unbalanced, the time complexity can degenerate to O(n). Therefore, the optimal time complexity for database retrieval is O(log n). That is, the computational complexity of read-scale mode is O(log 2 N). In write-scale mode, the computational time of O(log n) is already completed, so the computational complexity of retrieval is O(l). Data measured in the model development and verification environment show that, with 2 million users, 50 sources, and a total of 50,000 randomly authorized feeds, the read-scale feed stream acquisition time is approximately 80 seconds, while the write-scale feed stream acquisition time is approximately 60 milliseconds.

[0035] In order to comprehensively balance the contradiction between storage occupancy and user authorization efficiency, the following is combined with Figures 1-8The present invention describes a method and device for mixing and diffusing feed streams based on railway CMS derivative application scenarios.

[0036] like Figure 1 As shown, in one embodiment, a feed stream hybrid diffusion method based on a railway CMS derivative application scenario includes the following steps: Step S100: Acquire a multi-level authorization information storage structure, which includes a user table, a unit table, a feed table, and a column table.

[0037] The user table is the authorization object table, the unit table is the authorization object related entity table, the feed table is the authorized object table, and the column table is the authorized object related entity table.

[0038] Step S200: Generate diffusion mode determination rules based on the column list and the user activity model.

[0039] Whether to perform read or write propagation for the user authorization list of any feed stream source requires a feed stream priority determination for that source. Write propagation is performed only for sources that pass the priority determination, while read propagation is still used for other sources.

[0040] Step S300: Perform compound authorization calculation on the feed source according to the multi-level authorization information storage structure and diffusion mode determination rules, and perform aggregation calculation on the result of the first authorization calculation to generate a feed weighting table for the authorization source user.

[0041] 1. Multi-source permission circuit breaker mechanism A DAG (directed acyclic graph)-based permission calculation topology is used to perform parallel calculations on permission verification in three dimensions: user attributes (gender / major), organizational hierarchy (three levels: national railway group-railway bureau-station section), and content characteristics (column classification / security level).

[0042] A circuit breaker threshold is set for each computing node (for example, a circuit breaker is triggered when the response time of a single node is greater than 200ms) to ensure system-level service availability.

[0043] 2. Dynamic Weight Allocation Model Construct the authority factor weight matrix: W=[w_org, w_attr, w_cont]^T, where: w_org=0.6 (organizational structure weight), w_attr=0.3 (user attribute weight), w_cont=0.1 (content feature weight) Calculate user-content matching in real time: Score = Σ(w_i * match_level_i).

[0044] 3. Multi-level cache acceleration strategy Establish a three-level cache architecture: L1 cache: Redis hot data, L2 cache: local memory queue, L3 cache: SSD persistent cache.

[0045] The LRU-K algorithm is used to optimize the cache elimination strategy, increasing the cache hit rate to 92%.

[0046] 4. Core Value of Feed Stream Weighted Table By using a pre-generated weighted table, the time complexity of real-time authentication is reduced from O(n log n) to O(1).

[0047] Four weight update strategies are supported: scheduled full refresh, incremental real-time update (delay < 500ms), emergency strategy override (highest priority), and grayscale batch release.

[0048] By establishing a complete technical system of pre-calculation, dynamic adjustment, and hierarchical storage, the authentication efficiency bottleneck faced by traditional railway CMS systems in scenarios with millions of users has been effectively resolved, providing a new architectural paradigm for industry-level content distribution systems.

[0049] Step S400: Based on the request information of the feed stream, after passing through the dynamic and static request separation of the dynamic and static separation gateway and the read-write diffusion separation of the user authorized feed stream call module, it enters the write diffusion acquisition path to obtain the Json data of the feed stream.

[0050] By classifying dynamic requests and static requests, it is convenient to reduce the back-end computing pressure, and by separating read and write diffusion, it is convenient to improve decision-making efficiency and accuracy.

[0051] This feed stream hybrid diffusion method, based on railway CMS-derived application scenarios, simplifies the authorization method and authentication calculation amount of read diffusion by distinguishing between read and write diffusion scenarios in the hybrid diffusion mode. At the same time, the application of the write diffusion mode in complex authorization scenarios eliminates the IO bottleneck of the read diffusion mode. In write diffusion, each user only reads their own data, and this data can also be deployed in a distributed manner, thereby solving the problem of resource competition under high concurrency. At the same time, by enabling write diffusion to pre-place the corresponding calculations in the write operation link in the feed business, the complex authentication, weighting, and group flow calculation processes are pre-placed to directly form a result set. The acquirer can directly pull the source feed stream list from the result set, simplifying the business process and increasing the speed.

[0052] In this embodiment, see Figure 2 ,Based on the column list and user activity model, a diffusion pattern determination rule is generated,including the following steps: Step S211: perform a composite calculation on the resource sensitivity and the user activity to generate a priority determination function. The priority determination function is as follows: , in, For user activity, is resource sensitivity, is the user activity weight, Resource sensitivity weight, is the total weight.

[0053] The priority is determined by compound calculation of resource sensitivity and user activity, making it easy to determine whether the user authorization list of any feed source is read or write propagated.

[0054] In step S212, in response to the feed stream source passing the priority determination, write diffusion processing is performed on the feed stream source, and read diffusion processing is performed on other feed stream sources. The specific expression is as follows: .

[0055] In this embodiment, see Figure 3 ,Based on the column list and user activity model, a diffusion pattern determination rule is generated,including the following steps: Step S221: Generate a priority determination function based on the column classification method and user activity. The priority determination function is as follows: , in, For user activity.

[0056] Because it's difficult to standardize resource sensitivity calculations, and because changing a single feed to a highly sensitive resource triggers a computational storm involving re-authentication and write diffusion for all users, creating a serious risk of resource congestion, the hybrid diffusion model used in homogeneous multi-source feeds typically only matches user activity assessments.

[0057] In step S222, in response to the feed stream source passing the priority determination, write diffusion processing is performed on the feed stream source, and read diffusion processing is performed on other feed stream sources. The specific expression is as follows: .

[0058] After expert brainstorming and analysis using the ANP method, the decision was made to use a column-based approach. Business experts determined which columns were eligible for write-proliferation authorization. Columns using read-proliferation authorization were only authorized at the column level, further reducing database pressure.

[0059] At this point, the priority determination algorithm is further simplified to a key-value query.

[0060] In this embodiment, see Figure 4, aggregate the results of the first authorization calculation to generate a feed flow weighted table for the authorized source user, including the following steps: Step S310 : Based on the authorized feed stream prediction categories, the corresponding rounds of full feed stream calculation lists are started according to the release time. The feed stream prediction categories include popular selections and important reminders.

[0061] If the release time is set to automatic, it can only be released on the hour. Special tasks need to be released manually.

[0062] Step S320: Determine the feeds and users involved in the feed flow change in this round.

[0063] By precisely defining change boundaries, establishing an efficient retrieval system, and implementing intelligent predictive scheduling, the system can maintain a permission calculation accuracy of 99.998%, an automatic recovery rate for exceptions of 92%, and resource utilization fluctuations within ±3% while processing an average of 2,000+ content updates and serving 2 million+ users per day.

[0064] This refined change management has become the key technical cornerstone supporting the implementation of the hybrid diffusion model in ultra-large-scale scenarios.

[0065] Step S330: persist the feed flow table to the FastDFS distributed file system through write diffusion, and generate a feed flow weighted table for the authorized source user.

[0066] By pre-calculating and persisting user-level feed streams, real-time authentication overhead is eliminated, ensuring real-time response and shortening read latency. FastDFS separates the computing and storage layers, minimizing the impact of single-node failures. By performing a composite calculation of time weight and business weight during the file storage phase, dynamic weight integration is facilitated, shortening weight calculation time.

[0067] In this embodiment, see Figure 5 , determine the feeds and users involved in this round of feed flow changes, including the following steps: Step S321: Obtain a list of all feed IDs involved in this round according to the scheduled publishing time.

[0068] Taking "Popular Selections" as an example, because the database calculation time is relatively long, the system's aggregation calculation module will asynchronously call the corresponding stored procedure of the database, and input the feed ID list of all feeds involved in this round according to the scheduled release time.

[0069] Step S322 : Calculate user access frequency, session duration, and content interaction depth in real time to obtain the feeds and users involved in this round of feed stream changes.

[0070] The stored procedure first queries the feed role for user, confirms the list of authorized user IDs, and saves it in the session temporary table temp user table session.

[0071] Then, based on the corresponding feed ID and corresponding conditions in the feed role for user attribute and feed role for unit, the corresponding user is filtered out to obtain the corresponding user ID and user name, and the corresponding record is inserted into the temporary table again.

[0072] The resulting temporary table, temp user table session, contains all authorized users and the feeds involved.

[0073] In this embodiment, see Figure 6 , persist the feed flow table to the FastDFS distributed file system through write diffusion, and generate the feed flow weighted table of the authorized source user, including the following steps: Step S331: Obtain all related feeds and user information.

[0074] Step S332: Based on the aggregation calculation method, read the feed stream file list of each user in turn, and append the Json sequence in the list file and save it to generate the feed stream weighted table of the authorized source user.

[0075] At this point, the user first retrieves the file from the proxy cache. If a match is not found, the file is retrieved from FastDFS. Finally, the Json data in the feed is preloaded for the first 100 records, which are then processed and displayed by the front-end before continuing with traditional subsequent data.

[0076] In this embodiment, the results of the first authorization calculation are aggregated to generate a feed flow weighted table of the authorized source user, and the following steps are also included: Step S340: Call Purge to update the cache.

[0077] Specific application examples are as follows: Take the Railway Mobile Information Collaboration Platform (a three-level cluster of independently authorized content publishing sites) as an example. Content access is divided into public sources (a certain section of the website is completely public) and authorized sources (content within a section of the website requires individual authorization, and only authorized users can access it). Based on the circumstances of their organization, each employee (user) will receive authorized content push from at least one first-level site (the China Railway Group-level site) and up to three third-level sites (the China Railway Group-level, the affiliated railway bureau-level, and the affiliated station-depot-level). Four websites—the China Railway Group website, the Beijing Railway Bureau website, the Beijing Railway Station website, and the Beijing Locomotive Depot website—are used as examples. The data implementation and experimental data in the experimental environment are as follows.

[0078] The relevant attributes and table data of the authorization object (authorized user) table are as follows:

[0079] Table 1: Authorization object table (user table).

[0080] The relevant attributes and table data of one of the entities (unit) table associated with the authorization object are as follows:

[0081] Table 2: Example of an entity table associated with an authorization object (unit table).

[0082] The relevant attributes and table data of the authorized object (authorized feed) table are as follows:

[0083] Table 3: Authorized object table (Feed table).

[0084] The relevant attributes and table data of one of the entities (column) table associated with the authorized object are as follows:

[0085] Table 4: Example of entity table associated with authorized objects (column table).

[0086] Based on the above data, compound authorization and calculation are performed on specific news content (Feed).

[0087] For example, if the authorization for feed GT_001 is to filter by gender "male," and since the website for feed GT_001 is the "China Railway Group Website," a filter search of the relational database is required for all railway users. The final authorized users are "Zhang San, Wang Wu, and Zhou Qi."

[0088] Authorization for feed GT_002 is to filter for the unit "Beijing Railway Bureau" (including its subordinate units) and the unit professional category "Passenger Transport". Since the website of feed GT_002 is the "National Railway Group Website", a relational database filtering search is required for the entire railway audience. The final authorized users are "Wang Wu and Zhao Liu".

[0089] Table 5: Feed content authorization calculation table.

[0090] Then perform an aggregation calculation on the results of the first authorization calculation.

[0091] By sorting the feeds received by all authorized users by news time (here, the time order is assumed to be consistent with the news name order), a feed flow table for each user can be obtained. When a new feed appears in the system, it can be added to the last row of the feed flow table of the corresponding authorized user. At this time, it is necessary to further adjust the feed flow table of user "Wang Wu" based on the weight. Here, the feed flow table is rearranged based on whether it is top-ranked, and the final weighted table is shown in Table 6. The weighting method and approach are not limited to this model. It can also be calculated and sorted based on the number of user visits and other conditions. This calculation can occur on the back end or be distributed to the front end.

[0092]

[0093] Table 6: Weighted table of user "Wang Wu" feed stream.

[0094] At this point, the feed flow calculation for the authorized source user is complete. Further write-spread storage is required. The feed flow table can be saved directly to the database or local file as JSON text and written to the cache. For other public source feed flow tables, save them directly to the database or local file on a per-source basis and write them to the cache, where users can directly access them.

[0095] Read diffusion permissions are granted to the entire column, which minimizes the query pressure on the database. Therefore, the experimental setup only tests the performance of the write diffusion path.

[0096] In a test environment simulating 2 million users, 50 sources, 50,000 feed items, and corresponding permission records, the upper limit of write diffusion storage should be: .

[0097] The actual statistical results of the production environment are about 1 / 40 of the upper limit. records, the number of files is the same as the number of users * the number of sources.

[0098] , For a total of approximately Each file contains about 250 items, and the average file size is about 37KB.

[0099] In the performance test, the above data set was used, and 5 standard virtual machines were used for application load clustering. The backend was connected to an experimental environment with 5 standard virtual machine proxy service cache load clusters and a distributed file storage system. The test was conducted according to the standard requirement of 1000 concurrency.

[0100] Write diffusion performance: When using cache preloading, the time it takes to retrieve the first 100 data items in the feed is measured to be approximately 60 to 70 milliseconds. Without cache preloading, the retrieval time is approximately 3 seconds and gradually decreases.

[0101] The hybrid diffusion model distinguishes between read and write diffusion scenarios, simplifying the authorization methods and authentication computational complexity of read diffusion. At the same time, applying the write diffusion model to complex authorization scenarios eliminates the I / O bottlenecks inherent in the read diffusion model. In write diffusion, each user reads only their own data, and this data can be distributed, thus resolving resource contention issues in high-concurrency scenarios. The feed business is a typical read-heavy, write-short business scenario, with a read-write ratio exceeding 100:1. This means that, on average, each published message has at least 100 reads. Based on this specific application characteristic, write diffusion pre-positions the corresponding computations to the write operation, pre-empting the complex authentication, weighting, and grouping calculations to directly generate a result set. The acquirer directly pulls the source feed stream list from the result set, simplifying the business process and increasing speed.

[0102] However, the write-spread model also has certain drawbacks. First, this model consumes a significant amount of storage resources. Feed data is stored in multiple copies based on the number of users. In extreme cases, the storage capacity of a single feed can equal the number of users. Second, adding a new feed is computationally complex, and in extreme cases, the feed stream list for all users may need to be reconstructed.

[0103] The following describes the feed flow mixing and diffusion device based on the railway CMS derived application scenario provided by the present invention. The feed flow mixing and diffusion device based on the railway CMS derived application scenario described below and the feed flow mixing and diffusion method based on the railway CMS derived application scenario described above can be referenced to each other.

[0104] like Figure 7 As shown, in one embodiment, a feed flow mixing and diffusion device based on a railway CMS derivative application scenario includes an acquisition module 710 , a generation module 720 , a calculation module 730 and a separation module 740 .

[0105] The acquisition module 710 is used to acquire a multi-level authorization information storage structure, which includes a user table, an organization table, a feed table, and a column table.

[0106] The generation module 720 is used to generate a diffusion mode determination rule based on the column list and the user activity model.

[0107] The calculation module 730 is used to perform compound authorization calculation on the feed stream source according to the multi-level authorization information storage structure and diffusion mode determination rules, and perform aggregation calculation on the results of the first authorization calculation to generate a feed stream weighted table for the authorization source user.

[0108] The separation module 740 is used to obtain the Json data of the feed stream based on the request information of the feed stream, and then enter the write diffusion acquisition path after passing through the dynamic and static request separation of the dynamic and static separation gateway and the read-write diffusion separation of the user authorized feed stream call module.

[0109] In this embodiment, the generating module 720 is specifically configured to: A composite calculation is performed on the resource sensitivity and user activity to generate a priority determination function. The priority determination function is as follows: , in, For user activity, is resource sensitivity, is the user activity weight, Resource sensitivity weight, is the total weight; In response to a feed stream source passing the priority determination, write diffusion processing is performed on the feed stream source, and read diffusion processing is performed on other feed stream sources. The specific expression is as follows: .

[0110] In this embodiment, the generating module 720 is specifically configured to: Based on the column differentiation method and user activity, a priority determination function is generated. The priority determination function is as follows: , in, User activity; In response to a feed stream source passing the priority determination, write diffusion processing is performed on the feed stream source, and read diffusion processing is performed on other feed stream sources. The specific expression is as follows: 。

[0111] In this embodiment, the results of the first authorization calculation are aggregated to generate a feed flow weighting table for the authorized source user, specifically for: Based on the authorized feed prediction categories, the corresponding rounds of full feed calculation lists are started according to the release time. Feed prediction categories include popular selections and important tips. Determine the feeds and users involved in this round of feed flow changes; The feed flow table is persisted to the FastDFS distributed file system through write diffusion, and a feed flow weighted table for authorized source users is generated.

[0112] In this embodiment, determining the feeds and users involved in this round of feed stream changes is specifically used to: According to the scheduled publishing time, obtain the ID list of all feeds involved in this round; Calculate user access frequency, session duration, and content interaction depth in real time to obtain the feeds and users involved in this round of feed flow changes.

[0113] In this embodiment, the feed flow table is write-diffused, written to the distributed file system FastDFS, and the cache is cleared to generate a feed flow weighted table for the authorized source user, specifically for: Get all involved feeds and user information; Based on the aggregation calculation method, the feed stream file list of each user is read in turn, and the Json sequence in the list file is appended and saved to generate the feed stream weighted table of the authorized source user.

[0114] In this embodiment, the results of the first authorization calculation are aggregated to generate a feed flow weighting table of the authorization source user, which is also specifically used for: Call Purge to update the cache.

[0115] This feed stream hybrid diffusion device, based on railway CMS-derived application scenarios, simplifies the authorization method and authentication calculation amount of read diffusion by distinguishing between read and write diffusion scenarios in the hybrid diffusion mode. At the same time, the application of the write diffusion mode in complex authorization scenarios eliminates the IO bottleneck of the read diffusion mode. In write diffusion, each user only reads their own data, and this data can also be deployed in a distributed manner, thereby solving the resource competition problem under high concurrency. At the same time, by enabling write diffusion to pre-place the corresponding calculation to the write operation link in the feed business, the complex authentication, weighting, and group flow calculation processes are pre-placed to directly form a result set. The acquirer can directly pull the source feed stream list from the result set, simplifying the business process and increasing the speed.

[0116] Figure 8 The following is a schematic diagram of the physical structure of an electronic device. The electronic device may be a smart terminal, and its internal structure diagram may be as follows: Figure 8 As shown. The electronic device includes a processor, a memory, and a network interface connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a feed stream hybrid diffusion method based on a railway CMS derivative application scenario is implemented, the method comprising: Obtain a multi-level authorization information storage structure, which includes a user table, an organization table, a feed table, and a column table; Generate diffusion pattern determination rules based on the column list and user activity model; Based on the multi-level authorization information storage structure and diffusion mode judgment rules, a composite authorization calculation is performed on the feed flow source, and the results of the first authorization calculation are aggregated to generate a feed flow weighted table for the authorized source user; Based on the request information of the feed stream, it passes through the dynamic and static request separation of the dynamic and static separation gateway, the read-write diffusion separation of the user authorized feed stream call module, and then enters the write diffusion acquisition path to obtain the Json data of the feed stream.

[0117] Those skilled in the art will understand that Figure 8 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the electronic device to which the solution of the present invention is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0118] In another aspect, the present invention further provides a computer storage medium storing a computer program. When the computer program is executed by a processor, a feed stream hybrid diffusion method based on a railway CMS derivative application scenario is implemented. The method includes: Obtain a multi-level authorization information storage structure, which includes a user table, an organization table, a feed table, and a column table; Generate diffusion pattern determination rules based on the column list and user activity model; Based on the multi-level authorization information storage structure and diffusion mode judgment rules, a composite authorization calculation is performed on the feed flow source, and the results of the first authorization calculation are aggregated to generate a feed flow weighted table for the authorized source user; Based on the request information of the feed stream, it passes through the dynamic and static request separation of the dynamic and static separation gateway, the read-write diffusion separation of the user authorized feed stream call module, and then enters the write diffusion acquisition path to obtain the Json data of the feed stream.

[0119] In yet another aspect, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and when the processor executes the computer instructions, implements a feed stream mixing and diffusion method based on a railway CMS derivative application scenario, the method comprising: Obtain a multi-level authorization information storage structure, which includes a user table, an organization table, a feed table, and a column table; Generate diffusion pattern determination rules based on the column list and user activity model; Based on the multi-level authorization information storage structure and diffusion mode judgment rules, a composite authorization calculation is performed on the feed flow source, and the results of the first authorization calculation are aggregated to generate a feed flow weighted table for the authorized source user; Based on the request information of the feed stream, it passes through the dynamic and static request separation of the dynamic and static separation gateway, the read-write diffusion separation of the user authorized feed stream call module, and then enters the write diffusion acquisition path to obtain the Json data of the feed stream.

[0120] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0121] By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0122] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0123] The above-described embodiments merely illustrate several embodiments of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make various modifications and improvements without departing from the spirit of the present invention, and these modifications and improvements fall within the scope of the present invention. Therefore, the scope of the present invention shall be determined by the appended claims.

Claims

1. A feed flow hybrid diffusion method based on railway CMS derivative application scenarios, characterized by: The method comprises: Acquire a multi-level authorization information storage structure, wherein the multi-level authorization information storage structure includes a user table, an organization table, a feed table, and a column table; Generate diffusion pattern determination rules based on the column list and user activity model; Based on the multi-level authorization information storage structure and diffusion mode judgment rules, a composite authorization calculation is performed on the feed flow source, and the results of the first authorization calculation are aggregated to generate a feed flow weighted table for the authorized source user; Based on the request information of the feed stream, it passes through the dynamic and static request separation of the dynamic and static separation gateway, the read-write diffusion separation of the user authorized feed stream call module, and then enters the write diffusion acquisition path to obtain the Json data of the feed stream.

2. The feed flow mixed diffusion method based on railway CMS derived application scenarios according to claim 1 is characterized in that: The diffusion mode determination rules are generated based on the column list and the user activity model, including: A composite calculation is performed on the resource sensitivity and user activity to generate a priority determination function. The priority determination function is as follows: in, For user activity, is resource sensitivity, is the user activity weight, Resource sensitivity weight, is the total weight; In response to a feed stream source passing the priority determination, write diffusion processing is performed on the feed stream source, and read diffusion processing is performed on other feed stream sources. The specific expression is as follows: 。 3. The feed stream mixed diffusion method based on railway CMS derived application scenarios according to claim 1 is characterized in that: The diffusion mode determination rules are generated based on the column list and the user activity model, including: Based on the column differentiation method and user activity, a priority determination function is generated. The priority determination function is as follows: in, User activity; In response to a feed stream source passing the priority determination, write diffusion processing is performed on the feed stream source, and read diffusion processing is performed on other feed stream sources. The specific expression is as follows: 。 4. The feed flow mixed diffusion method based on railway CMS derivative application scenarios according to any one of claims 1 to 3 is characterized in that: The aggregation calculation is performed on the result of the first authorization calculation to generate a feed flow weighted table of the authorized source user, including: Based on the authorized feed prediction categories, the corresponding rounds of full feed calculation lists are started according to the release time. The feed prediction categories include popular selections and important reminders. Determine the feeds and users involved in this round of feed flow changes; The feed flow table is persisted to the FastDFS distributed file system through write diffusion, and a feed flow weighted table for authorized source users is generated.

5. The feed flow mixed diffusion method based on railway CMS derived application scenarios according to claim 4 is characterized in that: Determining the feeds and users involved in this round of feed stream changes includes: According to the scheduled publishing time, obtain the ID list of all feeds involved in this round; Calculate user access frequency, session duration, and content interaction depth in real time to obtain the feeds and users involved in this round of feed flow changes.

6. The feed stream mixed diffusion method based on railway CMS derived application scenarios according to claim 5 is characterized in that: The feed flow table is persisted to the FastDFS distributed file system through write diffusion, and a feed flow weighted table of the authorized source user is generated, including: Get all involved feeds and user information; Based on the aggregation calculation method, the feed stream file list of each user is read in turn, and the Json sequence in the list file is appended and saved to generate the feed stream weighted table of the authorized source user.

7. The feed flow mixed diffusion method based on railway CMS derived application scenarios according to claim 6 is characterized in that: The aggregation calculation of the result of the first authorization calculation to generate a feed flow weighted table of the authorized source user also includes: Call Purge to update the cache.

8. A feed flow mixing and diffusion device based on railway CMS derivative application scenarios, characterized by: include: An acquisition module, configured to acquire a multi-level authorization information storage structure, wherein the multi-level authorization information storage structure includes a user table, an organization table, a feed table, and a column table; A generation module is used to generate diffusion pattern determination rules based on the column list and user activity model; The calculation module is used to perform composite authorization calculation on the feed flow source based on the multi-level authorization information storage structure and diffusion mode determination rules, and aggregate the results of the first authorization calculation to generate a feed flow weighted table for the authorized source user; The separation module is used to process the request information based on the feed stream, which passes through the dynamic and static request separation of the dynamic and static separation gateway and the read-write diffusion separation of the user authorization feed stream call module, and then enters the write diffusion acquisition path to obtain the Json data of the feed stream.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images