Cross-platform e-commerce multi-source heterogeneous data automatic collection and cleaning system and method

Abstract

The application provides a cross-platform e-commerce multi-source heterogeneous data automatic acquisition and cleaning system and method, and belongs to the technical field of data processing, and specifically comprises the following steps: determining different users as update push user update attention e-commerce platforms based on the similarity of the update push users of different update attention e-commerce platforms; determining a monitoring and analysis demand platform in the e-commerce platform; taking the update attention e-commerce platform of the user as the update push user as a push associated platform; taking the user existing in the push associated platform as a screening push user; determining a monitoring and analysis method for the acquisition and cleaning processing of the monitoring and analysis demand platform based on the similarity of the associated e-commerce platforms of the monitoring and analysis demand platform and different screening push users and the push associated platform of the screening push user; and guaranteeing the reliability of data acquisition and analysis processing.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, and in particular relates to an automated system for collecting and cleaning multi-source heterogeneous data from cross-platform e-commerce. Background Technology

[0002] The domestic e-commerce market continues its steady growth, with omnichannel operations becoming standard practice for e-commerce companies. Multi-platform deployments are commonplace for small and medium-sized enterprises (SMEs) and branded e-commerce businesses. However, the industry's data processing models are generally lagging behind. The vast majority of companies still rely on manual data export, organization, and aggregation, which is not only costly and inefficient but also prone to data errors, omissions, and statistical delays. Furthermore, the inconsistent data structures and interface standards across e-commerce platforms make heterogeneous data integration difficult. The market lacks a universal, intelligent, and compliant integrated data collection and cleaning system, highlighting a significant weakness in the industry's digital transformation.

[0003] Existing technical solutions, such as CN202411272035.0 "Intelligent Preprocessing Method, Apparatus, Equipment and Storage Medium for Multi-Platform Merchant Data", obtain multiple initial target business data subsets with various data types and formats from multiple different e-commerce platform servers. After multi-dimensional preprocessing, they are transformed into preprocessed data with a unified data structure and format. This achieves smooth processing of differentiated data from various e-commerce platform servers and facilitates efficient application of preprocessed data. However, it has the following technical shortcomings: When the data interface rules of a user's associated e-commerce platform change, targeted modifications are often required before the system can be put back into use. Therefore, identifying the users pushed to different e-commerce platforms when their data interface rules change, and determining the monitoring and processing methods for the associated e-commerce platforms of the pushed users during data collection and cleaning based on the identification results of the pushed users, thereby improving the reliability of the overall multi-source data collection and cleaning process of the pushed users, has become an urgent technical problem to be solved.

[0004] In summary, to solve the above-mentioned technical problems, this application provides a cross-platform e-commerce multi-source heterogeneous data automated collection and cleaning system and method. Summary of the Invention

[0005] To achieve the objectives of this invention, the following technical solution is adopted: Specifically, in the first aspect, this application provides a method for automated collection and cleaning of multi-source heterogeneous data from cross-platform e-commerce, specifically including: S1 determines the number of associated e-commerce platforms for different users based on their associated e-commerce platform data, determines the e-commerce platforms to be updated and followed based on the number of associated e-commerce platforms for different users, and determines the users to whom the updated e-commerce platforms to be updated and followed are pushed based on the degree of overlap between the updated and followed e-commerce platforms and the associated e-commerce platforms of different users. S2 determines the e-commerce platforms that different users follow for updates based on the similarity of their update push users. Then, based on the e-commerce platforms that different users follow for updates and the associated e-commerce platforms, S2 determines the monitoring and analysis demand platforms in the e-commerce platforms. S3 uses the e-commerce platforms that the user follows as the update push user as the push association platform, and uses users with push association platforms as the filter push users. Based on the similarity between the monitoring and analysis demand platform and the associated e-commerce platforms of different filter push users, as well as the push association platforms of the filter push users, the monitoring and analysis demand platform determines the monitoring and analysis method for data collection, cleaning and processing.

[0006] The beneficial effects of this invention are as follows: Based on the number of e-commerce platforms associated with different users, the impact of changes in the data interface rules of associated e-commerce platforms on the automated collection and cleanup of heterogeneous data across multiple platforms is determined. Based on the impact of changes in the data interface rules on different users, the e-commerce platforms that need to be updated are identified, thus laying the foundation for targeted push processing during the update process. At the same time, it also ensures that the data interfaces of different users can operate reliably when the data interface rules change.

[0007] Based on the similarity between the monitoring and analysis demand platform and the associated e-commerce platforms of different filtered and pushed users, the reliability of monitoring and processing of associated e-commerce platforms of different filtered and pushed users is determined. Based on the reliability of monitoring and processing of associated e-commerce platforms of different filtered and pushed users, the monitoring and analysis method for data collection, cleaning and processing of the monitoring and analysis demand platform is determined. This reduces the difficulty of monitoring and analysis processing while ensuring the reliability of data collection, cleaning and processing of associated e-commerce platforms of different users.

[0008] Furthermore, the associated e-commerce platform is the e-commerce platform where the user's products are listed.

[0009] Furthermore, the method for determining the e-commerce platform to be updated and followed is as follows: S11 uses the number of associated e-commerce platforms of different users to determine users whose number of associated e-commerce platforms is above the threshold of the number of associated platforms. S12 considers users whose number of associated e-commerce platforms exceeds the threshold for the number of associated platforms as potential users affected by the update. S13 determines the e-commerce platforms that are concerned about the update based on the number of users potentially affected by the update and the number of associated e-commerce platforms of different users.

[0010] Furthermore, the method for determining the monitoring and analysis method for the data collection and cleaning process of the monitoring and analysis demand platform is as follows: Based on the similarity between the monitoring and analysis demand platform and the associated e-commerce platforms of different screened and pushed users, the proportion of the monitoring and analysis demand platform among the associated e-commerce platforms of screened and pushed users is determined, and the proportion of the monitoring and analysis demand platform among the associated e-commerce platforms of screened and pushed users is taken as the monitoring and analysis platform proportion. The push impact coefficient of the selected push users is determined based on the push association platform of different selected push users; The monitoring and analysis method for the collection, cleaning, and processing of data from the monitoring and analysis platform is determined by using different push impact coefficients for different filtered users and the proportion of monitoring and analysis platforms.

[0011] Secondly, this application provides a cross-platform e-commerce multi-source heterogeneous data automated collection and cleaning system, employing the aforementioned cross-platform e-commerce multi-source heterogeneous data automated collection and cleaning method, specifically including: Push user identification module, monitoring platform determination module, monitoring and analysis module; The push user identification module is responsible for identifying the users who are following the e-commerce platform for updates. The monitoring platform determination module is responsible for determining the monitoring and analysis needs of the e-commerce platform. The monitoring and analysis module is responsible for determining the monitoring and analysis methods for the data collection, cleaning, and processing of the monitoring and analysis requirements platform.

[0012] Other features and advantages will be set forth in the following description, and the objects and other advantages of the invention are realized and obtained through the structures particularly pointed out in the description and the drawings.

[0013] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0014] The above and other features and advantages of the present invention will become more apparent from a detailed description of exemplary embodiments thereof with reference to the accompanying drawings.

[0015] Figure 1 This is a flowchart of a cross-platform e-commerce multi-source heterogeneous data automated collection and cleaning method; Figure 2 This is a flowchart illustrating the methods for determining which e-commerce platforms to update and monitor. Figure 3 This is a flowchart illustrating the method for determining users who are following the latest updates from e-commerce platforms; Figure 4 This is a framework diagram of a cross-platform e-commerce multi-source heterogeneous data automated collection and cleaning system. Detailed Implementation

[0016] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0017] Example 1 like Figure 1 As shown, this application provides an automated method for collecting and cleaning multi-source heterogeneous data from cross-platform e-commerce, specifically including: S1 determines the number of associated e-commerce platforms for different users based on their associated e-commerce platform data, determines the e-commerce platforms to be updated and followed based on the number of associated e-commerce platforms for different users, and determines the users to whom the updated e-commerce platforms to be updated and followed are pushed based on the degree of overlap between the updated and followed e-commerce platforms and the associated e-commerce platforms of different users. S2 determines the e-commerce platforms that different users follow for updates based on the similarity of their update push users. Then, based on the e-commerce platforms that different users follow for updates and the associated e-commerce platforms, S2 determines the monitoring and analysis demand platforms in the e-commerce platforms. S3 uses the e-commerce platforms that the user follows as the update push user as the push association platform, and uses users with push association platforms as the filter push users. Based on the similarity between the monitoring and analysis demand platform and the associated e-commerce platforms of different filter push users, as well as the push association platforms of the filter push users, the monitoring and analysis demand platform determines the monitoring and analysis method for data collection, cleaning and processing.

[0018] Furthermore, the associated e-commerce platform is the e-commerce platform where the user's products are listed.

[0019] Specifically, such as Figure 2 As shown, the method for determining the e-commerce platform to update and follow is as follows: In this embodiment, based on the number of e-commerce platforms associated with different users, the impact of changes in the data interface rules of the associated e-commerce platforms on the automated collection and cleaning of heterogeneous data across multiple platforms is determined. Based on the impact of changes in the data interface rules on different users on the automated collection and cleaning of heterogeneous data across multiple platforms, the e-commerce platforms that users should pay attention to when updating are identified. This lays the foundation for targeted push processing during the update process and also ensures that the data interfaces of different users can operate reliably when the data interface rules change.

[0020] The core objective of this embodiment is to utilize the associated e-commerce platform data of different users to identify e-commerce platforms that frequently update their data interface rules or have a significant impact on users as e-commerce platforms to be monitored for updates, and further determine the user group that needs to receive update push notifications. Its core logic is to assess the breadth of the impact of interface updates by evaluating the number of platforms associated with a user, determine the leniency of the screening of e-commerce platforms to be monitored for updates by the proportion of potentially affected users and the overall degree of association, and then determine the scope of push users based on the overlap between the e-commerce platforms to be monitored for updates and the users. The overall logic follows the process of "statistical analysis of the number of associated platforms → identification of potentially affected users → screening of platforms to be monitored for updates → determination of push users".

[0021] S11 uses the number of associated e-commerce platforms of different users to determine users whose number of associated e-commerce platforms is above the threshold of the number of associated platforms. The associated e-commerce platforms refer to the e-commerce platforms on which the user's products are listed; the threshold for the number of associated platforms refers to the critical value used to distinguish between single-platform users and multi-platform users. Users whose number of associated e-commerce platforms exceeds this threshold are considered multi-platform users and are more sensitive to updates to data interface rules.

[0022] Assuming there are multiple users in the system, and each user lists products on one or more e-commerce platforms, then count the number of e-commerce platforms on which each user lists products, and mark users whose number of listings exceeds the threshold for the number of associated platforms.

[0023] This step quantifies the user's platform coverage into a statistically significant metric. Its significance lies in transforming the abstract concept of "multi-platform dependence" into a comparable value, providing an objective basis for subsequent screening to identify users potentially affected by updates.

[0024] S12 considers users whose number of associated e-commerce platforms exceeds the threshold for the number of associated platforms as potential users affected by the update. The term "potentially affected users" refers to users who are associated with a large number of e-commerce platforms and whose impact is significant when data interface rules change. These users are the basis for evaluating the degree of impact of e-commerce platform updates.

[0025] If a subset of users have more than a threshold number of associated e-commerce platforms, these users are marked as potentially affected by the update, which will be used to determine the overall impact of the e-commerce platform update.

[0026] This step involves selecting the user group most sensitive to platform updates from all users. Its significance lies in focusing subsequent impact assessments on users most likely to be affected by interface updates, thus avoiding an underestimation of the impact due to including all users of a single platform.

[0027] It is understandable that if there are no users among the users who may be affected by the update, then it is determined that there is no update on the e-commerce platform.

[0028] This step determines that there is no need to pay attention to updates from any e-commerce platform when there are no users on multiple platforms. The significance of this step is that when all users are single-platform users, the impact of data interface updates is extremely limited, and there is no need to follow up on updates and filter e-commerce platforms, thereby improving overall processing efficiency.

[0029] It is also understood that when there are users among the users who may be affected by the update, the process proceeds to step S13; S13 determines the e-commerce platforms that are concerned about the update based on the number of users potentially affected by the update and the number of associated e-commerce platforms of different users.

[0030] In cases where potential updates may affect users, this step further filters out e-commerce platforms that require special attention based on the distribution characteristics and overall correlation of affected users. The significance of this step is to transform the impact assessment at the user level into a priority ranking of updates at the platform level, thereby achieving accurate identification of e-commerce platforms with high-frequency updates or high risks.

[0031] The above steps include the following: Case 1: If the proportion of potential users affected by the update among the users is greater than the preset threshold for the proportion of potential users affected, then the e-commerce platform that is concerned about the update is determined to be an e-commerce platform whose data interface rule update count is greater than the preset update count threshold within the most recent preset time period, or whose minimum interval between adjacent update times is less than the preset interval length threshold within the most recent preset time period. The preset potential impact user ratio threshold refers to a critical value used to determine whether users from multiple platforms occupy a high proportion; the number of times the data interface rules are updated refers to the cumulative number of times the e-commerce platform has modified and released its data interface rules within the most recent preset time period; the minimum value of the interval between adjacent update times refers to the minimum time difference between any two adjacent update times among all update times, and the smaller this value is, the more frequent the updates are.

[0032] Assuming a high proportion of users are potentially affected by updates, indicating that users across multiple platforms are the majority and the impact of data interface updates is wide-ranging, a broader selection criterion is adopted—any e-commerce platform that updates frequently in a recent period or with short intervals between updates (i.e., frequent updates) is identified as an e-commerce platform to watch for updates.

[0033] This step uses broad screening criteria when the proportion of potentially affected users is high. The significance of this is that when the scope of the update's impact is wide, the criteria can cover more high-frequency update platforms, avoiding the omission of high-risk platforms due to overly strict screening criteria.

[0034] Additionally, it should be noted that in Case 2: if the proportion of users potentially affected by updates among the users is not greater than a preset threshold for the proportion of potentially affected users, it is determined whether the average number of associated e-commerce platforms for different users is less than a preset threshold for the number of associated platforms. If not, then the e-commerce platforms that are considered for updates are those whose data interface rules have been updated more than a preset number of updates within the most recent preset time period, or whose minimum interval between adjacent update times within the most recent preset time period is less than a preset interval time threshold. If so, then the e-commerce platforms that are considered for updates are those whose data interface rules have been updated more than a preset number of updates within the most recent preset time period.

[0035] The preset threshold for the number of associated platforms refers to the critical value of the average number of associated platforms that is less than the preset threshold for the proportion of potentially affected users. This is used to further tighten the screening criteria when the overall degree of association is still acceptable.

[0036] If the proportion of users potentially affected by updates is not high, but the average number of associated e-commerce platforms for all users is high, indicating that although the proportion of users with multiple platforms is not high, the overall correlation is still strong, then a broad screening condition will still be used; if the average is also low, indicating that the overall correlation is weak, then a stricter screening condition will be used—only e-commerce platforms with more frequent updates will be identified as e-commerce platforms that are frequently updated, and platforms with very short update intervals but few updates will no longer be included.

[0037] This step further determines the leniency of the screening criteria based on the overall degree of correlation when the proportion of potentially affected users is not high. Its significance lies in avoiding the problem of using overly strict conditions when the overall degree of correlation is strong due to judging solely by the proportion, thus achieving a two-dimensional verification of the impact range assessment.

[0038] This embodiment, through the combination of steps S11 to S13, realizes the entire process from user-associated platform data to the determination of updated and focused e-commerce platforms. Its core value is reflected in three aspects: First, by setting a threshold for the number of associated platforms, it distinguishes between single-platform users and multi-platform users, focusing the impact assessment on sensitive user groups; second, by using a two-dimensional judgment of the proportion of potentially affected users and the average number of associated platforms, it achieves adaptive selection between broad and strict screening conditions; third, through the complementary coverage of scenarios 1 and 2, it ensures that the e-commerce platforms requiring key attention can be accurately identified under different user distribution characteristics.

[0039] Assume there are 10 e-commerce platforms in the system (P1 to P10). In S11, the threshold for the number of associated platforms is set to 4. The number of e-commerce platforms associated with each user is counted. Users with more than 4 associated e-commerce platforms are: U2 (5), U3 (5), and U8 (8), a total of 3 users. In S12, these 3 users are marked as potential users affected by the update.

[0040] In S13, the proportion of users potentially affected by the update is calculated as 3 ÷ 50 = 6%. The preset threshold for the proportion of potentially affected users is 30%. Since 6% is not greater than 30%, the judgment in case 2 is triggered. The average number of associated e-commerce platforms for different users is calculated: the total number of associated platforms for 50 users is 162, and the average value is 162 ÷ 50 = 3.24. The preset threshold for the number of associated platforms (which is less than the average value corresponding to the preset threshold for the proportion of potentially affected users) is set to 3.00. Since 3.24 is not less than 3.00, the judgment result for case 2 is "if not". At this time, the e-commerce platforms to be monitored for updates are defined as those whose data interface rule updates within the most recent preset period (90 days) are greater than the preset update frequency threshold of 6 times, or whose minimum interval between adjacent update times within the most recent 90 days is less than the preset interval duration threshold of 14 days.

[0041] Specifically, such as Figure 3 As shown, the method for determining users who are interested in receiving updates from the e-commerce platform is as follows: In this embodiment, based on the degree of overlap between the e-commerce platform being updated and the associated e-commerce platforms of different users, as well as the associated e-commerce platforms of the users, the impact of frequent updates on the reliability of the collection, analysis, and processing of multi-source heterogeneous data from different users is determined. Based on the impact of frequent updates on the reliability of the collection, analysis, and processing of multi-source heterogeneous data from different users, the update push users of the update-focused e-commerce platform are determined. This reduces the number of update push users while also ensuring the reliability of the verification processing after the update processing of the update-focused e-commerce platform.

[0042] The core objective of this embodiment is to analyze the similarity of users associated with each e-commerce platform based on the e-commerce platforms identified in S1, determine the e-commerce platforms each user needs to follow for updates, and then evaluate the importance of each e-commerce platform in update pushes from the user's perspective. Ultimately, this identifies the e-commerce platforms that require focused monitoring and analysis. Its core logic is to assess the concentration of update impact by evaluating the overlap of associated users, measure user sensitivity by the update impact weight coefficient, and comprehensively determine the scope of push users and the platforms requiring monitoring and analysis. The overall logic follows the process of "associated user identification → association coefficient evaluation → update impact weight calculation → push user determination → monitoring requirement platform identification".

[0043] S21 Based on the degree of overlap between the updated and followed e-commerce platforms and the associated e-commerce platforms of different users, determine the users in the associated e-commerce platforms who are updated and followed, and designate the users in the associated e-commerce platforms who are updated and followed as the associated users of the updated and followed e-commerce platforms. The "associated users of the updated e-commerce platform" refers to users whose associated e-commerce platform list includes the updated e-commerce platform. In other words, when the data interface rules of the updated e-commerce platform are updated, the e-commerce data collection and processing of these users will be affected.

[0044] Assuming the updated followed e-commerce platform is P3, then iterate through the list of all users' associated e-commerce platforms and mark users whose lists contain P3 as associated users of P3.

[0045] This step establishes a mapping relationship between the updated e-commerce platform and the users affected by it. Its significance lies in transforming the updated targets at the platform level into impact assessment targets at the user level, providing basic data for subsequent calculation of correlation coefficients and identification of push users.

[0046] It is understood that the above steps include the following: The correlation coefficient of the updated e-commerce platform is determined by the proportion of the number of users associated with the updated e-commerce platform among all users. It is then determined whether the correlation coefficient of the updated e-commerce platform is less than a preset correlation coefficient threshold. If so, the updated e-commerce platform push users are determined to be the users with the most associated e-commerce platforms among a preset number of users, thereby reducing the impact of data processing anomalies. If not, proceed to step S22. The correlation coefficient refers to the proportion of the number of users associated with a certain e-commerce platform that is followed by an update to the total number of all users. It is used to measure the coverage of users affected when the platform updates. The preset number refers to the upper limit of the number of users pushed for each update. By limiting the number of users pushed, the scope of impact when data processing anomalies are reduced.

[0047] Suppose that the number of users associated with a certain e-commerce platform that is being updated is small and the correlation coefficient is low, indicating that fewer users will be affected when the platform updates. In this case, only the top few users with the most associated e-commerce platforms will be selected as update push users, because these users have the highest dependence on the platform and need to be informed of the update information in a timely manner.

[0048] This step uses a smaller number of push users when the correlation coefficient is low. Its significance is that when the impact of platform updates is small, by pushing update information only to a portion of the related users, the overall workload of push processing is reduced, and the impact is reduced when data processing is abnormal.

[0049] S22, determine the number of associated e-commerce platforms of the associated users based on the associated e-commerce platforms of different associated users; This step counts the total number of e-commerce platforms associated with each user. Its significance lies in quantifying the degree of dependence of each user on data from multiple platforms, providing input data for subsequent calculation and updating of the influence weight coefficient.

[0050] It is understood that in the above steps, the update influence weight coefficient of the associated user is determined by multiplying the number of associated e-commerce platforms of the associated user with the preset ratio factor. It is then determined whether the average value of the update influence weight coefficient of different associated users is greater than the preset influence weight coefficient threshold. If so, the update push users of the e-commerce platforms that are followed are determined to be the associated users with the most associated e-commerce platforms among the preset number of such users and the associated users whose update influence weight coefficient is greater than the preset coefficient threshold. If not, the process proceeds to step S22. The preset proportional factor refers to the conversion factor used to convert the number of associated e-commerce platforms into a weight coefficient. Let the updated influence weight coefficient = number of associated e-commerce platforms × preset proportional factor. The value of the proportional factor is such that the weight coefficient ranges from 0 to 1. The second threshold value of the preset coefficient threshold is greater than the preset influence weight coefficient threshold value, which is used to further filter out associated users with particularly high weight coefficients when the average value is high.

[0051] If a user has a large number of associated e-commerce platforms, their update impact weight coefficient is high, indicating that the user has a strong dependence on data from multiple platforms and needs to be closely monitored when the platform updates. When the proportion of users with high weight coefficients is high, in addition to a preset number of core users, users with particularly high weight coefficients will also be included in the push notification range.

[0052] This step, when the correlation coefficient is high (i.e., a large number of users are affected), further determines the scope of users to be pushed based on the users' platform dependence. Its significance is that when the platform update has a wide impact, it will not only push to core users, but also include users with a particularly high degree of dependence on multiple platforms, thereby ensuring the comprehensiveness and accuracy of the identification and handling of anomalies after the update.

[0053] S23 determines the update push users for the e-commerce platforms they are following based on the associated users of the updated e-commerce platforms and the number of associated e-commerce platforms for different associated users.

[0054] This step, when the proportion of users with high weight coefficients among associated users is not high, uses a combination of association coefficients and weight coefficients to determine the final push users. Its significance lies in achieving accurate determination of the push user range through multi-dimensional comprehensive judgment in scenarios with moderate impact.

[0055] Specifically, the above steps include the following: S231, using the correlation coefficient of the updated e-commerce platform and the average value of the update influence weight coefficient of different related users, determine the update influence value of the updated e-commerce platform, and determine whether the update influence value of the updated e-commerce platform is greater than the preset influence threshold. If yes, determine that the update push users of the updated e-commerce platform are the related users with the largest number of related e-commerce platforms and the related users whose update influence weight coefficient is greater than the preset coefficient threshold (greater than the preset influence weight coefficient threshold). If not, proceed to step S232. The update impact value refers to the index calculated by combining the correlation coefficient and the average of the update impact weight coefficient. It is used to measure the overall impact of the update on the e-commerce platform during the update. Let the update impact value = correlation coefficient × 0.5 + average of update impact weight coefficient × 0.5.

[0056] If an update has a high correlation coefficient with the e-commerce platform it follows and a high average weight coefficient for the associated users, then the update has a significant impact. This indicates that the overall impact of the update on that platform is profound, and a larger push user range is needed to ensure coverage.

[0057] S232, determine whether there are any associated users whose update impact weight coefficient is greater than a preset impact weight coefficient threshold. If yes, determine that the update push user who is following the e-commerce platform is the associated user with the largest number of associated e-commerce platforms among the preset number of associated e-commerce platforms, or the associated user whose update impact weight coefficient is greater than the preset coefficient threshold and belongs only to the associated user who is following the e-commerce platform with an update impact weight coefficient of no more than the preset number of platforms. If no, determine that the update push user who is following the e-commerce platform is the associated user with the largest number of associated e-commerce platforms among the preset number of associated e-commerce platforms.

[0058] The term "associated users who follow e-commerce platforms that are above the preset number of platforms" refers to associated users whose number of updated e-commerce platforms followed by the push user is less than the preset threshold. These users are high-weight users, but they follow fewer e-commerce platforms, so the impact of anomalies during push verification is low.

[0059] Assuming there are users with high weighting coefficients, but some of these users are also push users of multiple e-commerce platforms that follow updates, if an anomaly occurs after an update from any one platform, the impact will be significant. Therefore, users who belong to only a few e-commerce platforms that follow updates will be prioritized as supplementary push targets to ensure comprehensive coverage while reducing the impact of anomalies.

[0060] This embodiment, through the combination of steps S21 to S232, realizes the entire process from updating the e-commerce platform to determining the target users. Its core value lies in four aspects: First, by calculating the correlation coefficient, it achieves a quantitative assessment of the platform's influence breadth, providing a basis for the initial determination of the target user scope; second, by calculating the update influence weight coefficient, it achieves a quantitative assessment of the user's platform dependence, providing a tool for accurately identifying highly sensitive users; third, by comprehensively calculating the update influence value, it achieves a unified assessment of both the platform's influence and user sensitivity, enabling adaptive adjustment of the target user scope; fourth, through the three-level progressive judgment of S21, S22, and S23, it ensures that an appropriate target user scope can be determined under different combinations of correlation coefficients and weight coefficients.

[0061] Based on the results of S1, the e-commerce platforms to be updated and monitored are P1, P3, P6, and P8, a total of four platforms. The process of determining the users to be updated will be illustrated using P3 as an example.

[0062] In S21, the list of 50 users' associated e-commerce platforms is iterated to determine the users associated with P3. U1 (associated with P1, P3, P6), U2 (associated with P1, P3, P4, P7, P8), U3 (associated with P1, P3, P6, P8, P9), U5 (associated with P1, P6, P8), U7 (associated with P3, P5, P9), and U8 (associated with P1, P3, P4, P6, P7, P8, P9, P10) are all associated with P3, totaling 6 associated users. The association coefficient = 6 ÷ 50 = 12%. The preset association coefficient threshold is 15%. Since 12% is less than 15%, the judgment result of S21 is "if". At this point, the update push users for P3 are determined to be the top 3 users with the most associated e-commerce platforms (the preset number is 3).

[0063] The number of e-commerce platforms associated with each user is as follows: U8 has 8, U2 has 5, U3 has 5, U1 has 3, U5 has 3, and U7 has 3. The top 3 with the most associated e-commerce platforms are U8 (8), U2 (5), and U3 (5). Therefore, the update push notifications for P3 are sent to users U8, U2, and U3, a total of 3 users.

[0064] Specifically, the method for determining the monitoring and analysis needs platform within the e-commerce platform is as follows: In this embodiment, based on the updated e-commerce platform data and associated e-commerce platform data of different users as update push users, the degree of impact on the reliability of normal data collection and cleaning for the corresponding users when push processing is performed immediately after the update of the e-commerce platform they follow is determined. Based on the degree of impact on the reliability of normal data collection and cleaning for different users, the monitoring and analysis demand platform in the e-commerce platform is determined, thereby laying the foundation for ensuring the overall reliability of user data collection and cleaning processing.

[0065] The core objective of this embodiment is to identify, from the user's perspective, which e-commerce platforms require focused data collection and analysis based on the user's associated push platforms (i.e., the e-commerce platforms the user follows as an update pusher) determined in S2, and then determine the monitoring and analysis methods for each type of platform. Its core logic is to assess the importance of e-commerce platforms in update pushes by filtering the coverage ratio and push impact coefficient of push users, and determine the type of platform requiring monitoring and analysis based on the importance classification. The overall logic follows the process of "screening and identifying push users → calculating push impact coefficient → determining platform type → determining monitoring and analysis methods".

[0066] S31 uses the e-commerce platforms that the user follows as the update push user as the push association platform, and uses users with push association platforms as the filter push users. The push-related platform refers to the e-commerce platform that a user follows as an update push user, that is, the e-commerce platform that the user needs to follow for updates; the filtered push users refer to users who are update push users in at least one update follow e-commerce platform, which is the set of users after deduplication of the sum of all update follow e-commerce platform push users.

[0067] Assuming that the push users of P3 are U8, U2, and U3, and the push users of P1 are U1, U8, and U5, then U8, U2, U3, U1, and U5 are all selected push users, and their push-related platforms are P3 and P1, respectively, which are different from each other.

[0068] This step involves compiling all users who need to follow the updates from the platform's push user list. Its significance lies in integrating the push user lists scattered across various platforms into a user-level set of filtered push users, providing a unified user base for subsequent evaluation of the importance of each e-commerce platform from the user's perspective.

[0069] The above steps include the following: Based on the number of users selected for push notifications, determine the proportion of users selected for push notifications among all users, and use this proportion as the proportion of users selected for push notifications. Determine whether the proportion of users selected for push notifications is greater than a preset user proportion threshold. If so, determine that the monitoring and analysis demand platform in the e-commerce platform is a preset type of e-commerce platform. If not, proceed to step S32.

[0070] The ratio of users selected for push notifications refers to the proportion of the number of users selected for push notifications to the total number of all users, which is used to measure the coverage of the updated push information; the preset user ratio threshold is a critical value used to determine whether the updated push information has a wide coverage.

[0071] If the proportion of users selected for push notifications is high, it indicates that the updated push information covers a large number of users. In this case, the update push processing of each e-commerce platform has a significant impact on the reliability of overall user data collection and cleaning. Therefore, it is necessary to identify some e-commerce platforms as monitoring and analysis platforms to focus on monitoring the data collection and cleaning strategies of these platforms after the update.

[0072] This step directly enters the process of determining the preset type of platform when the update push information has a wide coverage. Its significance lies in the fact that when the impact of the update push is large, it is necessary to monitor and analyze key e-commerce platforms to ensure that the data collection, cleaning and processing after the update can run reliably at the level of most users.

[0073] Specifically, the method for determining the preset type of e-commerce platform is as follows: Based on the degree of association between different users and related e-commerce platforms, the system identifies users of the e-commerce platform that belong to the associated e-commerce platform and uses them as the filtered associated users of the e-commerce platform. If the proportion of users of the e-commerce platform that belong to the associated e-commerce platform and is among the filtered users of the associated e-commerce platform is above a preset proportion threshold, or if the average value of the push influence coefficient of the e-commerce platform that belongs to the associated e-commerce platform and is among the filtered users of the associated e-commerce platform is above a preset influence coefficient threshold, then the e-commerce platform is identified as an e-commerce platform of a preset type. This ensures the reliability of its operation and avoids the situation where multiple e-commerce platforms have problems simultaneously due to inadequate data collection and cleaning strategies when the e-commerce platform used as the push update user has insufficient data collection and cleaning strategies, resulting in poor user reliability.

[0074] The e-commerce platform's filtered associated users refer to the portion of the e-commerce platform's associated users who are also filtered push users; the push influence coefficient refers to the proportion of the number of push-associated platforms of the filtered push user to the total number of its associated e-commerce platforms, used to measure the degree of user's dependence on push-associated platforms.

[0075] If a high proportion of users associated with an e-commerce platform are users who have selected and pushed content, or if the average push impact coefficient of these users is high, it indicates that the e-commerce platform is of high importance in the update push system and should be identified as a monitoring and analysis platform of the preset type.

[0076] S32 determines the push impact coefficient of the selected push users based on the proportion of the number of push-related platforms among the e-commerce platforms associated with the selected push users; The percentage of push-related platforms refers to the proportion of the number of push-related platforms for a certain selected push user to the total number of its associated e-commerce platforms, i.e., the push influence coefficient. The higher the coefficient, the more concentrated the user's updated push information is, and the higher the user's dependence on push-related platforms.

[0077] Suppose a user who receives targeted push notifications is associated with 6 e-commerce platforms, 2 of which are the platforms they are associated with. Then their push notification influence coefficient = 2 ÷ 6 ≈ 0.33, indicating that the user's updated push information is relatively concentrated.

[0078] This step quantifies the degree of dependence concentration of each screened push user on the push-related platform. Its significance lies in providing quantitative input for subsequent judgment of whether there are users with high push impact and for calculating the corrected weight coefficient.

[0079] It is understandable that in the above steps, the push impact coefficient of different filtered push users is used to determine whether to push to filtered push users whose push impact coefficient is above the preset impact coefficient threshold. If yes, then proceed to step S33; if no, then determine that the monitoring and analysis demand platform in the e-commerce platform is the second preset type of e-commerce platform.

[0080] This step directly identifies the monitoring and analysis demand platform as the second preset type when there is no high push impact on users. The significance of this step is that when the push impact coefficient of all screened push users is low, it indicates that the updated push information is relatively dispersed and the concentrated impact of each e-commerce platform is not high. At this time, the relatively lenient second preset type is used to determine the monitoring and analysis demand platform.

[0081] Specifically, the method for determining the e-commerce platform of the second preset type is as follows: Based on the degree of association between different users and related e-commerce platforms, the system determines the users of the e-commerce platform that belong to the associated e-commerce platform and selects them as the filtered associated users of the e-commerce platform. If the proportion of the e-commerce platform that belongs to the filtered associated users of the associated e-commerce platform is above a preset threshold, then the e-commerce platform is determined as a second preset type of e-commerce platform. This ensures the reliability of its operation and avoids the situation where multiple e-commerce platforms have problems simultaneously due to inadequate data collection and cleaning strategies when the e-commerce platform used as the push update user has insufficient data collection and cleaning strategies, resulting in poor user reliability.

[0082] This step uses a single condition (the proportion of associated users) to determine the second preset type of platform. Its significance lies in determining the monitoring and analysis needs platform under stricter conditions in scenarios with low push impact coefficients, thereby reducing the difficulty of monitoring, analysis and processing.

[0083] S33 determines the monitoring and analysis needs platform in the e-commerce platform by using different push impact coefficients for filtered push users and the number of filtered push users.

[0084] This step further refines the determination of platform type by comprehensively considering the push impact coefficient and the number of users being pushed to when there are users with high push impact. Its significance lies in achieving accurate classification of platform types for monitoring and analysis needs through multi-dimensional comprehensive judgment in scenarios with high concentrated impact.

[0085] It is understandable that the monitoring and analysis needs of the e-commerce platform are determined by the push impact coefficient of different filtered push users and the number of filtered push users, specifically including: The average of the push impact coefficients of different filtered push users is used as the correction weight coefficient. Based on the product of the correction demand coefficient and the proportion of filtered push users, the proportion of corrected push users is determined. It is then determined whether the proportion of corrected push users is greater than the preset value of the correction proportion. If so, the monitoring and analysis demand platform in the e-commerce platform is determined to be a preset type of e-commerce platform. If not, the monitoring and analysis demand platform in the e-commerce platform is determined to be a third preset type of e-commerce platform.

[0086] The corrected weighting coefficient refers to the arithmetic mean of the push impact coefficients of all screened push users; the corrected push user ratio refers to the product of the corrected weighting coefficient and the screened push user ratio, which is used to measure the effective push user coverage after the push impact concentration correction.

[0087] Assuming that the correction weight coefficient is high and the proportion of filtered push users is also high, the proportion of corrected push users is high, indicating that the effective push user coverage is still relatively wide after the correction of the concentration of push influence. At this time, the monitoring and analysis platform is determined to be the preset type of e-commerce platform.

[0088] This step makes a judgment by adjusting the comprehensive indicator of the proportion of push users. Its significance lies in further evaluating the actual impact range after concentration adjustment in the case of users with high push impact, so as to achieve accurate differentiation between the preset type and the third preset type platform.

[0089] Specifically, the method for determining the third preset type of e-commerce platform is as follows: Based on the degree of association between different users and related e-commerce platforms, the system identifies users of the e-commerce platform that belong to the associated e-commerce platforms and uses them as the filtered associated users of the e-commerce platform. If the proportion of users of the e-commerce platform that belong to the associated e-commerce platforms is above a preset proportion threshold, or if the average value of the push influence coefficient of the users of the e-commerce platform that belong to the associated e-commerce platforms is above a preset influence coefficient threshold, and the proportion of users of the e-commerce platform that belong to the associated e-commerce platforms is above a second preset proportion threshold (less than the preset proportion threshold), then the e-commerce platform is identified as a third preset type of e-commerce platform. This ensures the reliability of its operation and avoids the situation where multiple e-commerce platforms have problems simultaneously due to inadequate data collection and cleaning strategies when the e-commerce platform used as the push update user has insufficient data collection and cleaning strategies, resulting in poor user reliability.

[0090] This step determines the third preset type of platform through the OR combination of two conditions. Its significance lies in the fact that even when the proportion of users receiving corrected push notifications is not high, some important e-commerce platforms can still be identified as platforms requiring monitoring and analysis by using relatively lenient conditions (the proportion meets the standard, or the proportion is close to meeting the standard and the push impact coefficient is high), thus avoiding the omission of important platforms in scenarios with low correction ratios.

[0091] It should be noted that the monitoring and analysis platform mentioned refers to e-commerce platforms other than those that are updated and followed.

[0092] Based on the results of S2, the e-commerce platforms to be monitored are updated to P1, P3, P6, and P8. The push users for each platform are summarized as follows: Push users for P1: U1, U8, U5 (3 users); Push users for P3: U8, U2, U3 (3 users); Push users for P6: U1, U5, U6 (3 users); Push users for P8: U2, U5, U8, U9 (4 users).

[0093] In step S31, the push-related platforms are determined as follows: U1's push-related platforms are P1 and P6 (U1 is a push user of P1 and P6); U2's push-related platforms are P3 and P8; U3's push-related platform is P3; U5's push-related platforms are P1, P6, and P8; U6's push-related platform is P6; U8's push-related platforms are P1, P3, and P8; and U9's push-related platform is P8. There are 7 users with push-related platforms: U1, U2, U3, U5, U6, U8, and U9. These 7 users are selected as the push-related users. The selected push-related user ratio = 7 ÷ 50 = 14%. The preset user ratio threshold is 25%. Since 14% is not greater than 25%, proceed to step S32.

[0094] In step S32, the push impact coefficient (number of push-related platforms ÷ total number of associated e-commerce platforms) is calculated for each selected user: U1 is associated with 6 platforms and pushes to 2 platforms, coefficient = 2 ÷ 6 ≈ 0.33; U2 is associated with 5 platforms and pushes to 2 platforms, coefficient = 2 ÷ 5 = 0.40; U3 is associated with 5 platforms and pushes to 1 platform, coefficient = 1 ÷ 5 = 0.20; U5 is associated with 3 platforms and pushes to 3 platforms, coefficient = 3 ÷ 3 = 1.00; U6 is associated with 3 platforms and pushes to 1 platform, coefficient = 1 ÷ 3 ≈ 0.33; U8 is associated with 8 platforms and pushes to 3 platforms, coefficient = 3 ÷ 8 = 0.375; U9 is associated with 3 platforms and pushes to 1 platform, coefficient = 1 ÷ 3 ≈ 0.33. Selected users with a push impact coefficient above the preset threshold of 0.40 are: U2 (0.40) and U5 (1.00), a total of 2. Since such users exist, proceed to step S33.

[0095] In S33, the adjusted weight coefficient = the average of the influence coefficients of each selected push user = (0.33 + 0.40 + 0.20 + 1.00 + 0.33 + 0.375 + 0.33) ÷ 7 ≈ 3.0 ÷ 7 ≈ 0.429. The adjusted push user ratio = adjusted weight coefficient × selected push user ratio = 0.429 × 14% ≈ 0.060. The preset adjusted ratio is 0.10. Since 0.060 is not greater than 0.10, the judgment result of S33 is "if not", determining the monitoring and analysis demand platform as the third preset type of e-commerce platform.

[0096] The third preset type of e-commerce platforms is determined by: traversing each e-commerce platform and judging whether the proportion of filtered associated users reaches the preset threshold of 30%, or whether it simultaneously meets the following conditions: the average push impact coefficient reaches the preset impact coefficient threshold of 0.40 or higher, and the proportion of filtered associated users reaches the second preset threshold of 20% (less than 30%). Therefore, the platforms requiring monitoring and analysis are the four platforms that belong to the third preset type, excluding P1, P3, P6, and P8.

[0097] Specifically, the method for determining the third preset type of e-commerce platform is as follows: Based on the degree of association between different users and related e-commerce platforms, the system identifies users of the e-commerce platform that belong to the associated e-commerce platforms and uses them as the filtered associated users of the e-commerce platform. If the proportion of users of the e-commerce platform that belong to the associated e-commerce platforms is above a preset proportion threshold, or if the average value of the push influence coefficient of the users of the e-commerce platform that belong to the associated e-commerce platforms is above a preset influence coefficient threshold, and the proportion of users of the e-commerce platform that belong to the associated e-commerce platforms is above a second preset proportion threshold (less than the preset proportion threshold), then the e-commerce platform is identified as a third preset type of e-commerce platform. This ensures the reliability of its operation and avoids the situation where multiple e-commerce platforms have problems simultaneously due to inadequate data collection and cleaning strategies when the e-commerce platform used as the push update user has insufficient data collection and cleaning strategies, resulting in poor user reliability.

[0098] Furthermore, the method for determining the monitoring and analysis method for the data collection and cleaning process of the monitoring and analysis demand platform is as follows: In this embodiment, the reliability of monitoring and processing of the associated e-commerce platforms of different users is determined based on the similarity between the monitoring and analysis demand platform and the associated e-commerce platforms of different users. Based on the reliability of monitoring and processing of the associated e-commerce platforms of different users, the monitoring and analysis method for data collection and cleaning of the monitoring and analysis demand platform is determined. This reduces the difficulty of monitoring and analysis while ensuring the reliability of data collection and cleaning of the associated e-commerce platforms of different users.

[0099] The core objective of this embodiment is to determine the specific methods for data collection, cleaning, and monitoring analysis of each monitoring and analysis platform based on the monitoring and analysis requirement platforms identified in S3, and the degree of similarity between the screened push users and these platforms. Its core logic is to assess the coverage concentration of each platform among the screened push users by evaluating the platform ratio, and to assess the concentration of user impact from platform updates by using the push impact coefficient. Combining these two factors, a user grouping and rotation monitoring strategy is determined. The overall logic follows the process of "platform ratio calculation → push impact correction → deviation user identification → hierarchical determination of monitoring methods."

[0100] S41 determines the proportion of monitoring and analysis demand platforms among the e-commerce platforms associated with different screened and pushed users based on the similarity between the monitoring and analysis demand platform and the associated e-commerce platforms of different screened and pushed users, and uses the proportion of monitoring and analysis demand platforms among the associated e-commerce platforms of screened and pushed users as the monitoring and analysis platform proportion. The monitoring and analysis platform ratio refers to the proportion of the number of e-commerce platforms associated with a user that meet the monitoring and analysis requirements, relative to the total number of e-commerce platforms associated with that user. It is used to measure the reliability of monitoring the user's associated e-commerce platforms.

[0101] Suppose a user who receives targeted push notifications is associated with multiple e-commerce platforms, some of which are platforms that require monitoring and analysis. The higher this proportion, the more concentrated the user's updates will be, and the higher the reliability of the monitoring and analysis.

[0102] This step quantifies the reliability of monitoring and analysis of each screened user, and its significance lies in providing a user-level concentration indicator for subsequent judgment on whether to adopt high-frequency monitoring methods.

[0103] It is understandable that in the above steps, based on the average of the proportion of different filtered push users on the monitoring and analysis platform, it is determined whether the average of the proportion of different filtered push users on the monitoring and analysis platform is less than the preset value of the proportion of the monitoring and analysis platform. If so, the monitoring and analysis method for the collection and cleaning of the monitoring and analysis demand platform is determined to be based on the preset proportion, randomly dividing the users of the associated e-commerce platform on the monitoring and analysis demand platform into a group, and monitoring the data collection and cleaning of the users in the group on the monitoring and analysis demand platform in real time during each preset time period, and switching the group according to the preset period. If not, proceed to step S42. The preset value of the monitoring and analysis platform ratio refers to the critical value used to judge whether the reliability of the monitoring and analysis processing of the screened users is low. When the average value is low, it indicates that the reliability of the monitoring and analysis processing is low, and a more comprehensive rotation monitoring method can be adopted.

[0104] If the average proportion of monitoring and analysis platforms for each screened user is low, it indicates that user monitoring and analysis is not particularly concentrated on platforms with monitoring and analysis needs. Therefore, a rotation monitoring method that randomly groups users is adopted to ensure that users in each group can be monitored within a certain period.

[0105] This step employs a comprehensive rotation monitoring method in scenarios where the impact of updates is relatively dispersed. Its significance lies in the fact that when the impact of user updates is not concentrated on a few platforms, it is necessary to conduct comprehensive monitoring of users on all platforms with related monitoring and analysis needs, and to achieve full coverage through random grouping and rotation.

[0106] S42 determines the push influence coefficient of the selected push users based on the push association platforms of different selected push users; The push impact coefficient refers to the proportion of the number of push-related platforms calculated in S3 to the total number of related e-commerce platforms. It is reused in S4 to measure the degree of user dependence on push-related platforms.

[0107] This step introduces the push impact coefficient calculated in S3 into the analysis of S4. Its significance lies in avoiding double calculation and maintaining the consistency of the push impact coefficient in S3 and S4.

[0108] It is understood that the above steps include the following: The proportion of the monitoring and analysis platform is adjusted by using the push impact coefficient of different filtered push users to obtain the adjusted platform proportion. It is then determined whether the average of the adjusted platform proportions of different filtered push users is above the preset adjustment proportion threshold. If not, proceed to step S43. If yes, the monitoring and analysis method for the collection and cleaning processing of the monitoring and analysis demand platform is determined as follows: based on the preset proportion, the filtered push users of the associated e-commerce platform that exist in the monitoring and analysis demand platform are randomly divided into a group. In each preset time period, the data collection and cleaning processing of the filtered push users in the group on the monitoring and analysis demand platform is monitored in real time, and the group switching processing is performed according to the preset period.

[0109] The corrected platform ratio refers to the indicator after correcting the monitoring and analysis platform ratio according to the push impact coefficient. Let the corrected platform ratio = (1 - push impact coefficient) × monitoring and analysis platform ratio. The higher the value, the lower the degree of user still needs to be monitored after the correction of the concentration of push impact.

[0110] Assuming that the average correction rate of each screened push user is high, it indicates that even after considering the concentration of push impact, these users still need relatively comprehensive monitoring. Therefore, a rotation monitoring method that randomly groups screened push users is adopted to monitor only screened push users, thereby narrowing the monitoring scope and improving efficiency.

[0111] This step employs a rotational monitoring method targeting only selected push users when the average platform ratio is high. Its significance lies in improving monitoring efficiency by narrowing the monitoring scope to selected push users when the impact of user pushes is highly concentrated but monitoring is still required after correction, while ensuring that key users are not overlooked.

[0112] It should be noted that the correction platform ratio is determined by multiplying (1 minus the push impact coefficient of the screened push users) and the monitoring and analysis platform ratio.

[0113] S43 determines the monitoring and analysis method for the collection, cleaning, and processing of data from the monitoring and analysis demand platform by using different push impact coefficients for users with different filtering criteria and the proportion of monitoring and analysis platforms.

[0114] This step further refines user grouping based on the push impact coefficient and the monitoring and analysis platform ratio when the average correction ratio of the platform is low. Its significance lies in achieving targeted and differentiated monitoring grouping by distinguishing between users with monitoring deviations and normal users in scenarios with low correction ratios.

[0115] It is understandable that, based on the push impact coefficient of different filtered users and the proportion of the monitoring and analysis platform, the monitoring and analysis method for the collection, cleaning, and processing of data from the monitoring and analysis demand platform is determined, specifically including: S431 identifies users whose average correction ratio is not above a preset correction ratio threshold as monitoring deviation users. It then determines whether the proportion of monitoring deviation users among the selected users is greater than a preset deviation ratio threshold. If so, the monitoring and analysis method for the data collection and cleaning process of the monitoring and analysis demand platform is determined to be based on a preset ratio. Users associated with the e-commerce platform and existing on the monitoring and analysis demand platform are randomly divided into a group. The data collection and cleaning process of users in a group on the monitoring and analysis demand platform is monitored in real time for each preset time period. The group is switched according to the preset period. If not, proceed to step S432. The term "monitored deviation users" refers to screened push users whose average correction platform ratio does not reach the preset correction ratio threshold. In other words, the push impact of these users is highly concentrated and the monitoring and analysis platform ratio is low. After correction, the degree to which they still need to be fully monitored is not high. The preset deviation ratio threshold is a critical value used to determine whether monitored deviation users account for a high proportion.

[0116] If the proportion of users with monitoring deviations is high among the selected push users, it indicates that the impact of pushes on a large number of users is highly concentrated and the monitoring demand is not prominent. In this case, a more comprehensive monitoring method is needed to rotate the monitoring of users on all platforms with related monitoring and analysis needs, so as to avoid missing important information by only monitoring selected push users.

[0117] This step employs a comprehensive user rotation monitoring method when the proportion of users with monitoring deviations is high. The significance of this method is that when the impact of push notifications on a large number of users is highly concentrated, although the monitoring needs of these users themselves are not prominent, the e-commerce platforms they are associated with as a whole still need to be monitored. Therefore, a comprehensive monitoring method is adopted to ensure that nothing is missed.

[0118] S432, based on the proportion of users with monitoring deviations in the filtered and pushed users and the average proportion of different filtered and pushed users on the correction platform, a monitoring deviation risk value is determined. It is then determined whether the monitoring deviation risk value is greater than a preset risk threshold. If so, the monitoring and analysis method for the data collection and cleaning processing of the monitoring and analysis demand platform is determined to be based on a preset proportion, randomly dividing users on the associated e-commerce platform who exist on the monitoring and analysis demand platform into a group. Each preset time period, the data collection and cleaning processing of users in one group on the monitoring and analysis demand platform is monitored in real time, and the group is switched according to the preset period. If not, the monitoring and analysis method for the data collection and cleaning processing of the monitoring and analysis demand platform is determined to be based on a preset proportion, dividing users on the associated e-commerce platform who exist on the filtered and pushed users on the associated e-commerce platform and users on the associated e-commerce platform who exist on the monitoring and analysis demand platform, and users on the associated e-commerce platform who exist on multiple monitoring and analysis demand platforms, into a group. Each preset time period, the data collection and cleaning processing of users in one group on the monitoring and analysis demand platform is monitored in real time, and the group is switched according to the preset period.

[0119] The monitoring deviation risk value refers to the indicator calculated by combining the proportion of users with monitoring deviations and the average proportion of correction platforms. Let the monitoring deviation risk value = proportion of users with monitoring deviations × 0.5 + (1 - average proportion of correction platforms) × 0.5. The higher the value, the stronger the tendency to adopt a more comprehensive approach between comprehensive monitoring and precise monitoring.

[0120] If the proportion of users with monitoring deviations is not high but the average proportion of correction platforms is high, then the risk value of monitoring deviations is low, indicating that a more accurate monitoring method can be adopted, which can be used to monitor only users who are selected and pushed to the platform and users who are associated with multiple monitoring and analysis platforms, thus narrowing the monitoring scope.

[0121] This step involves selecting a differentiated monitoring method based on the average proportion of corrected platforms when the proportion of users with monitoring deviations is not high. Its significance lies in narrowing the monitoring scope and improving monitoring efficiency through precise monitoring methods in low-deviation-risk scenarios. At the same time, by paying extra attention to users who are associated with multiple monitoring and analysis platforms, it ensures that key platforms are not overlooked.

[0122] This embodiment, through the combination of steps S41 to S432, realizes the entire process from assessing the proportion of the monitoring and analysis platform to determining the monitoring and analysis method for data collection, cleaning, and processing. Its core value lies in four aspects: First, by calculating the average proportion of the monitoring and analysis platform, a quantitative assessment of the concentration of the update impact is achieved, providing the first layer of decision-making basis for the selection of monitoring methods; second, by correcting the calculation of the platform proportion, a comprehensive assessment of the concentration of the push impact and the degree of monitoring demand is achieved, realizing precise quantification of user monitoring priorities; third, by identifying users with monitoring deviations and calculating the risk value of monitoring deviations, refined classification of scenarios is achieved, ensuring that appropriate monitoring methods can be selected under different combinations of concentration levels; fourth, through the progressive design of three monitoring and analysis methods (full user rotation monitoring, filtered push user rotation monitoring, and precise user rotation monitoring), a dynamic balance between monitoring comprehensiveness and monitoring efficiency is achieved.

[0123] Example 2 Secondly, such as Figure 4 As shown, this application provides an automated collection and cleaning system for multi-source heterogeneous data in cross-platform e-commerce, employing the aforementioned automated collection and cleaning method for multi-source heterogeneous data in cross-platform e-commerce, specifically including: Push user identification module, monitoring platform determination module, monitoring and analysis module; The push user identification module is responsible for identifying the users who are following the e-commerce platform for updates. The monitoring platform determination module is responsible for determining the monitoring and analysis needs of the e-commerce platform. The monitoring and analysis module is responsible for determining the monitoring and analysis methods for the data collection, cleaning, and processing of the monitoring and analysis requirements platform.

[0124] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0125] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0126] The above description is merely one or more embodiments of this specification and is not intended to limit this specification. Various modifications and variations can be made to the one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for automated collection and cleaning of multi-source heterogeneous data in cross-platform e-commerce, characterized in that, Specifically, it includes: S1 determines the number of associated e-commerce platforms for different users based on their associated e-commerce platform data, determines the e-commerce platforms to be updated and followed based on the number of associated e-commerce platforms for different users, and determines the users to whom the updated e-commerce platforms to be updated and followed are pushed based on the degree of overlap between the updated and followed e-commerce platforms and the associated e-commerce platforms of different users. S2 determines the e-commerce platforms that different users follow for updates based on the similarity of their update push users. Then, based on the e-commerce platforms that different users follow for updates and the associated e-commerce platforms, S2 determines the monitoring and analysis demand platforms in the e-commerce platforms. S3 uses the e-commerce platforms that the user follows as the update push user as the push association platform, and uses users with push association platforms as the filter push users. Based on the similarity between the monitoring and analysis demand platform and the associated e-commerce platforms of different filter push users, as well as the push association platforms of the filter push users, the monitoring and analysis demand platform determines the monitoring and analysis method for data collection, cleaning and processing.

2. The method for automated collection and cleaning of cross-platform e-commerce multi-source heterogeneous data as described in claim 1, characterized in that, The associated e-commerce platform is the e-commerce platform where the user's products are listed.

3. The method for automated collection and cleaning of cross-platform e-commerce multi-source heterogeneous data as described in claim 1, characterized in that, The method for determining the e-commerce platform to be updated and followed in the aforementioned e-commerce platform is as follows: Users whose number of associated e-commerce platforms exceeds a threshold are identified based on the number of associated e-commerce platforms they belong to. Users whose number of associated e-commerce platforms exceeds the threshold for the number of associated platforms are considered potential users affected by the update. Based on the number of users potentially affected by the update and the number of associated e-commerce platforms for different users, the e-commerce platforms that are concerned about the update are identified.

4. The method for automated collection and cleaning of cross-platform e-commerce multi-source heterogeneous data as described in claim 3, characterized in that, If there are no users among the users who may be affected by the update, then it is determined that there is no e-commerce platform that is being updated.

5. The method for automated collection and cleaning of cross-platform e-commerce multi-source heterogeneous data as described in claim 3, characterized in that, If the proportion of users potentially affected by the update is greater than a preset threshold for the proportion of potentially affected users, then the e-commerce platform that is considered to be an e-commerce platform whose data interface rules have been updated more than a preset number of times in the most recent preset time period, or whose minimum interval between adjacent update times is less than a preset interval time threshold, is determined to be an e-commerce platform.

6. The method for automated collection and cleaning of cross-platform e-commerce multi-source heterogeneous data as described in claim 1, characterized in that, The method for determining users who are interested in receiving updates from the e-commerce platform is as follows: Based on the degree of overlap between the updated e-commerce platform and the associated e-commerce platforms of different users, users who have updated e-commerce platforms are identified among the associated e-commerce platforms, and these users are identified as associated users of the updated e-commerce platforms. The number of associated e-commerce platforms of the associated users is determined based on the associated e-commerce platforms of different associated users; Based on the users associated with the updated e-commerce platforms and the number of e-commerce platforms associated with different users, the users who receive the updated e-commerce platform updates are determined.

7. The method for automated collection and cleaning of cross-platform e-commerce multi-source heterogeneous data as described in claim 6, characterized in that, The update impact value of the e-commerce platform being updated is determined by the average of the correlation coefficient of the updated e-commerce platform and the update impact weight coefficient of different related users. If the update impact value of the updated e-commerce platform is greater than the preset impact threshold, then the update push users of the e-commerce platform being updated are determined to be the related users with the largest number of related e-commerce platforms among the preset number of related users and the related users whose update impact weight coefficient is greater than the preset coefficient threshold (greater than the preset impact weight coefficient threshold).

8. The method for automated collection and cleaning of cross-platform e-commerce multi-source heterogeneous data as described in claim 1, characterized in that, The method for determining the monitoring and analysis method for the data collection and cleaning process of the monitoring and analysis demand platform is as follows: Based on the similarity between the monitoring and analysis demand platform and the associated e-commerce platforms of different screened and pushed users, the proportion of the monitoring and analysis demand platform among the associated e-commerce platforms of screened and pushed users is determined, and the proportion of the monitoring and analysis demand platform among the associated e-commerce platforms of screened and pushed users is taken as the monitoring and analysis platform proportion. The push impact coefficient of the selected push users is determined based on the push association platform of different selected push users; The monitoring and analysis method for the collection, cleaning, and processing of data from the monitoring and analysis platform is determined by using different push impact coefficients for different filtered users and the proportion of monitoring and analysis platforms.

9. The method for automated collection and cleaning of cross-platform e-commerce multi-source heterogeneous data as described in claim 8, characterized in that, Based on the average proportion of different filtered push users on the monitoring and analysis platform, if the average proportion of different filtered push users on the monitoring and analysis platform is less than the preset value of the monitoring and analysis platform proportion, then the monitoring and analysis method for the collection and cleaning processing of the monitoring and analysis demand platform is determined to be based on the preset proportion, randomly dividing users of the associated e-commerce platform on the monitoring and analysis demand platform into a group, and monitoring the data collection and cleaning processing of users in a group on the monitoring and analysis demand platform in real time during each preset time period, and switching the group according to the preset period.

10. A cross-platform e-commerce multi-source heterogeneous data automated collection and cleaning system, employing the cross-platform e-commerce multi-source heterogeneous data automated collection and cleaning method according to any one of claims 1-9, characterized in that, Specifically, it includes: Push user identification module, monitoring platform determination module, monitoring and analysis module; The push user identification module is responsible for identifying the users who are following the e-commerce platform for updates. The monitoring platform determination module is responsible for determining the monitoring and analysis needs of the e-commerce platform. The monitoring and analysis module is responsible for determining the monitoring and analysis methods for the data collection, cleaning, and processing of the monitoring and analysis requirements platform.

Images