Dynamic user profile updating method based on multi-modal data

Abstract

This invention discloses a dynamic user profile update method based on multimodal data, relating to the field of user modeling technology. The method includes: acquiring the collaborative analysis software of each software in the user's device; obtaining the association weight based on the association filtering value of the collaborative analysis software; acquiring status update words and updating the user's profile based on the collaborative analysis software of the updated reference software and the association weight of the collaborative analysis software. This invention addresses the problem in existing dynamic user profile update methods that lack analysis of multimodal data generated by users based on user software usage data. This results in the inability to effectively acquire dynamic profile data related to the user's immediate interests and needs, leading to inaccurate user profiles and a loss of dynamism after data processing.

Description

Technical Field

[0001] This invention relates to the field of user modeling technology, specifically to a method for updating dynamic user profiles based on multimodal data. Background Technology

[0002] Dynamic user profiles refer to user models built based on real-time or recent user behavior data and that can be updated over time. They capture users' real-time changes in interests and needs in different scenarios, such as browsing, searching, and purchasing behavior. Dynamic user profiles emphasize timeliness and behavior-driven features. The core characteristics of dynamic user profiles include dynamism, scenario dependence, and predictive ability.

[0003] Existing methods for updating dynamic user profiles typically involve acquiring data related to the dynamic user profile, processing the data through a central control module, and combining the resulting processing parameters with the system's operating parameters to achieve data processing, identification, and storage of the user profile. While this improved method can quickly process and store user profiles based on relevant data, it lacks analysis of multimodal data generated by users based on their software usage data. This results in an inability to effectively acquire dynamic profile data related to users' immediate interests and needs, leading to inaccurate user profiles and a loss of dynamism after data processing. For example, patent application CN115146155A discloses a dynamic user profile... In management systems, this approach involves establishing multiple modules to process data related to dynamic user profiles, thereby generating and storing user profiles. Other methods for updating dynamic user profiles typically involve building a model, analyzing collected multi-dimensional data, and using the analysis results as the updated dynamic user profile. This improved method still processes already collected data and lacks analysis of multi-modal data generated by users based on their software usage data. This results in an inability to effectively obtain dynamic profile data related to users' immediate interests and needs, leading to inaccurate and outdated user profiles after data processing. Therefore, it is necessary to improve existing methods for updating dynamic user profiles. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in the prior art by proposing a dynamic user profile update method based on multimodal data. This method addresses the lack of analysis on filtering multimodal data generated by users based on user software usage data in existing dynamic user profile update methods. As a result, it is impossible to effectively obtain dynamic profile data related to users' real-time interests and needs, leading to inaccurate user profiles and loss of dynamism after data processing.

[0005] To achieve the above objectives, this application provides a method for updating dynamic user profiles based on multimodal data, comprising the following steps:

[0006] Based on the software used by users, software data analysis methods are used to analyze the records of users' software usage, and based on the analysis results, the collaborative analysis software corresponding to each software in the user's device and the association filtering value of each collaborative analysis software are obtained.

[0007] Based on the association screening values ​​of the collaborative analysis software corresponding to all software, obtain the association ratio between each software and each collaborative analysis software;

[0008] When updating a user's dynamic profile, the system obtains the user's updated reference software based on the latest acquired user software usage records; and obtains status update keywords based on the collaborative analysis software of the updated reference software and the association weight of the collaborative analysis software.

[0009] Update user profiles based on status update terms.

[0010] Furthermore, based on the software used by users, software data analysis methods are used to analyze user records of software usage, including:

[0011] The device that acquires and records dynamic user profiles is designated as the profile analysis device; all software within the profile analysis device is designated as software to be analyzed; the usage records of all software to be analyzed within the profile analysis device are acquired, and all usage records are sequentially recorded from earliest to latest on a daily basis as software usage record RJ1 to software usage record RJ. t Where t is the number of days the user uses the profile analysis device;

[0012] The software data analysis method was used to analyze the usage records of each software in turn, and based on the analysis results, the corresponding collaborative analysis software for each software to be analyzed and the association screening value of each collaborative analysis software were determined.

[0013] Furthermore, software data analysis methods include:

[0014] For any software usage record: establish a Cartesian coordinate system, denoted as the software daily usage analysis coordinate system, where the units of the X-axis and Y-axis of the software daily usage analysis coordinate system are h and min, respectively;

[0015] For any two adjacent coordinate points f and g in the software daily usage analysis coordinate system with abscissa from 0h to 24h: the software used by the user between time f and g in the software usage record is recorded as the software to be marked, and the number of software to be marked is recorded as r; the coordinate points f and g are divided into r regions on average, and the midpoint of each of the r regions is recorded as the software abscissa point, where f and g are both positive integers less than or equal to 24 and greater than or equal to 0, and g is f+1.

[0016] Furthermore, software data analysis methods also include:

[0017] For any software to be labeled: record the time the software is used by the user between f and g as the software ordinate point; obtain the software ordinate points of all software to be labeled, and based on the values ​​of the software ordinate points in ascending order, record all software to be labeled sequentially as software to be labeled DB1 to software to be labeled DB1. r ;

[0018] For the software to be labeled, DB1 to DB r Any of the software DBs to be tagged u Within the software daily analysis coordinate system, obtain the x-coordinate of the u-th software ax-point from left to right among all software ax-points, representing the software DB to be labeled. u The software's ordinate point is the point on the vertical axis, and is denoted as the software DB to be labeled. u punctuation marks;

[0019] Obtain the interval punctuation of all software to be labeled between f and g.

[0020] Furthermore, software data analysis methods also include:

[0021] In the software daily analysis coordinate system, obtain the interval punctuation points of all adjacent coordinate points corresponding to all software to be labeled in the range of horizontal coordinates from 0h to 24h;

[0022] For any software α to be analyzed and labeled: obtain the curve obtained by fitting all interval points of the software α to be analyzed in the daily use analysis coordinate system, and record it as the daily use curve of the software α to be analyzed; record the points with the largest vertical coordinate and the largest absolute value of the slope in the daily use curve as the daily use peak point and the daily use frequency conversion point, respectively.

[0023] The horizontal axis of the peak daily usage point is marked as X1, and the nearest integer point to the left and right of X1 within the horizontal axis is marked as P1 and P2 respectively; among the software to be labeled between P1 and P2, all software to be labeled except for the software to be analyzed α are marked as software to be screened.

[0024] For any software to be filtered: when there is software to be filtered among all the software to be labeled between (P1-1) and P1, and there is software to be filtered among all the software to be labeled between P2 and (P2+1), the software to be filtered is recorded as the software to be associated, and the sum of the ordinates of the intervals of the software to be associated in the intervals [(P1-1), P1], [P1, P2] and [P2, (P2+1)] is recorded as the association filtering value of the software to be associated;

[0025] Retrieve all software to be associated and the association filtering value for each software from all software to be filtered.

[0026] Furthermore, software data analysis methods also include:

[0027] The horizontal coordinate of the daily frequency conversion point is marked as X2, and the nearest integer point to the left and the nearest integer point to the right of X2 in the horizontal axis are marked as Q1 and Q2 respectively; among the software to be labeled between Q1 and Q2, all software to be labeled except for the software to be analyzed α are marked as gradient software.

[0028] When the slope of the daily frequency conversion point is positive, for any software to be gradiented, if the software to be gradiented satisfies any one of conditions A1, A2 and A3, the software to be gradiented is recorded as shared software.

[0029] When the slope of the daily frequency conversion point is negative, for any software to be gradiented, if the software to be gradiented satisfies any one of conditions B1, B2 and B3, the software to be gradiented is recorded as shared software.

[0030] When any software β to be analyzed is simultaneously denoted as software to be associated and shared software, the software β to be analyzed is denoted as the collaborative analysis software of the software α to be analyzed.

[0031] Furthermore, software data analysis methods also include:

[0032] Condition A1 is: the ordinate of the interval marker in the interval [(Q1-1), Q1] of the gradient software is less than or equal to the ordinate of the interval marker in the interval [Q1, Q2].

[0033] Condition A2 is: the ordinate of the interval marker in the interval [Q1,Q2] of the gradient software is less than or equal to the ordinate of the interval marker in the interval [Q2, (Q2+1)].

[0034] Condition A3 is: the ordinate of the interval marker in the interval [(Q1-1), Q1] of the gradient software is less than or equal to the ordinate of the interval marker in the interval [(Q1-1), (Q2+1)].

[0035] Condition B1 is: the ordinate of the interval marker in the interval [(Q1-1), Q1] of the gradient software is greater than or equal to the ordinate of the interval marker in the interval [Q1, Q2].

[0036] Condition B2 is: the ordinate of the interval marker in the interval [Q1,Q2] of the gradient software is greater than or equal to the ordinate of the interval marker in the interval [Q2, (Q2+1)].

[0037] Condition B3 is: the ordinate of the interval marker in the interval [(Q1-1), Q1] of the gradient software is greater than or equal to the ordinate of the interval marker in the interval [(Q1-1), (Q2+1)].

[0038] Furthermore, based on the association screening values ​​of the collaborative analysis software corresponding to all software, the association weight between each software and each collaborative analysis software is obtained, including:

[0039] Based on all software usage records, obtain all collaborative analysis software for the software to be analyzed, α; for any collaborative analysis software of the software to be analyzed, α, the sum of all associated filter values ​​corresponding to the collaborative analysis software and the software to be analyzed, α, is recorded as the total filter value of the collaborative analysis software;

[0040] The sum of the total screening values ​​of all collaborative analysis software of the software to be analyzed is recorded as the total screening value; for any collaborative analysis software, the total screening value of the collaborative analysis software divided by the total screening value is recorded as the correlation ratio between the software to be analyzed α and the collaborative analysis software, where the correlation ratio is a value greater than or equal to 0 and less than or equal to 1;

[0041] Obtain the collaborative analysis software for all software to be analyzed, as well as the correlation ratio between each software to be analyzed and all its corresponding collaborative analysis software.

[0042] Furthermore, when updating a user's dynamic profile, based on the latest acquired user's software usage records, the user's update reference software includes:

[0043] When updating a user's dynamic profile, the update time is recorded as the current update time, and the time when the user's dynamic profile was last updated is recorded as the previous update time; the user's software usage records between the previous update time and the current update time are obtained and recorded as update reference records;

[0044] The software that contains collaborative analysis software among all the software used by the user in the update reference record is recorded as the update reference software.

[0045] Furthermore, based on the updated reference software and the association weight of the collaborative analysis software, status update words are obtained; the user profile is updated based on the status update words, including:

[0046] For any update reference software: obtain all the text input, clicked images, and played videos of the user in the update reference software and all collaborative analysis software of the update reference software in the update reference record, and extract keywords from the input text, clicked images, and played videos using pre-trained language models, image description generation models, and video text description models respectively.

[0047] For updated reference software: the keywords obtained from the updated reference software shall be recorded as the main keywords;

[0048] For any keyword obtained by any collaborative analysis software from the updated reference software: multiply the number of times the keyword appears in all the input text, clicked images, and played videos corresponding to the collaborative analysis software by the value of the association ratio between the updated reference software and the collaborative analysis software, and record it as the keyword's weight count.

[0049] Obtain the weight and frequency of all keywords from all collaborative analysis software, and record the keyword that is recorded as the main keyword and has the highest weight and frequency as the status update keyword for the updated reference software;

[0050] Obtain the status update terms of all updated reference software and use these status update terms to populate the user profile.

[0051] The beneficial effects of this invention are as follows: This application first analyzes the records of user software usage using software data analysis methods, and based on the analysis results, obtains the collaborative analysis software corresponding to each software on the user's device and the association filtering value of each collaborative analysis software. The advantage of this is that by analyzing the records of user software usage, it is possible to identify the software that has a strong correlation with the user's device usage, i.e., the collaborative analysis software corresponding to each software. Furthermore, by obtaining the association filtering value, it is possible to evaluate the degree of correlation between each software and the collaborative analysis software, so as to obtain the correlation weight in the future, thereby enabling more accurate data analysis when updating the user profile in real time.

[0052] This application also obtains the association weight between each software and each collaborative analysis software based on the association screening value; when updating the user's dynamic profile, it obtains the user's update reference software based on the latest obtained user's software usage records; based on the collaborative analysis software of the update reference software and the association weight of the collaborative analysis software, it obtains the status update words; finally, it updates the user profile based on the status update words. The advantage of this is that by obtaining the association weight between each software and each collaborative analysis software, the user's dynamic profile can be updated, and when obtaining the status update words through the association weight, it ensures that the things corresponding to the status update words account for the largest proportion in the user's immediate interests and needs, thereby ensuring that the user profile is accurate enough and its dynamism is effective after updating the user profile based on the status update words. Attached Figure Description

[0053] Figure 1 This is a flowchart of the steps of the method of the present invention;

[0054] Figure 2 This is a schematic diagram illustrating the acquisition of the interval punctuation marks according to the present invention;

[0055] Figure 3 This is a schematic diagram of the daily use curve of the present invention;

[0056] Figure 4 This is a schematic diagram of the electronic device of the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Example 1, please refer to Figure 1 As shown, this application provides a method for updating dynamic user profiles based on multimodal data, including the following steps:

[0059] Step S1: Based on the software used by the user, the records of the user's software use are analyzed using software data analysis method, and the collaborative analysis software corresponding to each software in the user's device and the association filtering value of each collaborative analysis software are obtained based on the analysis results.

[0060] Step S1 includes: Step S101, acquiring the device that records dynamic user profiles, denoted as the profile analysis device; denoting all software within the profile analysis device as software to be analyzed; acquiring the usage records of all software to be analyzed within the profile analysis device, and sequentially recording all usage records from first to last on a daily basis as software usage record RJ1 to software usage record RJ. t Where t is the number of days the user uses the profile analysis device;

[0061] In practical implementation, for example, if the device used to record dynamic user profiles is a tablet, then the tablet can be designated as the profile analysis device, and all software on the tablet can be designated as software to be analyzed. If the user uses the tablet for 30 days, then starting from the first day of the user's tablet use, the user's daily tablet usage records can be sequentially recorded as Software Usage Record RJ1 to Software Usage Record RJ. 30 ;

[0062] Step S102: Use software data analysis to analyze the usage records of each software in sequence, and based on the analysis results, analyze the collaborative analysis software corresponding to each software to be analyzed and the association screening value of each collaborative analysis software.

[0063] Step S103, the software data analysis method includes: Step S1031, for any software usage record: establish a plane rectangular coordinate system, denoted as the software daily usage analysis coordinate system, wherein the units of the X-axis and Y-axis of the software daily usage analysis coordinate system are h and min, respectively;

[0064] Step S1034: For any two adjacent coordinate points f and g in the software daily usage analysis coordinate system with abscissas from 0h to 24h: the software used by the user between time f and g in the software usage record is recorded as the software to be marked, and the number of software to be marked is recorded as r; the coordinate points f and g are divided into r regions on average, and the midpoint of each of the r regions is recorded as the software abscissa point, where f and g are both positive integers less than or equal to 24 and greater than or equal to 0, and g is f+1;

[0065] In specific implementation, such as in the data analysis of this embodiment, for a software usage record with a time between 12h and 13h, the values ​​of f and g can be set to 12 and 13 respectively during analysis, and all software used by the user between 12h and 13h in the software usage record can be recorded as software to be labeled; through data acquisition, all software to be labeled are: Taobao, Ele.me, Meituan, and WeChat; please refer to Figure 2 As shown, the value of r can be denoted as 4, and the area between coordinate points 12 and 13 can be divided into 4 equal regions. The midpoint of each of the 4 regions can be marked as a software abbreviated point. Figure 2RB1 to RB4 in the data; through data acquisition, it was found that users used Taobao, Ele.me, Meituan, and WeChat for 11 minutes, 9 minutes, 10 minutes, and 30 minutes respectively between 12 and 13 hours. Therefore, the software to be labeled DB1 to DB4 can be recorded as Ele.me, Meituan, Taobao, and WeChat respectively. In the software daily usage analysis coordinate system, the points (RB1, 9), (RB2, 10), (RB3, 11), and (RB4, 30) are recorded as the corresponding interval markers of Ele.me, Meituan, Taobao, and WeChat respectively.

[0066] Step S1035: For any software to be labeled: record the time the software to be labeled is used by the user between f and g as the software ordinate point; obtain the software ordinate points of all software to be labeled, and based on the values ​​of the software ordinate points from smallest to largest, record all software to be labeled sequentially as software to be labeled DB1 to software to be labeled DB1. r ;

[0067] Step S1036, for the software to be labeled DB1 to the software to be labeled DB r Any of the software DBs to be tagged u Within the software daily analysis coordinate system, obtain the x-coordinate of the u-th software ax-point from left to right among all software ax-points, representing the software DB to be labeled. u The software's ordinate point is the point on the vertical axis, and is denoted as the software DB to be labeled. u punctuation marks;

[0068] Step S1037: Obtain the interval punctuation of all software to be labeled between f and g.

[0069] The software data analysis method also includes: step S1038, obtaining the interval punctuation points of all adjacent coordinate points of all software to be marked in the software daily analysis coordinate system with a horizontal coordinate of 0h to 24h;

[0070] Step S1039: For any software α to be analyzed that is marked as software to be labeled: obtain the curve obtained by fitting all interval points of software α to be analyzed in the daily use analysis coordinate system, and record it as the daily use curve of software α; record the points with the largest vertical coordinate and the largest absolute value of the slope in the daily use curve as the daily use peak point and the daily use frequency conversion point, respectively.

[0071] In the specific implementation process, by acquiring all the interval markers corresponding to the software to be labeled, and obtaining the corresponding daily usage curve based on all the interval markers, it is possible to analyze the user's preferences for using the software, for example... Figure 3The daily usage curve RQ obtained from the data shows that the points with the largest vertical axis and the largest absolute value of the slope within the daily usage curve RQ are BD and XD, respectively. This indicates that users spend the most time using WeChat between 12:00 and 13:00 in a 24-hour period, and the frequency of WeChat usage changes the fastest between 15:00 and 16:00. Therefore, points BD and XD can be recorded as the daily peak point and daily frequency variation point of WeChat usage, respectively.

[0072] In this embodiment, by using the time period with the longest WeChat usage time and the fastest change in WeChat usage frequency as the benchmark for subsequent analysis, it is possible to identify the software used by the user that is highly associated with WeChat, namely, WeChat's collaborative analysis software. The purpose of obtaining WeChat's collaborative analysis software is that if user usage data related to WeChat is analyzed, the data corresponding to WeChat's collaborative analysis software can be used to effectively extract WeChat-related data, thereby obtaining data that can accurately and effectively update the user's dynamic profile, i.e., obtaining status update words.

[0073] Step S1040: Mark the horizontal coordinate of the daily peak point as X1, and mark the nearest integer point to the left and the nearest integer point to the right of X1 on the horizontal axis as P1 and P2 respectively; mark all the software to be labeled between P1 and P2, except for the software to be analyzed α, as the software to be screened.

[0074] Step S1041: For any software to be filtered: when there is software to be filtered among all the software to be labeled between (P1-1) and P1, and there is software to be filtered among all the software to be labeled between P2 and (P2+1), the software to be filtered is recorded as the software to be associated, and the sum of the ordinates of the intervals of the software to be associated in the intervals [(P1-1), P1], [P1, P2] and [P2, (P2+1)] is recorded as the association filtering value of the software to be associated.

[0075] In the specific implementation process, such as in the analysis above, Figure 3The nearest whole number to the left and right of the peak daily usage point for WeChat is 12h and 13h, respectively. The vertical coordinate of the interval marker corresponding to the user's WeChat usage between 12h and 13h is 30min. Therefore, P1 and P2 can be denoted as 12 and 13, respectively. If there is a software to be filtered, "Taobao," which is the software to be labeled in the interval [11, 12] and also in the interval [13, 14], it indicates that the user used "Taobao" during the longest period of WeChat usage. This means that "Taobao" may contain similar usage data to WeChat, so "Taobao" can be used as a software to be associated for further filtering. Data shows that the vertical coordinates of the interval markers for Taobao in the intervals [11, 12] and [13, 14] are 10min and 9min, respectively. Furthermore, the above analysis shows that the vertical coordinate of the interval marker corresponding to Taobao in the interval [12, 13] is 11min. Therefore, the association filtering value for Taobao is 30min.

[0076] Step S1042: Obtain all software to be associated and the association filtering value for each software from all software to be filtered.

[0077] The software data analysis method also includes: step S1043, marking the horizontal coordinate of the daily frequency conversion point as X2, and marking the nearest integer point to the left and the nearest integer point to the right of X2 on the horizontal axis as Q1 and Q2 respectively; marking all the software to be labeled between Q1 and Q2, except for the software to be analyzed α, as the software to be labeled.

[0078] Step S1044: When the slope of the daily frequency conversion point is positive, for any software to be gradiented, if the software to be gradiented satisfies any one of conditions A1, A2 and A3, the software to be gradiented is recorded as shared software.

[0079] Step S1045: When the slope of the daily frequency conversion point is negative, for any software to be gradiented, if the software to be gradiented satisfies any one of conditions B1, B2 and B3, the software to be gradiented is recorded as shared software.

[0080] In practice, the selection criteria for shared software are as follows: if the usage duration of the software to be graded and the software to be analyzed show the same trend, then the software to be graded can be considered as shared software of the software to be analyzed; for example, for... Figure 3When analyzing the daily usage frequency of WeChat, the nearest whole number to the left and right of the daily usage frequency is 15h and 16h, respectively. Therefore, Q1 and Q2 can be recorded as 15 and 16, respectively. Through data acquisition, it is found that among the software to be labeled between 15h and 16h, the software "Taobao" exists besides WeChat. Since the slope of the daily usage frequency of WeChat is less than 0, if "Taobao" is to be recorded as shared software, "Taobao" should satisfy any one of conditions B1, B2, and B3. Through data analysis, it is found that the ordinate of "Taobao" in the interval [14,15] is greater than or equal to the ordinate of the software to be labeled in the interval [15,16]. This indicates that when the frequency of users using WeChat decreases, the frequency of using "Taobao" also decreases. That is, the trend of the usage time of WeChat and "Taobao" is the same in the period when the frequency of WeChat usage changes the fastest. Therefore, "Taobao" can be used as shared software for subsequent screening.

[0081] Step S1046: When any software β to be analyzed is simultaneously recorded as software to be associated and shared software, the software β to be analyzed is recorded as the collaborative analysis software of the software α to be analyzed.

[0082] In the data analysis of this embodiment, if the same software is simultaneously recorded as both a software to be associated and a shared software, it indicates that when the software to be analyzed is used, the software may have similar usage data to the software to be analyzed, and the software and the software have the same usage frequency during the period when the usage frequency of the software to be analyzed changes the most. That is, the software and the software to be analyzed have a strong correlation, and the software can be recorded as a collaborative analysis software of the software to be analyzed, and in subsequent data analysis, it can provide data related to dynamic user profile updates. Through the above analysis, it can be seen that Taobao is simultaneously recorded as a shared software and a software to be associated with WeChat, therefore Taobao is a collaborative analysis software of WeChat.

[0083] Condition A1 is: the ordinate of the interval marker in the interval [(Q1-1), Q1] of the gradient software is less than or equal to the ordinate of the interval marker in the interval [Q1, Q2].

[0084] Condition A2 is: the ordinate of the interval marker in the interval [Q1,Q2] of the gradient software is less than or equal to the ordinate of the interval marker in the interval [Q2, (Q2+1)].

[0085] Condition A3 is: the ordinate of the interval marker in the interval [(Q1-1), Q1] of the gradient software is less than or equal to the ordinate of the interval marker in the interval [(Q1-1), (Q2+1)].

[0086] Condition B1 is: the ordinate of the interval marker in the interval [(Q1-1), Q1] of the gradient software is greater than or equal to the ordinate of the interval marker in the interval [Q1, Q2].

[0087] Condition B2 is: the ordinate of the interval marker in the interval [Q1,Q2] of the gradient software is greater than or equal to the ordinate of the interval marker in the interval [Q2, (Q2+1)].

[0088] Condition B3 is: the ordinate of the interval marker in the interval [(Q1-1), Q1] of the gradient software is greater than or equal to the ordinate of the interval marker in the interval [(Q1-1), (Q2+1)].

[0089] Step S2: Based on the association screening values ​​of the collaborative analysis software corresponding to all software, obtain the association ratio between each software and each collaborative analysis software;

[0090] Step S2 includes: Step S201, obtaining all collaborative analysis software of the software to be analyzed α based on all software usage records; for any collaborative analysis software of the software to be analyzed α, the sum of all associated filter values ​​corresponding to the collaborative analysis software and the software to be analyzed α is recorded as the total filter value of the collaborative analysis software;

[0091] In the specific implementation process, the same collaborative analysis software of the software to be analyzed can obtain the corresponding associated filter values ​​in multiple software usage records. That is, "Taobao" in the above analysis can be recorded as the collaborative analysis software of WeChat in multiple software usage records, and the corresponding associated filter values ​​can be obtained respectively. The sum of all the obtained associated filter values ​​is recorded as the total filter value of Taobao.

[0092] Step S202: The sum of the total screening values ​​of all collaborative analysis software of the software to be analyzed is recorded as the total screening value; for any collaborative analysis software, the total screening value of the collaborative analysis software divided by the total screening value is recorded as the correlation ratio between the software to be analyzed α and the collaborative analysis software, wherein the correlation ratio is a value greater than or equal to 0 and less than or equal to 1.

[0093] In the specific implementation process, for example, through the above analysis, the total screening value of all collaborative analysis software of WeChat is 800min, and when Taobao is counted as a collaborative analysis software of WeChat, the corresponding total screening value is 200min. Then, it can be calculated that the association ratio between WeChat and Taobao is 0.25. By obtaining the association ratio, we can obtain the association strength between each software and WeChat among the software that has a strong association with WeChat. The larger the association ratio, the higher the association strength with WeChat.

[0094] Step S203: Obtain the collaborative analysis software for all software to be analyzed, as well as the correlation ratio between each software to be analyzed and all its corresponding collaborative analysis software.

[0095] Step S3: When updating the user's dynamic profile, obtain the user's updated reference software based on the latest obtained user's software usage records; obtain the status update words based on the collaborative analysis software of the updated reference software and the association weight of the collaborative analysis software.

[0096] Update user profiles based on status update keywords;

[0097] Step S3 includes: Step S301, when updating the user's dynamic profile, the update time node is recorded as the current update time, and the time when the user's dynamic profile was last updated is recorded as the previous update time; the user's software usage records between the previous update time and the current update time are obtained and recorded as update reference records;

[0098] Step S302: Record all software in the user's software that contains collaborative analysis software as updated reference software in the update reference record.

[0099] Step S3 also includes: Step S303, for any update reference software: obtain all the text input, clicked images and played videos of the user in the update reference software and all collaborative analysis software of the update reference software in the update reference record, and extract keywords from the input text, clicked images and played videos using a pre-trained language model, an image description generation model and a video text description model respectively.

[0100] Step S304, for the updated reference software: record the keywords obtained from the updated reference software as the main keywords;

[0101] Step S305: For any keyword obtained by any collaborative analysis software of the updated reference software: multiply the number of times the keyword appears in all the input text, clicked images and played videos corresponding to the collaborative analysis software by the value of the association ratio between the updated reference software and the collaborative analysis software, and record it as the keyword's weight count.

[0102] In the specific implementation process, for example, in one data analysis, the updated reference software was WeChat, and the keywords obtained from WeChat included: laptop, metallic silver, cost-effectiveness, and reviews; since Taobao is a collaborative analysis software of WeChat, the keywords obtained by obtaining the corresponding keywords from Taobao included: laptop, ultrabook, metallic silver, and desktop computer, with corresponding occurrences of 20, 10, 15, and 10 times respectively; since the association ratio between WeChat and Taobao in the above analysis was 0.25, the corresponding occurrences of laptop, ultrabook, metallic silver, and desktop computer were 5, 2, and 5 times respectively. 5 times, 3.75 times, and 2.5 times; By analyzing other collaborative analysis software on WeChat, the keywords recorded as primary keywords and with the highest proportion on Taobao are "notebook" and "metallic silver". Therefore, "notebook" and "metallic silver" can be used as status update words on WeChat. When using status update words to fill in user profiles, the profiles can be filled in with the objects corresponding to the status update words. For example, when the status update words are "notebook" and "metallic silver", "buy a notebook" can be added to the intent of the user profile, and "metallic silver" can be added to the preferences of the user profile.

[0103] Step S306: Obtain the proportion and frequency of all keywords in all collaborative analysis software, and record the keyword that is recorded as the main keyword and has the highest proportion and frequency as the status update word of the update reference software.

[0104] Step S307: Obtain the status update words of all updated reference software and use the status update words to populate the user profile.

[0105] Example 2, please refer to Figure 4 As shown, Figure 4This example illustrates the structure of an electronic device, which may include a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other via the communication bus. The memory stores computer-readable instructions, and the processor can call these instructions. When the processor executes a computer-readable instruction, it performs steps similar to those in a dynamic user profile update method based on multimodal data to achieve the following functions: First, based on the software used by the user, software data analysis is used to analyze the user's software usage records. Based on the analysis results, the collaborative analysis software corresponding to each software used by the user and the association filtering value of each collaborative analysis software are obtained. Then, based on the association filtering values ​​of all software corresponding to the collaborative analysis software, the association weight between each software and each collaborative analysis software is obtained. When updating the user's dynamic profile, based on the latest obtained user software usage records, the user's update reference software is obtained. Based on the collaborative analysis software of the update reference software and the association weight of the collaborative analysis software, status update words are obtained. Finally, the user's profile is updated based on the status update words.

[0106] Furthermore, when the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0107] Example 3: This application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute a dynamic user profile update method based on multimodal data provided by the above methods. The method includes: first, based on the software used by the user, analyzing the user's software usage records using software data analysis, and obtaining the collaborative analysis software corresponding to each software in the user's device and the association filtering value of each collaborative analysis software based on the analysis results; then, based on the association filtering values ​​of the collaborative analysis software corresponding to all software, obtaining the association weight between each software and each collaborative analysis software; when updating the user's dynamic profile, obtaining the user's update reference software based on the latest obtained user software usage records; obtaining status update words based on the collaborative analysis software of the update reference software and the association weight of the collaborative analysis software; and finally, updating the user's profile based on the status update words.

[0108] Example 4: This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it performs the steps of the above-described dynamic user profile update method based on multimodal data to achieve the following functions: First, based on the software used by the user, the software data analysis method is used to analyze the records of the user's software usage, and based on the analysis results, the collaborative analysis software corresponding to each software in the user's device and the association filtering value of each collaborative analysis software are obtained; then, based on the association filtering values ​​of the collaborative analysis software corresponding to all software, the association weight between each software and each collaborative analysis software is obtained; when updating the user's dynamic profile, based on the latest obtained user's software usage records, the user's update reference software is obtained; based on the collaborative analysis software of the update reference software and the association weight of the collaborative analysis software, the status update words are obtained; finally, the user's profile is updated based on the status update words.

[0109] Based on the above description of the embodiments, the embodiments of the present invention can be provided as methods, systems, or computer program products. Based on this understanding, the above technical solutions, in essence or in terms of their contribution to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or certain parts of the embodiments.

[0110] In the embodiments provided in this application, it should be understood that the disclosed system or method can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules or units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces. The indirect coupling or communication connection between systems, modules, and units may be electrical, mechanical, or other forms.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for updating dynamic user profiles based on multimodal data, characterized in that, Includes the following steps: Based on the software used by users, software data analysis methods are used to analyze the records of users' software usage, and based on the analysis results, the collaborative analysis software corresponding to each software in the user's device and the association filtering value of each collaborative analysis software are obtained. Based on the association screening values ​​of the collaborative analysis software corresponding to all software, obtain the association ratio between each software and each collaborative analysis software; When updating the user's dynamic profile, the user's latest software usage records are used to obtain the user's updated reference software; Based on the updated reference software and the association weight of the collaborative analysis software, the status update words are obtained. Update user profiles based on status update terms.

2. The dynamic user profile update method based on multimodal data according to claim 1, characterized in that, Based on the software used by users, software data analysis methods are used to analyze user records of software usage, including: The device that acquires and records dynamic user profiles is denoted as the profile analysis device; all software within the profile analysis device is denoted as software to be analyzed; the usage records of all software to be analyzed within the profile analysis device are acquired, and all usage records are recorded sequentially from first to last in days as software usage record RJ1 to software usage record RJt, where t is the number of days the user uses the profile analysis device; The software data analysis method was used to analyze the usage records of each software in turn, and based on the analysis results, the corresponding collaborative analysis software for each software to be analyzed and the association screening value of each collaborative analysis software were determined.

3. The dynamic user profile update method based on multimodal data according to claim 2, characterized in that, Software data analysis methods include: For any software usage record: establish a Cartesian coordinate system, denoted as the software daily usage analysis coordinate system, where the units of the X-axis and Y-axis of the software daily usage analysis coordinate system are h and min, respectively; For any two adjacent coordinate points f and g in the software daily usage analysis coordinate system with abscissa from 0h to 24h: the software used by the user between time f and g in the software usage record is recorded as the software to be marked, and the number of software to be marked is recorded as r; the coordinate points f and g are divided into r regions on average, and the midpoint of each of the r regions is recorded as the software abscissa point, where f and g are both positive integers less than or equal to 24 and greater than or equal to 0, and g is f+1.

4. The dynamic user profile update method based on multimodal data according to claim 3, characterized in that, Software data analysis methods also include: For any software to be labeled: record the time the software is used by the user between f and g as the software ordinate point; obtain the software ordinate points of all software to be labeled, and based on the values ​​of the software ordinate points in ascending order, record all software to be labeled sequentially as software to be labeled DB1 to software to be labeled DB1. r ; For the software to be labeled, DB1 to DB r Any of the software DBs to be tagged u Within the software daily analysis coordinate system, obtain the x-coordinate of the u-th software ax-point from left to right among all software ax-points, representing the software DB to be labeled. u The software's ordinate point is the point on the vertical axis, and is denoted as the software DB to be labeled. u punctuation marks; Obtain the interval punctuation of all software to be labeled between f and g.

5. The dynamic user profile update method based on multimodal data according to claim 4, characterized in that, Software data analysis methods also include: In the software daily analysis coordinate system, obtain the interval punctuation points of all adjacent coordinate points corresponding to all software to be labeled in the range of horizontal coordinates from 0h to 24h; For any software α to be analyzed and labeled: obtain the curve obtained by fitting all interval points of the software α to be analyzed in the daily use analysis coordinate system, and record it as the daily use curve of the software α to be analyzed; record the points with the largest vertical coordinate and the largest absolute value of the slope in the daily use curve as the daily use peak point and the daily use frequency conversion point, respectively. The horizontal axis of the peak daily usage point is marked as X1, and the nearest integer point to the left and right of X1 within the horizontal axis is marked as P1 and P2 respectively; among the software to be labeled between P1 and P2, all software to be labeled except for the software to be analyzed α are marked as software to be screened. For any software to be filtered: when there is software to be filtered among all the software to be labeled between (P1-1) and P1, and there is software to be filtered among all the software to be labeled between P2 and (P2+1), the software to be filtered is recorded as the software to be associated, and the sum of the ordinates of the intervals of the software to be associated in the intervals [(P1-1), P1], [P1, P2] and [P2, (P2+1)] is recorded as the association filtering value of the software to be associated; Retrieve all software to be associated and the association filtering value for each software from all software to be filtered.

6. The dynamic user profile update method based on multimodal data according to claim 5, characterized in that, Software data analysis methods also include: The horizontal coordinate of the daily frequency conversion point is marked as X2, and the nearest integer point to the left and the nearest integer point to the right of X2 in the horizontal axis are marked as Q1 and Q2 respectively; among the software to be labeled between Q1 and Q2, all software to be labeled except for the software to be analyzed α are marked as gradient software. When the slope of the daily frequency conversion point is positive, for any software to be gradiented, if the software to be gradiented satisfies any one of conditions A1, A2 and A3, the software to be gradiented is recorded as shared software. When the slope of the daily frequency conversion point is negative, for any software to be gradiented, if the software to be gradiented satisfies any one of conditions B1, B2 and B3, the software to be gradiented is recorded as shared software. When any software β to be analyzed is simultaneously denoted as software to be associated and shared software, the software β to be analyzed is denoted as the collaborative analysis software of the software α to be analyzed.

7. The dynamic user profile update method based on multimodal data according to claim 6, characterized in that, Software data analysis methods also include: Condition A1 is: the ordinate of the interval marker in the interval [(Q1-1), Q1] of the gradient software is less than or equal to the ordinate of the interval marker in the interval [Q1, Q2]. Condition A2 is: the ordinate of the interval marker in the interval [Q1,Q2] of the gradient software is less than or equal to the ordinate of the interval marker in the interval [Q2, (Q2+1)]. Condition A3 is: the ordinate of the interval marker in the interval [(Q1-1), Q1] of the gradient software is less than or equal to the ordinate of the interval marker in the interval [(Q1-1), (Q2+1)]. Condition B1 is: the ordinate of the interval marker in the interval [(Q1-1), Q1] of the gradient software is greater than or equal to the ordinate of the interval marker in the interval [Q1, Q2]. Condition B2 is: the ordinate of the interval marker in the interval [Q1,Q2] of the gradient software is greater than or equal to the ordinate of the interval marker in the interval [Q2, (Q2+1)]. Condition B3 is: the ordinate of the interval marker in the interval [(Q1-1), Q1] of the gradient software is greater than or equal to the ordinate of the interval marker in the interval [(Q1-1), (Q2+1)].

8. The dynamic user profile update method based on multimodal data according to claim 7, characterized in that, Based on the association screening values ​​of all software corresponding to the collaborative analysis software, the association weight between each software and each collaborative analysis software is obtained, including: Based on all software usage records, obtain all collaborative analysis software for the software to be analyzed, α; for any collaborative analysis software of the software to be analyzed, α, the sum of all associated filter values ​​corresponding to the collaborative analysis software and the software to be analyzed, α, is recorded as the total filter value of the collaborative analysis software; The sum of the total screening values ​​of all collaborative analysis software of the software to be analyzed is recorded as the total screening value; for any collaborative analysis software, the total screening value of the collaborative analysis software divided by the total screening value is recorded as the correlation ratio between the software to be analyzed α and the collaborative analysis software, where the correlation ratio is a value greater than or equal to 0 and less than or equal to 1; Obtain the collaborative analysis software for all software to be analyzed, as well as the correlation ratio between each software to be analyzed and all its corresponding collaborative analysis software.

9. The dynamic user profile update method based on multimodal data according to claim 8, characterized in that, When updating a user's dynamic profile, based on the latest obtained software usage records, the user's update reference software includes: When updating a user's dynamic profile, the update time is recorded as the current update time, and the time when the user's dynamic profile was last updated is recorded as the previous update time; the user's software usage records between the previous update time and the current update time are obtained and recorded as update reference records; The software that contains collaborative analysis software among all the software used by the user in the update reference record is recorded as the update reference software.

10. The dynamic user profile update method based on multimodal data according to claim 9, characterized in that, Based on the updated reference software and the association weight of the collaborative analysis software, the status update words are obtained. Updating user profiles based on state update terms includes: For any update reference software: obtain all the text input, clicked images, and played videos of the user in the update reference software and all collaborative analysis software of the update reference software in the update reference record, and extract keywords from the input text, clicked images, and played videos using pre-trained language models, image description generation models, and video text description models respectively. For updated reference software: the keywords obtained from the updated reference software shall be recorded as the main keywords; For any keyword obtained by any collaborative analysis software from the updated reference software: multiply the number of times the keyword appears in all the input text, clicked images, and played videos corresponding to the collaborative analysis software by the value of the association ratio between the updated reference software and the collaborative analysis software, and record it as the keyword's weight count. Obtain the weight and frequency of all keywords from all collaborative analysis software, and record the keyword that is recorded as the main keyword and has the highest weight and frequency as the status update keyword for the updated reference software; Obtain the status update terms of all updated reference software and use these status update terms to populate the user profile.

Images