Multi-modal data fusion user portrait construction method and device, and electronic device

Abstract

The present application relates to the technical field of electric digital data processing, and particularly relates to a multi-modal data fusion user portrait construction method and device and electronic equipment. The method comprises the following steps: constructing a communication behavior sequence; establishing a time sequence correlation relationship between multi-modal behaviors; extracting communication behavior features of the communication behavior sequence according to the communication behavior sequence and the time sequence correlation relationship; performing collaborative verification on a communication behavior feature vector set to identify the collaborative consistency degree and the conflict degree between the communication behavior features; performing confidence evaluation and conflict resolution processing on the communication behavior features; and constructing a user portrait label system based on the communication behavior features subjected to the confidence evaluation and the conflict resolution processing, and generating each dimension portrait label. The present application realizes accurate portrayal of user communication behavior features by constructing multi-modal behavior sequences, establishing time sequence correlations and causal influence relationships between behaviors, and implementing multi-level collaborative verification and conflict resolution.

Description

Technical Field

[0001] This invention relates to the field of electronic digital data processing technology, and in particular to a method, apparatus and electronic device for constructing user profiles through multimodal data fusion. Background Technology

[0002] With the deep integration of mobile communications and internet services, users generate a wealth of behavioral data during their use of communication services, including call records, SMS interactions, mobile data usage, location changes, application access, and business transaction records. This data reflects users' communication habits, social relationships, activity patterns, and business preferences from different dimensions, providing a data foundation for building comprehensive and accurate user profiles.

[0003] Existing technologies often employ direct fusion or weighted fusion after feature extraction when processing multi-source heterogeneous data. They lack a systematic approach to utilizing the mutual verification and complementary relationships between different modalities. In particular, in communication scenarios, there are close temporal correlations and causal influences among users' call behavior, SMS behavior, data usage behavior, and location trajectory behavior. Existing technologies have failed to fully leverage the inherent relationships between multimodal behaviors to improve the accuracy and robustness of profile construction. Summary of the Invention

[0004] The purpose of this invention is to provide a user profile construction method, apparatus, and electronic device based on multimodal data fusion, so as to solve the problem that the existing technology does not combine the inherent relationship between multimodal behaviors to achieve accurate user profile depiction.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for constructing user profiles through multimodal data fusion includes:

[0007] Receive communication behavior data according to user identification information, and construct communication behavior sequences respectively;

[0008] Using user identification information as the merge key, the communication behavior sequence is time-axis aligned to establish the temporal correlation between multimodal behaviors;

[0009] Based on the communication behavior sequence and its temporal correlation, the communication behavior features of the communication behavior sequence are extracted to form a communication behavior feature vector set;

[0010] Using temporal correlation as a constraint, a collaborative verification is performed on the feature vector set of communication behavior to identify the degree of consistency and conflict among the communication behavior features.

[0011] Based on the collaborative verification results of the communication behavior feature vector set, confidence assessment and conflict resolution are performed on the communication behavior features.

[0012] Based on communication behavior characteristics that have undergone confidence assessment and conflict resolution, a user profile tagging system is constructed to generate profile tags for each dimension.

[0013] Furthermore, the establishment of the temporal correlation includes two sub-steps: establishing synchronous correlation and establishing causal correlation.

[0014] Synchronization association establishment: Extract the movement interval between two adjacent stop point groups in the location trajectory sequence; for each movement interval, find the call session records in the call behavior sequence whose call session start time or call session end time falls into the movement interval, and find the traffic usage records in the traffic usage sequence whose traffic usage start time or traffic usage end time falls into the movement interval, and establish a synchronization association mark between the call session records and traffic usage records falling into the same movement interval and the movement interval.

[0015] Extract the dwell time interval for each dwell point group in the location trajectory sequence. For each dwell time interval, find call session records in the call behavior sequence where both the call session start time and the call session end time fall within the dwell time interval. Find SMS interaction cluster records in the SMS interaction sequence where both the SMS interaction cluster start time and the SMS interaction cluster end time fall within the dwell time interval. Find data usage records in the data usage sequence where both the data usage start time and the data usage end time fall within the dwell time interval. Establish dwell association tags with the call session records, SMS interaction cluster records, and data usage records that fall within the same dwell time interval and associate them with the dwell point group.

[0016] Causal association establishment: For each call session record in the call behavior sequence, find the first SMS interaction cluster record or the first data usage record of the same user within the preset causal influence time window after the end time of the call session record. If found, establish a causal association mark between the call session record and the found SMS interaction cluster record or data usage record, and mark the causal direction as the call triggering subsequent behavior.

[0017] For each SMS interaction cluster record in the SMS interaction sequence, find the first call session record or the first data usage record of the same user within the preset causal influence time window after the end time of the SMS interaction cluster record. If found, establish a causal association marker and mark the causal direction as SMS triggering subsequent behavior.

[0018] For each data usage record in the data usage sequence, find the first call session record or the first SMS interaction record of the same user within the preset causal influence time window after the end time of the data usage record. If found, establish a causal association marker, and mark the causal direction as data usage triggering subsequent behavior.

[0019] Furthermore, call behavior characteristics include call activity, call time period preference, call recipient stability, and call location characteristics; SMS behavior characteristics include SMS activity, SMS direction characteristics, and SMS time period preference; data usage behavior characteristics include data usage volume, data continuity, access network type preference, and access service type preference; location trajectory behavior characteristics include movement range, stop point characteristics, and movement frequency.

[0020] Furthermore, the collaborative verification is based on synchronization association tags and causal association tags:

[0021] For the collaborative verification of call behavior and location trajectory behavior: extract call session records and location trajectory movement ranges with synchronization association markers, extract call session records and stop point groups with stop association markers, and calculate the ratio of call duration to stop duration as the call density during the stop period;

[0022] For the joint verification of call behavior and SMS behavior: compare the average response time from call to SMS and the average response time from SMS to call. If the former is shorter than the latter, it is recorded that the user is more likely to initiate SMS interaction after a call; otherwise, it is recorded that the user is more likely to initiate a call after SMS interaction.

[0023] For the joint verification of traffic usage behavior and location trajectory behavior: compare the average value of mobile traffic density with the average value of stationary traffic density. If the former is greater than the latter, it is registered as the user's traffic usage intensity during the movement is higher than that during the stationary period.

[0024] Furthermore, regarding conflict identification: when there is a logical contradiction between the communication behavior characteristics of the same user within the same time interval, it is registered as a behavioral conflict. Conflict types include location-origin conflict, time period preference conflict, and behavioral pattern conflict; when the location of the caller's number is inconsistent with the administrative division of the user's location during the call, and the number of times the inconsistency occurs exceeds a preset conflict tolerance threshold, it is registered as a location-origin conflict; when the user's main call period and main traffic usage period differ in location within a day by more than a preset time period deviation threshold, and this deviation pattern continues to occur for more than a preset duration threshold, it is registered as a time period preference conflict; when there is an inverse correlation between the user's SMS sending / receiving ratio and call type marker, such as sending being the main activity but the call being the called party, or receiving being the main activity but the call being the called party, it is registered as a behavioral pattern conflict; all identified behavioral conflicts are registered in the conflict record table according to the conflict type and the time of occurrence.

[0025] Furthermore, for each communication behavior feature, its baseline confidence level is determined by the reliability level of its data source. The deviation level calculated in all collaborative verifications involving the communication behavior feature is read, and the average of all consistency scores is taken as the collaborative consistency score of the communication behavior feature. The collaborative consistency score is multiplied by the initial confidence level to obtain the collaboratively corrected confidence level. The number of conflict records involving the communication behavior feature in the conflict record table is read. Each conflict record reduces the confidence level by a preset conflict penalty value. The collaboratively corrected confidence level is subtracted from the total conflict penalty value to obtain the final confidence level.

[0026] Furthermore, after assessing the confidence level of the communication behavior characteristics, conflict resolution is performed on them:

[0027] For location-home conflicts, the frequency distribution of the other party's home location in different locations is statistically analyzed. If a certain home location has the highest frequency in a certain location and is higher than other combinations, then that combination is used, and the home location information in other conflicting combinations is corrected to that home location.

[0028] For time period preference conflicts, when the shortest time difference is greater than or equal to the preset extreme deviation threshold, the main call period and the main data usage period are registered as independent active periods, and a time period split mark is added to the user profile; when the shortest time difference is greater than or equal to the preset time period deviation threshold and less than the preset extreme deviation threshold, the two intervals of the main call period and the main data usage period on the 24-hour time axis are used as endpoints, and the continuous time interval between the two intervals is registered as the user's comprehensive active extended period. Within this comprehensive active extended period, the occurrence distribution of call behavior and data usage behavior is statistically analyzed in half-hour increments, and the half-hour interval with the highest distribution density is registered as the comprehensive active peak period.

[0029] For behavioral pattern conflicts, check whether there is a similar reverse correlation pattern in the historical profile records. If it exists and continues to exceed the preset historical persistence threshold, it is determined to be a stable feature and only a special feature is added. If it does not exist, check whether there are abnormal event records before and after the conflict.

[0030] Furthermore, the method for generating communication behavior dimension tags is as follows: for call activity tags, the call frequency is compared with a preset call frequency classification threshold, and the total call duration is compared with a preset call duration classification threshold, and the tags are generated by combining them; for call time period preference tags, the primary call time period and secondary call time period are directly registered; for primary contact tags and contact network tags, they are determined by the primary contact number and the number of frequent contacts.

[0031] The method for generating spatiotemporal activity dimension labels is as follows:

[0032] For activity range tags, the coverage area of ​​the movement range is compared with preset large range thresholds and preset medium range thresholds to generate wide-area / medium-area / local activity tags;

[0033] For the dwell feature label, the total number of dwell points is compared with a preset multi-dwelling threshold to generate dwell point quantity features of multi-dwelling type and few-dwelling type; the average dwell time is compared with a preset long dwelling threshold to generate dwell time features of long dwelling type and short dwelling type. The dwell feature label is formed by combining the dwell point quantity feature and the dwell time feature, and the combination method is to concatenate the judgment results of the above two dimensions with hyphens.

[0034] For the mobility frequency tag, the number of moves is compared with a preset high-frequency mobility threshold and a preset medium-frequency mobility threshold to generate high-frequency mobility type, medium-frequency mobility type, and low-frequency mobility type tags.

[0035] On the other hand, the present invention also provides a user profile construction apparatus for multimodal data fusion, comprising:

[0036] The window creation module is used to establish a profile building session, set the profile building time window, and register user identification information and multimodal data source configuration;

[0037] The sequence generation module is used to receive communication behavior data according to user identification information and construct communication behavior sequences respectively;

[0038] The correlation analysis module is used to align the communication behavior sequence with the user identification information as the merging key on the time axis in order to establish the temporal correlation between multimodal behaviors.

[0039] The feature extraction module is used to extract communication behavior features from the communication behavior sequence and its temporal correlation, forming a communication behavior feature vector set.

[0040] The collaborative verification module is used to perform collaborative verification on the communication behavior feature vector set based on temporal correlation, and to identify the degree of consistency and conflict between communication behavior features.

[0041] The feature evaluation module is used to evaluate the confidence level and resolve conflicts of communication behavior features based on the collaborative verification results of the communication behavior feature vector set.

[0042] The tag generation module is used to build a user profile tag system based on communication behavior characteristics that have undergone confidence assessment and conflict resolution, and generate profile tags for various dimensions.

[0043] On the other hand, the present invention also provides an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the method described above.

[0044] The beneficial effects of this invention are as follows: By establishing temporal correlations between four types of communication behaviors—calls, text messages, data usage, and location trajectories—multi-source heterogeneous data is transformed from independent discrete records into a logical whole with synchronous and causal correlation markers. Based on this temporal correlation, collaborative verification is performed as a constraint to identify the degree of deviation and logical conflict between the characteristics of each modality of behavior. The confidence level of features is quantified and corrected according to the degree of deviation. Differentiated resolution strategies are implemented for conflicts between home location, time period preferences, and behavioral patterns, effectively solving the technical problem of profile distortion caused by deviations or abnormal events in single-source data, and improving the robustness of the profile under fluctuating data quality conditions. At the same time, by constructing a multi-dimensional tag system that includes communication behaviors, spatiotemporal activities, and social relationships, and setting an incremental update mechanism triggered by the magnitude of change, the profile tags can be dynamically and adaptively adjusted with the evolution of user behavior, solving the technical problem that static profiles cannot reflect changes in user behavior patterns, and providing reliable data support for accurate operation and personalized adaptation in communication service scenarios. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart of the user profile construction method based on multimodal data fusion in this embodiment.

[0047] Figure 2This is a schematic diagram of the user profile construction device for multimodal data fusion in this embodiment.

[0048] Figure 3 This is a schematic diagram of the electronic device in this embodiment. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0050] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Those skilled in the art can make similar modifications without departing from the spirit of the present invention; therefore, the present invention is not limited to the specific embodiments disclosed below.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0052] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0053] Figure 1 This is a flowchart illustrating a user profile construction method based on multimodal data fusion, provided in an embodiment of the present invention. Figure 1 As shown, the method provided in this embodiment includes:

[0054] Step S1: Establish a profile building session, set the profile building time window, and register user identification information and multimodal data source configuration.

[0055] Specifically, the profile building session is a logical unit used to uniformly process and manage the communication behavior data of a single user within a continuous period of time. The profile building session uses a fixed time window as the processing unit. The start and end times of the time window are uniformly set by the profile building platform according to the data access cycle. In this embodiment, the profile building time window is set to 1 day.

[0056] For example, the data access cycle can be set to 60 minutes, that is, a profile building session is started every 60 minutes. The time window adopts a boundary caliber that includes the start time and excludes the end time. For example, if the start time is 10:00:00 on July 1, 2026 and the end time is 11:00:00 on July 1, 2026, then all data whose event reception time satisfies [10:00:00, 11:00:00) belong to the current session.

[0057] When a profile building session is established, the data batch registration table of the user to be processed is read first. The data batch registration table records at least the batch number, data source system number, data access start time, data access end time and batch data integrity mark. The profile building session number is formed by concatenating the construction date, batch number and data source system number in a fixed order. The same user belongs to only one profile building session within the same time window.

[0058] The registration of user identification information adopts a multi-level matching approach, specifically including: when a unique communication user ID exists in the source record, the unique communication user ID is directly used as the user identification information for registration; when the unique communication user ID is missing but a mobile phone number exists, the binding record between the mobile phone number and the unique communication user ID in the user identity relationship table is read. If the binding start time is not later than the current time, the unbinding time is empty or later than the current time, and the binding status is valid, the bound unique communication user ID is used as the user identification information; when both the unique communication user ID and the mobile phone number are missing, temporary user identification information is generated in a fixed order of device unique identifier, session identifier, and first data time. The temporary user identification information is maintained until a valid unique communication user ID is matched later, at which point the candidate profile data formed therein is transferred to the corresponding formal user identification information for reprocessing.

[0059] The multimodal data source configuration shall at least register call data source configuration, SMS data source configuration, data usage data source configuration, and location data source configuration. Each data source configuration shall at least include the source system number, data field list, field integrity requirements, and data trust level. The specific rules for classifying the data trust level are as follows:

[0060] The data trust level is pre-configured by the profiling platform based on the system attributes of the data source, the data generation link, and the field verification mechanism. It includes at least three levels: high trust, medium trust, and low trust. The criteria for a high trust source are: data directly generated by core network elements or verified by the billing system. Specifically, this includes: user subscription data directly output by the Home Location Register or Home Subscriber Server; call detail records generated by the Mobile Switching Center or Serving GPRS Support Node and verified by the Operation Support System; and transaction final status records confirmed by the online or offline billing system. The field integrity requirement for a high trust source is no less than a preset first integrity threshold, which ranges from 95% to 100%, preferably 98%. The criteria for a medium trust source are: data reported by wireless access network equipment or obtained from a third party through a compliant authorized interface, with acceptable latency or delays during transmission or parsing. Accuracy errors specifically include: measurement reports reported by base stations or radio network controllers; network status logs and cell handover records reported by user terminals at preset cycles; and auxiliary information obtained from third-party platforms through compliant data interfaces that has undergone deduplication and preliminary verification. The field integrity requirement of trusted sources is not lower than a preset second integrity threshold, which ranges from 80% to 95%, preferably 90%. The criteria for judging low-trust sources are: data generated locally by the user terminal and not immediately confirmed by the network side, or derived from publicly estimated data. Specifically, this includes: reported records cached locally by the user terminal; client operation status self-check reports that have not been confirmed by the network side; and auxiliary geographic location information estimated through reverse analysis of crowdsourced data or publicly available geographic location databases. This type of data is greatly affected by the terminal operating environment and network fluctuations, and has a high risk of false alarms or missing data. Its field integrity requirement is lower than the preset second integrity threshold.

[0061] Step S2: Receive communication behavior data according to user identification information. The communication behavior data includes call behavior data, SMS interaction data, data usage data, and location trajectory data. Construct communication behavior sequences for each of these sequences, transforming the original communication behavior data into ordered and structured behavior sequences to lay the foundation for subsequent correlation analysis. The communication behavior sequences include call behavior sequences, SMS interaction sequences, data usage sequences, and location trajectory sequences.

[0062] Specifically, a call behavior sequence is constructed based on call behavior data, wherein: the receiving scope of the call behavior data is: call records generated by the user in the calling or called scenario, and each call record has at least the call occurrence time, call duration, call type, location of the other party's number, reason for call termination and base station number, and the call type includes calling and called;

[0063] The call behavior sequence is constructed by arranging all call records of the same user within the profile construction time window in chronological order of the call occurrence time to form the original call behavior sequence.

[0064] The original call behavior sequence is processed by time period merging. Records with a time interval between two adjacent call records that is less than a preset time period merging threshold are grouped into the same call session group. The preset time period merging threshold ranges from 120 seconds to 600 seconds. Preferably, in this embodiment, it is set to 300 seconds. It is used to distinguish between continuous calls and discrete calls. If the interval between two calls is less than 5 minutes, it can be regarded as the same call process being interrupted.

[0065] Multiple call records within the same call session group are merged into one call session record. The call start time of the merged call session record is the earliest call occurrence time within the group, and the call end time is the latest call end time within the group. The total call duration is the sum of the call durations of all records within the group. The call type is determined by comparing the number of calls made by the caller and the number of calls made by the callee. When the number of calls made is greater than the number of calls made by the callee, it is registered as a caller-led session; otherwise, it is registered as a callee-led session. When the number of calls made is equal to the number of calls made by the callee, it is registered as a two-way balanced session.

[0066] The merged call session records are rearranged in chronological order to form a call behavior sequence. Each element in the call behavior sequence includes at least the call session start time, call session end time, total call session duration, call type flag, location information of the other party's number, and base station number.

[0067] Specifically, an SMS interaction sequence is constructed based on SMS interaction data, wherein: the receiving scope of SMS interaction data is: SMS records sent or received by the user, and each SMS record has at least the SMS generation time, SMS direction, SMS length and the recipient's number, and the SMS direction includes sending and receiving;

[0068] The SMS interaction sequence is constructed as follows: all SMS records of the same user within the profile construction time window are arranged in chronological order of SMS generation time to form an original SMS interaction sequence; density clustering is performed on the original SMS interaction sequence, the time interval between two adjacent SMS records is calculated, and records with a time interval less than a preset dense interaction threshold are grouped into the same SMS interaction cluster, wherein the preset dense interaction threshold ranges from 60 seconds to 300 seconds, preferably 120 seconds in this embodiment; multiple SMS records within the same SMS interaction cluster are merged into one SMS interaction cluster record, the start time of the merged SMS interaction cluster record is the earliest SMS generation time within the cluster, the end time is the latest SMS generation time within the cluster, the total number of SMS messages is the number of records within the cluster, and the number of sent and received messages are counted separately; the clustered SMS interaction cluster records are rearranged in chronological order to form an SMS interaction sequence, each element in the SMS interaction sequence including at least the SMS interaction cluster start time, SMS interaction cluster end time, total number of SMS messages, number of sent messages, and number of received messages.

[0069] Specifically, a traffic usage sequence is constructed based on traffic usage data, wherein: the receiving scope of traffic usage data is: traffic usage records generated by users during the use of mobile data traffic, and each traffic usage record has at least the traffic usage start time, traffic usage end time, traffic usage amount, access network type and access service type, wherein the access network type includes 4G, 5G or WiFi;

[0070] The traffic usage sequence is constructed as follows: all traffic usage records of the same user within the profile construction time window are arranged in chronological order of traffic usage start time to form an original traffic usage sequence; the original traffic usage sequence is then subjected to overlapping and merging processing: if the traffic usage end time of the previous traffic usage record is later than or equal to the traffic usage start time of the next traffic usage record, the two records are merged into one continuous traffic usage record. The start time of the merged traffic usage is the start time of the previous record, the end time of the traffic usage is the end time of the next record, and the traffic usage amount is the sum of the traffic usage amounts of the two records. The access network type and access service type are respectively marked by the type with the longest usage time as the type label of the merged record. The merged traffic usage records are rearranged in chronological order to form a traffic usage sequence. Each element in the traffic usage sequence includes at least the traffic usage start time, traffic usage end time, traffic usage amount, access network type, and access service type.

[0071] Specifically, a location trajectory sequence is constructed based on the location trajectory data, wherein: the receiving scope of the location trajectory data is: the location update records generated by the user in the network, and each location trajectory record has at least the location record time, base station number, base station latitude and longitude coordinates and location update reason;

[0072] The location trajectory sequence is constructed as follows: all location trajectory records of the same user within the profile construction time window are arranged in chronological order to form an original location trajectory sequence; the original location trajectory sequence is processed for stop point identification, and the spatial distance between the base station latitude and longitude coordinates of two adjacent location trajectory records is calculated. The spatial distance is calculated using the spherical distance formula, and the unit is meters. If the spatial distance is less than a preset stop point determination distance threshold, the two records are grouped into the same stop point candidate group. The preset stop point determination distance threshold ranges from 100 meters to 1000 meters, preferably 300 meters, which is used to determine whether the user is active in the same area; within the same stop point candidate group... Records are arranged in chronological order. If the time interval between any two adjacent records within a group is greater than a preset dwell time threshold, the candidate group is split into two independent dwell point groups at the time interval. The preset dwell time threshold ranges from 10 minutes to 60 minutes, preferably 30 minutes. The dwell start time for each dwell point group is the earliest location record time within the group, the dwell end time is the latest location record time within the group, and the dwell location is the geometric center of the latitude and longitude coordinates of each base station within the group, i.e., the average longitude and average latitude. The identified dwell point groups are arranged in chronological order to form a location trajectory sequence. Each element in the location trajectory sequence includes at least the dwell start time, dwell end time, and dwell location coordinates.

[0073] Step S3: Using user identification information as the merging key, align the communication behavior sequences along the time axis to establish the temporal correlation between multimodal behaviors and realize the logical connection between the communication behavior sequences.

[0074] Specifically, the time axis alignment process is as follows: using the profile construction time window as a unified time axis, the call behavior sequence, SMS interaction sequence, data usage sequence and location trajectory sequence are mapped onto the unified time axis respectively. During mapping, each element in each communication behavior sequence occupies the corresponding time interval on the unified time axis with its start time and end time.

[0075] In this embodiment, the establishment of the temporal correlation includes two sub-steps: establishing synchronous correlation and establishing causal correlation.

[0076] Specifically, the steps for establishing synchronous association are as follows: extract the movement interval between two adjacent stop point groups in the location trajectory sequence, wherein the start time of the movement interval is the end time of the previous stop point group and the end time is the start time of the next stop point group;

[0077] For each mobile zone, find call session records in the call behavior sequence whose call session start time or call session end time falls within that mobile zone, and find data usage records in the data usage sequence whose data usage start time or data usage end time falls within that mobile zone. Establish a synchronization association mark between the call session records and data usage records falling within the same mobile zone and that mobile zone. The synchronization association mark includes at least a mobile zone identifier, an associated call session record identifier, and an associated data usage record identifier. The synchronization association mark indicates that the user generated both call behavior and data usage behavior during the mobile process.

[0078] Extract the dwell time interval for each dwell point group in the location trajectory sequence. For each dwell time interval, find call session records in the call behavior sequence whose call session start time and call session end time both fall within the dwell time interval; find SMS interaction cluster records in the SMS interaction sequence whose SMS interaction cluster start time and SMS interaction cluster end time both fall within the dwell time interval; find data usage records in the data usage sequence whose data usage start time and data usage end time both fall within the dwell time interval. Establish a dwell association marker with the call session records, SMS interaction cluster records, and data usage records falling within the same dwell time interval and the dwell point group. The dwell association marker includes at least a dwell point group identifier, an associated call session record identifier, an associated SMS interaction cluster record identifier, and an associated data usage record identifier. The dwell association marker indicates that the user generated multiple communication behaviors simultaneously during the dwell period.

[0079] Specifically, the steps for establishing a causal relationship are as follows: For each call session record in the call behavior sequence, find the first SMS interaction cluster record or the first data usage record of the same user within a preset causal influence time window after the end time of the call session record. If found, establish a causal relationship marker between the call session record and the found SMS interaction cluster record or data usage record. The causal direction is marked as the call triggering subsequent behavior. The preset causal influence time window ranges from 30 seconds to 600 seconds. Preferably, the preset causal influence time window is 120 seconds, which is used to define the maximum time interval between behaviors to determine whether there is a causal relationship.

[0080] For each SMS interaction cluster record in the SMS interaction sequence, find the first call session record or the first data usage record of the same user within the preset causal influence time window after the end time of the SMS interaction cluster record. If found, establish a causal association marker and mark the causal direction as SMS triggering subsequent behavior.

[0081] For each data usage record in the data usage sequence, find the first call session record or the first SMS interaction record of the same user within the preset causal influence time window after the end time of the data usage record. If found, establish a causal association marker, and mark the causal direction as data usage triggering subsequent behavior.

[0082] The synchronous association and causal association together constitute the temporal association between multimodal behaviors, and are stored in the temporal association table with user identification information as the index.

[0083] Step S4: Based on the communication behavior sequence and its temporal correlation, extract the communication behavior features of the communication behavior sequence to form a communication behavior feature vector set. Extract quantified features from the processed sequence to characterize the user's specific behavior patterns. The communication behavior features include call behavior features, SMS behavior features, data usage behavior features, and location trajectory behavior features.

[0084] Specifically, call behavior characteristics include call activity, call time preference, call recipient stability, and call location characteristics:

[0085] The analysis of call activity includes: counting the total number of call session records within the time window of the profile construction as the call frequency; counting the total call duration of all call session records as the total call duration; calculating the deviation of the call duration of each call session record from the average call duration of all call session records; and taking the sum of the absolute values ​​of all differences and dividing by the total number of call session records to obtain the call duration dispersion.

[0086] The analysis of call time period preferences includes: dividing the profile construction time window into 24 time period intervals by hour, counting the number of call session records in each time period interval, registering the time period interval with the most records as the primary call period, and registering the time period interval with the second most records as the secondary call period; the analysis of call object stability includes: extracting the other party's number from all call session records, counting the frequency of different other party's numbers, registering the other party's number with the highest frequency as the primary contact number, calculating the proportion of the number of calls to the primary contact number to the total number of calls as the primary contact concentration, and counting the number of other party's numbers whose frequency exceeds a preset frequent contact threshold as the number of frequent contacts. The preset frequent contact threshold ranges from 3 to 10 times, preferably 5 times.

[0087] The analysis of call location characteristics includes: extracting the location information of the other party's number from all call session records, counting the frequency of different locations, registering the location with the highest frequency as the primary call location, and counting the total number of location types as the call location diversity.

[0088] Specifically, SMS behavior characteristics include SMS activity, SMS direction characteristics, and SMS time period preferences: For SMS activity, the analysis includes: counting the total number of SMS interaction cluster records within the time window of the profile construction as the SMS cluster frequency, counting the sum of the total number of SMS messages in all SMS interaction cluster records as the total number of SMS messages, and calculating the degree of deviation between the total number of SMS messages in each SMS interaction cluster record and the average number of SMS messages in all SMS interaction cluster records to obtain the SMS cluster density dispersion;

[0089] The analysis of SMS direction characteristics includes: summing the number of sent messages and the number of received messages in all SMS interaction cluster records, and calculating the ratio of the sum of the number of sent messages to the sum of the number of received messages as the SMS sending-receiving ratio;

[0090] The analysis of SMS time period preferences includes: using the same method as for call time period preferences, registering the time interval with the most messages as the main SMS time period.

[0091] Specifically, traffic usage behavior characteristics include traffic usage volume, traffic continuity, access network type preference, and access service type preference:

[0092] The analysis of traffic usage includes: the sum of traffic usage of all traffic usage records within the time window for building the statistical profile, which is taken as the total traffic usage;

[0093] For traffic continuity, the analysis includes: counting the number of traffic usage records, dividing the total traffic usage by the number of traffic usage records to obtain the average traffic usage per instance, and calculating the average time interval between two adjacent traffic usage records as the average traffic usage interval.

[0094] The analysis of access network type preference includes: summing the traffic usage in traffic usage records for 4G, 5G and WiFi access network types respectively, and registering the access network type with the highest traffic usage as the primary access network type;

[0095] The analysis of access service type preferences includes: calculating the sum of traffic usage for each access service type, and registering the access service type with the highest traffic usage as the primary access service type.

[0096] Specifically, the location trajectory behavior characteristics include movement range, stop point characteristics, and movement frequency: For movement range, the analysis includes: extracting the stop point coordinates of all stop point groups, finding the maximum and minimum values ​​in the longitude direction and the maximum and minimum values ​​in the latitude direction, and multiplying the difference between the maximum and minimum longitude values ​​by the difference between the maximum and minimum latitude values ​​to obtain the movement range coverage area;

[0097] The analysis of stop point characteristics includes: counting the total number of stop point groups as the total number of stop points, calculating the stay duration of each stop point group, and taking the average stay duration of all stop point groups as the average stay duration;

[0098] The analysis of movement frequency includes counting the number of movement intervals between adjacent stop groups as the number of movements.

[0099] Step S5: Using temporal correlation as a constraint, perform collaborative verification on the communication behavior feature vector set to identify the degree of collaborative consistency and conflict among the communication behavior features.

[0100] Specifically, the collaborative verification is based on synchronization association markers and causal association markers:

[0101] For the collaborative verification of call behavior and location trajectory behavior: extract call session records and location trajectory movement intervals with synchronization association markers. For each synchronization association marker, calculate the position ratio of the call occurrence time interval within the movement interval, specifically: (midpoint time of call occurrence time interval - start time of movement interval) / total duration of movement interval, to obtain the relative time position of the call during the movement process.

[0102] Calculate the average and standard deviation of the relative time positions among all synchronization-related tags for the same user;

[0103] Extract call session records and stop point groups with stop association markers, and calculate the proportion of call duration to stop duration as the call density during the stop period;

[0104] For the collaborative verification of call behavior and SMS behavior: extract call session records and subsequent SMS interaction cluster records with causal association markers and causal direction of call triggering subsequent SMS, calculate the time interval between their start times as the call to SMS response time, and calculate their average value.

[0105] Extract SMS interaction cluster records and subsequent call session records that have causal association tags and whose causal direction is SMS triggering subsequent calls. Calculate the time interval between their start times as the SMS-to-call response time and calculate their average value.

[0106] Compare the average response time from call to SMS to average response time from SMS to call. If the former is shorter than the latter, it is recorded that the user is more likely to initiate SMS interaction after a call; otherwise, it is recorded that the user is more likely to initiate a call after SMS interaction.

[0107] For the collaborative verification of traffic usage behavior and location trajectory behavior: extract traffic usage records and location trajectory movement intervals with synchronous association tags, calculate the traffic usage per unit movement time as the movement traffic density, and calculate its average value. The traffic usage per unit movement time = traffic usage / total movement interval duration.

[0108] Extract traffic usage records and stop point groups with stop association tags, calculate traffic usage per unit stop time as stop traffic density, and calculate its average value. The traffic usage per unit stop time = traffic usage / stop time.

[0109] Compare the average mobile traffic density with the average stationary traffic density. If the former is greater than the latter, it is recorded that the user's traffic usage intensity during the mobile process is higher than that during the stationary period.

[0110] For deviation analysis, in this invention, deviation is a quantitative indicator that measures the difference between any two related communication behavior features. Its calculation method is determined based on the feature type involved in the verification. All deviations are normalized before being included in the confidence calculation, so that their value range is uniformly mapped to the [0,1] interval.

[0111] The method for analyzing the degree of deviation between call behavior and location trajectory behavior is as follows:

[0112] For the relative time position of a call during movement, the first degree of deviation will be determined based on its standard deviation. If the standard deviation is less than 0.15, it means that the user's call behavior is highly stable in time distribution during movement, and the first degree of deviation is 0.05. If the standard deviation is greater than 0.45, it means that the distribution is extremely dispersed, and the first degree of deviation is 0.95. In other cases, the first degree of deviation is the original value of the standard deviation.

[0113] For call density during stay, the call density during stay under all stay-related tags of the same user is compared pairwise, and the average of the absolute differences of all comparison pairs is calculated as the second degree of bias. When the call density during stay differs significantly among multiple stay points, it indicates that there are large differences in the user's call behavior during stay in different locations. The criterion for judging significant differences is that the average of the absolute differences is greater than 0.5.

[0114] The method for analyzing the deviation between call behavior and SMS behavior is as follows: Calculate the absolute difference between the average response time from call to SMS and the average response time from SMS to call. Divide this absolute difference by the total duration of the preset causal influence time window to obtain the normalized third deviation degree. The third deviation degree is used to quantify the strength of the one-way response tendency between call and SMS behavior. When the third deviation degree is less than 0.2, it indicates that the mutual response speed between call and SMS is similar and the two-way switching is smooth. When the third deviation degree is greater than 0.6, it indicates that there is a significant one-way behavior dominance tendency.

[0115] The method for analyzing the degree of deviation between traffic usage behavior and location trajectory behavior is as follows: compare the average value of mobile traffic density with the average value of stationary traffic density, calculate the difference between the two and divide by the sum of the two to obtain the fourth degree of deviation. The value range of the fourth degree of deviation is [-1,1]. It is converted into an absolute value deviation in the range of [0,1] by the mapping formula: (fourth degree of deviation + 1 / 2). The fourth degree of deviation reflects the magnitude of the difference in traffic usage intensity between the mobile state and the stationary state.

[0116] The method for analyzing the degree of deviation between the call location and the location trajectory behavior is as follows: Extract the location information of the other party's number in the call session record with synchronous association markers, as well as the administrative division information of the user's location during the call. Match the location information with the administrative division information. If they match, the match is successful; if they do not match, the match fails. Calculate the proportion of the number of failed matches among all synchronous association markers of the same user to the total number of synchronous association markers. This proportion is used as the fifth degree of deviation. The value of the fifth degree of deviation ranges from 0 to 1. The larger the value, the more serious the deviation between the call location and the actual location.

[0117] For conflict identification: When there is a logical contradiction between the communication behavior characteristics of the same user within the same time interval, it is registered as a behavioral conflict. Conflict types include home location conflict, time period preference conflict and behavior pattern conflict.

[0118] If the location of the other party's number is inconsistent with the administrative division of the user's location during the call, and the number of times the inconsistency occurs exceeds a preset conflict tolerance threshold, it is registered as a location-location conflict. The preset conflict tolerance threshold ranges from 3 to 10 times, and preferably has a value of 5 times.

[0119] If the position of a user's main call time period and main data usage time period differs from the position of the user's main data usage time period by more than a preset time period deviation threshold, and this deviation pattern continues to occur for more than a preset duration threshold, it is registered as a time period preference conflict. The preset time period deviation threshold is 6 hours, and the preset duration threshold is 3 days.

[0120] There is an inverse correlation between a user’s SMS sending and receiving ratio and call type marking. For example, if the user is primarily sending but the call is primarily receiving, or if the user is primarily receiving but the call is primarily receiving, this is registered as a behavioral pattern conflict.

[0121] All identified behavioral conflicts are recorded in the conflict record table according to the conflict type and the time of occurrence.

[0122] Step S6: Based on the collaborative verification results of the communication behavior feature vector set, the confidence level of the communication behavior features is evaluated and conflict resolution is performed. For each communication behavior feature, its baseline confidence level is determined by the credibility level of its data source. For example, high confidence, medium confidence, and low confidence correspond to initial confidence levels of 0.9, 0.7, and 0.5, respectively.

[0123] In addition, this embodiment also introduces a dynamic fine-tuning mechanism for the level based on data consistency: if the communication behavior characteristics from a low-trust source have the same trend as the communication behavior characteristics from a high-trust source of the same user within a continuous preset verification time window, that is, the correlation coefficient between the two changes is greater than 0.8, then the initial confidence of the low-trust source characteristic is temporarily increased to 0.6. The preset verification time window ranges from 3 profile building sessions to 10 profile building sessions, preferably within 5 profile building sessions.

[0124] If a feature parameter from a high-confidence source continuously and severely conflicts with a feature parameter from a medium-confidence or low-confidence source of the same user within a consecutive preset verification time window, and the number of conflict records exceeds a preset conflict tolerance threshold, the preset conflict tolerance threshold ranges from 3 to 10 times, preferably 5 times, then the initial confidence level of the high-confidence source feature is temporarily reduced to 0.7 until the conflict is resolved.

[0125] Read the degree of deviation calculated from all collaborative verifications involving this communication behavior feature. For each collaborative verification, set its consistency score to 1 - degree of deviation, take the average of all consistency scores as the collaborative consistency score of this communication behavior feature, and multiply the collaborative consistency score by the initial confidence level to obtain the collaboratively corrected confidence level.

[0126] The number of conflict records involving the communication behavior feature is read from the conflict record table. Each conflict record reduces the confidence level by a preset conflict penalty value. The final confidence level is obtained by subtracting the total conflict penalty value from the collaboratively corrected confidence level. When the final confidence level is lower than the preset minimum confidence level threshold, the communication behavior feature is marked as a low confidence feature. In this embodiment, the preset minimum confidence level threshold is set to 0.1, and the preset conflict penalty value ranges from 0.05 to 0.15. Preferably, the preset conflict penalty value is 0.1.

[0127] After assessing the confidence level of the communication behavior characteristics, conflict resolution is performed:

[0128] For location-home conflicts, the frequency distribution of the other party's home location in different locations is statistically analyzed. If a certain home location has the highest frequency in a certain location and is higher than other combinations, then that combination is used, and the home location information in other conflicting combinations is corrected to that home location.

[0129] For time period preference conflicts, the main call period and the main traffic usage period involved in the conflict are read. The main call period and the main traffic usage period are represented by time intervals in the 24-hour system, such as "20:00-21:00" and "09:00-10:00".

[0130] Calculate the shortest time difference between two time intervals on a 24-hour time axis. The shortest time difference is calculated by calculating the absolute value of the linear time difference T1 between the midpoint times of the two time intervals, and the circular time difference T2 (24 hours minus T1). The smaller value between T1 and T2 is taken as the shortest time difference. For example, if the main call period is 22:00-23:00 with a midpoint of 22:30, and the main traffic usage period is 07:00-08:00 with a midpoint of 07:30, the absolute value of the linear time difference is 15 hours, and the circular time difference is 24-15=9 hours. The smaller value of 9 hours is taken as the shortest time difference.

[0131] When the shortest time difference is greater than or equal to a preset extreme deviation threshold, the preset extreme deviation threshold ranges from 10 hours to 14 hours, preferably 12 hours, and is determined to be an uncoordinated time period preference. In this case, the main call period and the main data usage period are registered as independent active periods, and a time period split mark is added to the user profile. The value of the time period split mark is call-data usage period split, indicating that the user's call activity peak and data usage activity peak are in completely opposite work and rest cycles, and the two have no overlap or very little overlap in time. In this case, the two time period preferences should be treated as independent behavioral patterns and processed separately in subsequent recommendation scheduling and service adaptation.

[0132] When the shortest time difference is greater than or equal to the preset time period deviation threshold and less than the preset extreme deviation threshold, it is determined that the time period preference is partially reconcilable. In this case, taking the two intervals on the 24-hour time axis of the main call period and the main traffic usage period as endpoints, the continuous time interval between the two intervals is registered as the user's comprehensive active extended time period. The starting point of the comprehensive active extended time period is the start time of the earlier period of the main call period and the main traffic usage period, and the ending point is the end time of the later period. Within the comprehensive active extended time period, the occurrence distribution of call behavior and traffic usage behavior is statistically analyzed in half-hour increments. The half-hour interval with the highest distribution density is registered as the comprehensive active peak period. The registration information of the comprehensive active extended time period and the comprehensive active peak period is used to guide business scenarios with compatibility requirements for the two types of behavior.

[0133] For behavioral pattern conflicts, check whether there is a similar reverse correlation pattern in the historical profile records. If it exists and continues to exceed the preset historical duration threshold, it is determined to be a stable feature and only a specialness mark is added. If it does not exist, check whether there are abnormal event records before and after the conflict. If they exist, they are attributed to an anomaly and a mark is added. Otherwise, the side with lower confidence is marked as pending verification. In this embodiment, the preset historical duration threshold is 30 days. The abnormal event records include holidays and package changes, etc.

[0134] Step S7: Based on the communication behavior characteristics after confidence assessment and conflict resolution, construct a user profile tag system and generate profile tags for each dimension. The user profile tag system includes at least communication behavior dimension tags, spatiotemporal activity dimension tags, and social relationship dimension tags.

[0135] Specifically, the method for generating communication behavior dimension tags is as follows: For call activity tags, the call frequency is compared with a preset call frequency classification threshold, and the total call duration is compared with a preset call duration classification threshold. The tags are then combined to generate tags. In this embodiment, the preset call frequency classification threshold is: high frequency level is more than 20 times / day, medium frequency level is 5-20 times / day, and low frequency level is less than 5 times / day. The preset call duration classification threshold is: long duration level is more than 60 minutes / day, medium duration level is 20-60 minutes / day, and short duration level is less than 20 minutes / day.

[0136] For call time preference tags, register directly using the primary call time and secondary call time.

[0137] For the primary contact tag and the contact network tag, it is determined by the primary contact number and the number of frequent contacts. For example, if the number of frequent contacts is more than 10, it is considered a wide social network; if the number of frequent contacts is between 5 and 10, it is considered a medium social network; and if the number of frequent contacts is less than 5, it is considered a narrow social network.

[0138] Specifically, the spatiotemporal activity dimension label is generated as follows: For the activity range label, the coverage area of ​​the movement range is compared with a preset large range threshold and a preset medium range threshold to generate wide-area / medium-area / local activity labels. In this embodiment, the preset large range threshold is 100 square kilometers, the preset medium range threshold is 10 square kilometers, the corresponding range of the wide-area activity label is (preset large range threshold, +∞), the corresponding range of the medium-area activity label is [preset medium range threshold, preset large range threshold], and the corresponding range of the local activity label is [0, preset medium range threshold).

[0139] For dwell feature tags, all dwell point groups identified in the location trajectory sequence are read. Each dwell point group records at least the dwell start time, dwell end time, and dwell location coordinates. The total number of dwell point groups under this user identification information within the current profile construction time window is counted as the total number of dwell points. For each dwell point group, its dwell time is calculated. The dwell time is equal to the dwell end time minus the dwell start time, in minutes. The dwell time of all dwell point groups is summed and divided by the total number of dwell point groups to obtain the average dwell time. The average dwell time reflects the average length of time a user stays at a single dwell point, in minutes.

[0140] The total number of dwell points is compared with a preset multi-dwell point threshold, which ranges from 3 to 8, preferably 5. When the total number of dwell points is greater than or equal to the preset multi-dwell point threshold, the user's dwell point count characteristic is determined to be multi-dwell point type, indicating that the user has visited many different locations within the profile building time window and the activity range is relatively dispersed. Otherwise, the user's dwell point count characteristic is determined to be few-dwell point type, indicating that the locations visited by the user are relatively concentrated.

[0141] The average dwell time is compared with a preset long dwell time threshold, which ranges from 60 minutes to 180 minutes, preferably 120 minutes. When the average dwell time is greater than or equal to the preset long dwell time threshold, the user's dwell time characteristic is determined to be long dwell type, indicating that the user's average dwell time at a single dwell point is relatively long and the behavior pattern tends to involve staying at a fixed location for a relatively long time. Otherwise, the user's dwell time characteristic is determined to be short dwell type, indicating that the user's average dwell time at a single dwell point is relatively short and the behavior pattern tends to involve rapid turnover or frequent movement.

[0142] The dwell time characteristic label is formed by combining the number of dwell points and the dwell time characteristic. The combination method is to concatenate the judgment results of the above two dimensions with a hyphen. For example, the "multiple dwell time - long dwell time" label corresponds to the user behavior pattern of visiting many places and staying for a long time at each place. Typical scenarios are sales people, field inspection personnel, or tourists. The "multiple dwell time - short dwell time" label corresponds to the user behavior pattern of visiting many places but staying for a short time at each place. Typical scenarios are delivery personnel, ride-hailing drivers, or users who are just running errands. The "few dwell time - long dwell time" label corresponds to the user behavior pattern of visiting few places but staying for a long time at each place. Typical scenarios are office workers, home users, or long-term hospitalized users. The "few dwell time - short dwell time" label corresponds to the user behavior pattern of visiting few places and staying for a short time at each place. Typical scenarios are home users who occasionally go out to run errands or passersby who leave after a short stay.

[0143] For the movement frequency tag, the number of movement intervals between adjacent stop point groups in the location trajectory sequence is used as the movement count. The movement interval refers to the time period between the end position of one stop point group and the beginning position of the next stop point group. The number of movement intervals is counted by iterating through all stop point groups arranged in chronological order in the location trajectory sequence and calculating the number of movement intervals between adjacent groups. If there is only one stop point group in the location trajectory sequence, the movement count is 0.

[0144] The number of moves is compared with a preset high-frequency move threshold and a preset mid-frequency move threshold. The preset high-frequency move threshold ranges from 8 to 15, preferably 10. The preset mid-frequency move threshold ranges from 3 to 7, preferably 5. The above thresholds satisfy the relationship that the preset high-frequency move threshold is greater than the preset mid-frequency move threshold.

[0145] When the number of moves is greater than or equal to the preset high-frequency movement threshold, the user's movement frequency characteristic is judged as high-frequency movement type, indicating that the user has frequently changed locations within the profile building time window and has strong cross-regional activity ability. Typical scenarios are business field staff, logistics and delivery personnel, or users with long commuting distances.

[0146] When the number of moves is greater than or equal to the preset medium-frequency movement threshold and less than the preset high-frequency movement threshold, the user's movement frequency characteristic is determined to be medium-frequency movement type, indicating that the user has a certain number of location transfer behaviors, but has not yet reached the level of frequent movement. Typical scenarios are ordinary users who occasionally go out to run errands or users who commute regularly but do not go out every day.

[0147] When the number of moves is less than the preset medium-frequency movement threshold, the user's movement frequency characteristic is judged as low-frequency movement type, which means that the user has less location transfer behavior within the profile building time window and spends most of the time concentrated at a few stops. Typical scenarios are users working from home, students who stay in one place for a long time, or users with limited mobility.

[0148] The mobile frequency tag is registered directly based on the judgment result, and the registration format is high frequency mobile type, medium frequency mobile type or low frequency mobile type.

[0149] Figure 2 This is a schematic diagram of the user profile construction device for multimodal data fusion in this embodiment, including:

[0150] The window creation module is used to establish a profile building session, set the profile building time window, and register user identification information and multimodal data source configuration;

[0151] The sequence generation module is used to receive communication behavior data according to user identification information and construct communication behavior sequences respectively;

[0152] The correlation analysis module is used to align the communication behavior sequence with the user identification information as the merging key on the time axis in order to establish the temporal correlation between multimodal behaviors.

[0153] The feature extraction module is used to extract communication behavior features from the communication behavior sequence and its temporal correlation, forming a communication behavior feature vector set.

[0154] The collaborative verification module is used to perform collaborative verification on the communication behavior feature vector set based on temporal correlation, and to identify the degree of consistency and conflict between communication behavior features.

[0155] The feature evaluation module is used to evaluate the confidence level and resolve conflicts of communication behavior features based on the collaborative verification results of the communication behavior feature vector set.

[0156] The tag generation module is used to build a user profile tag system based on communication behavior characteristics that have undergone confidence assessment and conflict resolution, and generate profile tags for various dimensions.

[0157] Figure 3 This is a schematic diagram of the structure of an electronic device in this embodiment. The electronic device 60 in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), wearable electronic devices, etc., as well as fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Figure 3The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0158] like Figure 3 As shown, the electronic device 60 may include a processing unit 61, which can perform various appropriate actions and processes to implement the methods of the embodiments described herein, based on a program stored in ROM 62 or a program loaded from storage device 68 into RAM 63. RAM 63 also stores various programs and data required for the operation of the electronic device 60. The processing unit 61, ROM 62, and RAM 63 are interconnected via bus 64. I / O interface 65 is also connected to bus 64. Typically, the following devices can be connected to I / O interface 65: input devices 66 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 67 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 68 including, for example, magnetic tapes, hard disks, etc.; and communication devices 69. Communication device 69 allows the electronic device 60 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 60 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0159] Specifically, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts, thereby implementing the methods as described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 69, or installed from a storage device 68, or installed from a ROM 62. When the computer program is executed by the processing device 61, it performs the functions defined in the methods of the embodiments of the present invention.

[0160] Specifically, the computer-readable medium described in this embodiment may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0161] Specifically, the computer-readable medium described in this embodiment carries one or more programs, and when the electronic device executes the one or more programs, the electronic device performs the functions defined in the method of this embodiment.

[0162] In the above description, the disclosure of this invention is not intended to limit itself to these aspects. Rather, within the scope of the objectives of this disclosure, components can be selectively and operationally combined in any number. Furthermore, terms such as “comprising,” “encompassing,” and “having” should be interpreted by default as inclusive or open-ended, rather than exclusive or closed, unless explicitly defined as such. All technical, scientific, or other terms are to be understood by those skilled in the art, unless defined as such. Public terms found in dictionaries should not be interpreted too ideally or impractically in the context of the relevant technical documents, unless explicitly defined as such in this disclosure. Any modifications or alterations made by those skilled in the art based on the foregoing disclosure are within the scope of the claims.

Claims

1. A method for constructing user profiles through multimodal data fusion, characterized in that, include: Receive communication behavior data according to user identification information, and construct communication behavior sequences respectively; Using user identification information as the merge key, the communication behavior sequence is time-axis aligned to establish the temporal correlation between multimodal behaviors; Based on the communication behavior sequence and its temporal correlation, the communication behavior features of the communication behavior sequence are extracted to form a communication behavior feature vector set; Using temporal correlation as a constraint, a collaborative verification is performed on the feature vector set of communication behavior to identify the degree of consistency and conflict among the communication behavior features. Based on the collaborative verification results of the communication behavior feature vector set, confidence assessment and conflict resolution are performed on the communication behavior features. Based on communication behavior characteristics that have undergone confidence assessment and conflict resolution, a user profile tagging system is constructed to generate profile tags for each dimension.

2. The user profile construction method based on multimodal data fusion according to claim 1, characterized in that, The establishment of the temporal correlation includes two sub-steps: establishing synchronous correlation and establishing causal correlation. Synchronization association establishment: Extract the movement interval between two adjacent stop point groups in the location trajectory sequence; for each movement interval, find the call session records in the call behavior sequence whose call session start time or call session end time falls into the movement interval, and find the traffic usage records in the traffic usage sequence whose traffic usage start time or traffic usage end time falls into the movement interval, and establish a synchronization association mark between the call session records and traffic usage records falling into the same movement interval and the movement interval. Extract the dwell time interval for each dwell point group in the location trajectory sequence. For each dwell time interval, find call session records in the call behavior sequence where both the call session start time and the call session end time fall within the dwell time interval. Find SMS interaction cluster records in the SMS interaction sequence where both the SMS interaction cluster start time and the SMS interaction cluster end time fall within the dwell time interval. Find data usage records in the data usage sequence where both the data usage start time and the data usage end time fall within the dwell time interval. Establish dwell association tags with the call session records, SMS interaction cluster records, and data usage records that fall within the same dwell time interval and associate them with the dwell point group. Causal association establishment: For each call session record in the call behavior sequence, find the first SMS interaction cluster record or the first data usage record of the same user within the preset causal influence time window after the end time of the call session record. If found, establish a causal association mark between the call session record and the found SMS interaction cluster record or data usage record, and mark the causal direction as the call triggering subsequent behavior. For each SMS interaction cluster record in the SMS interaction sequence, find the first call session record or the first data usage record of the same user within the preset causal influence time window after the end time of the SMS interaction cluster record. If found, establish a causal association marker and mark the causal direction as SMS triggering subsequent behavior. For each data usage record in the data usage sequence, find the first call session record or the first SMS interaction record of the same user within the preset causal influence time window after the end time of the data usage record. If found, establish a causal association marker, and mark the causal direction as data usage triggering subsequent behavior.

3. The user profile construction method based on multimodal data fusion according to claim 1, characterized in that, Call behavior characteristics include call activity, call time preference, call recipient stability, and call location characteristics; SMS behavior characteristics include SMS activity, SMS direction characteristics, and SMS time preference. Traffic usage behavior characteristics include traffic usage volume, traffic continuity, preference for access network type, and preference for access service type; Location trajectory behavior characteristics include movement range, dwell point characteristics, and movement frequency.

4. The user profile construction method based on multimodal data fusion according to claim 1, characterized in that, The collaborative verification is based on synchronous association markers and causal association markers: For the collaborative verification of call behavior and location trajectory behavior: extract call session records and location trajectory movement ranges with synchronization association markers, extract call session records and stop point groups with stop association markers, and calculate the ratio of call duration to stop duration as the call density during the stop period; For the joint verification of call behavior and SMS behavior: compare the average response time from call to SMS to SMS and the average response time from SMS to call. If the former is less than the latter, it is recorded that the user is more likely to initiate SMS interaction after a call. Conversely, it is registered as if the user is more inclined to initiate a call after exchanging text messages; For the joint verification of traffic usage behavior and location trajectory behavior: compare the average value of mobile traffic density with the average value of stationary traffic density. If the former is greater than the latter, it is registered as the user's traffic usage intensity during the movement is higher than that during the stationary period.

5. The user profile construction method based on multimodal data fusion according to claim 4, characterized in that, For conflict identification: When there is a logical contradiction between the communication behavior characteristics of the same user within the same time interval, it is registered as a behavioral conflict. Conflict types include home location conflict, time period preference conflict and behavior pattern conflict. If the location of the other party's number in a call is inconsistent with the administrative division of the user's location during the call, and the number of times the inconsistency occurs exceeds the preset conflict tolerance threshold, it is registered as a location-location conflict. If the position of a user's main call time period and main data usage time period differs from the preset time period deviation threshold within a day, and this deviation pattern continues to occur for more than the preset number of consecutive days threshold, it is registered as a time period preference conflict. There is an inverse correlation between a user's SMS sending and receiving ratio and call type marking. For example, if the sending is dominant but the call is dominated by the called party, or if the receiving is dominant but the call is dominated by the called party, it is registered as a behavioral pattern conflict. All identified behavioral conflicts are registered in the conflict record table according to the conflict type and the time of occurrence.

6. The user profile construction method based on multimodal data fusion according to claim 1, characterized in that, For each communication behavior feature, its baseline confidence level is determined by the reliability level of its data source. The deviation level calculated in all collaborative verifications involving this communication behavior feature is read, and the average of all consistency scores is taken as the collaborative consistency score of this communication behavior feature. The collaborative consistency score is multiplied by the initial confidence level to obtain the collaboratively corrected confidence level. The number of conflict records involving this communication behavior feature in the conflict record table is read. Each conflict record reduces the confidence level by a preset conflict penalty value. The collaboratively corrected confidence level is subtracted from the total conflict penalty value to obtain the final confidence level.

7. The user profile construction method based on multimodal data fusion according to claim 6, characterized in that, After assessing the confidence level of the communication behavior characteristics, conflict resolution is performed: For location-home conflicts, the frequency distribution of the other party's home location in different locations is statistically analyzed. If a certain home location has the highest frequency in a certain location and is higher than other combinations, then that combination is used, and the home location information in other conflicting combinations is corrected to that home location. For time period preference conflicts, when the shortest time difference is greater than or equal to the preset extreme deviation threshold, the main call period and the main traffic usage period are registered as independent active periods, and a time period split mark is added to the user profile; When the shortest time difference is greater than or equal to the preset time period deviation threshold and less than the preset extreme deviation threshold, the two intervals on the 24-hour time axis of the main call period and the main traffic usage period are taken as endpoints, and the continuous time interval between the two intervals is registered as the user's comprehensive active extended period. Within the comprehensive active extended period, the occurrence distribution of call behavior and traffic usage behavior is statistically analyzed in half-hour increments, and the half-hour interval with the highest distribution density is registered as the comprehensive active peak period. For behavioral pattern conflicts, check whether there is a similar reverse correlation pattern in the historical profile records. If it exists and continues to exceed the preset historical persistence threshold, it is determined to be a stable feature and only a special feature is added. If it does not exist, check whether there are abnormal event records before and after the conflict.

8. The user profile construction method based on multimodal data fusion according to claim 1, characterized in that, The method for generating communication behavior dimension tags is as follows: For call activity tags, the call frequency is compared with a preset call frequency classification threshold, and the total call duration is compared with a preset call duration classification threshold, and the tags are generated by combining the results. For call time preference tags, register directly using the primary call time and secondary call time. The primary contact label and the contact network label are determined by the primary contact number and the number of frequent contacts; The method for generating spatiotemporal activity dimension labels is as follows: For activity range tags, the coverage area of ​​the movement range is compared with preset large range thresholds and preset medium range thresholds to generate wide-area / medium-area / local activity tags; For the dwell feature label, the total number of dwell points is compared with a preset multi-dwelling threshold to generate dwell point quantity features of multi-dwelling type and few-dwelling type; The average dwell time is compared with a preset long dwell threshold to generate dwell time features for long and short dwell types. The dwell feature label is formed by combining the dwell point quantity feature and the dwell time feature. The combination method is to concatenate the judgment results of the above two dimensions with hyphens. For the mobility frequency tag, the number of moves is compared with a preset high-frequency mobility threshold and a preset medium-frequency mobility threshold to generate high-frequency mobility type, medium-frequency mobility type, and low-frequency mobility type tags.

9. A user profile construction apparatus based on multimodal data fusion, applied to the user profile construction method based on multimodal data fusion as described in any one of claims 1-8, characterized in that, include: The window creation module is used to establish a profile building session, set the profile building time window, and register user identification information and multimodal data source configuration; The sequence generation module is used to receive communication behavior data according to user identification information and construct communication behavior sequences respectively; The correlation analysis module is used to align the communication behavior sequence with the user identification information as the merging key on the time axis in order to establish the temporal correlation between multimodal behaviors. The feature extraction module is used to extract communication behavior features from the communication behavior sequence and its temporal correlation, forming a communication behavior feature vector set. The collaborative verification module is used to perform collaborative verification on the communication behavior feature vector set based on temporal correlation, and to identify the degree of consistency and conflict between communication behavior features. The feature evaluation module is used to evaluate the confidence level and resolve conflicts of communication behavior features based on the collaborative verification results of the communication behavior feature vector set. The tag generation module is used to build a user profile tag system based on communication behavior characteristics that have undergone confidence assessment and conflict resolution, and generate profile tags for various dimensions.

10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-8.

Images