Personalized recommendation method and system based on large model

Abstract

The invention provides a personalized recommendation method and system based on a large model. According to the method, the content data stream is disassembled into independent feature packets in different modals through the multi-channel acquisition server; and analyzing a cross-modal semantic relationship among the feature packets by using a large model, and establishing a cross-modal similarity corresponding table. And generating a dynamic compensation coefficient through deviation analysis in combination with a cross-modal conversion record in a historical behavior. And constructing a bipartite graph structure of the content and the user, converting a compensation coefficient into a connection strength adjustment factor, and integrating weights of adjacent nodes to generate an expression vector. And finally, extracting a user purpose, content association and aging characteristic value of the vector, constructing a dynamic weight model in combination with a preference change track, integrating multiple characteristics to generate a decision vector, realizing cross-modal matching sorting, and outputting a personalized recommendation result. According to the method, the cross-modal data deviation is dynamically compensated and the user track is fused, so that high-precision matching recommendation of the multi-modal content and the user demand is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cross-modal large model recommendation methods, in particular to a personalized recommendation method and system based on a large model. BACKGROUND

[0002] In the multi-source heterogeneous content platform scenario, a large number of users continuously generate interactive content containing multiple modalities such as text, images, videos, and audio. These content sources are diverse and have significant structural differences, which poses a core demand for personalized recommendation systems: it must efficiently integrate multi-source heterogeneous data to build a unified representation, accurately understand the deep semantic associations between different modalities of content to bridge the cross-modal gap, and capture the dynamic changes in user interest preferences to address the cold start and interest drift problems, and ultimately achieve high-precision matching of content and user needs.

[0003] A current representative solution is a recommendation framework based on data augmentation of large language models combined with graph neural networks. The core process is to use a large language model to analyze user historical behavior and item text information, generate simulated user interaction records and fine-grained portrait labels to expand sparse data, and supplement missing attributes of items. Then, through a graph neural network, the relationship between users and items is modeled, and techniques such as noise pruning are used to improve the reliability of the generated data, and finally the recommendation results are output. This solution has shown some effect in improving coverage using text information.

[0004] Although the above-mentioned solution enhances the use of text data, it still has obvious limitations. The generated data may deviate from the platform's true distribution and introduce implicit noise, rely on offline batch processing mechanisms, and cause recommendation lag due to the inability to respond to the latest user dynamics in real time. The high cost of large model inference limits the scalability of large-scale item libraries, and lacks deep integration capabilities for non-text modalities such as image style or video dynamic features. Cross-modal alignment often relies on artificial rules, making it difficult to fully exploit the collaborative value of multi-source heterogeneous information. SUMMARY

[0005] The present application provides a personalized recommendation method and system based on a large model to solve the problem of insufficient dynamic compensation for cross-modal data bias, capturing user interest drift, and deep integration of multi-modal information in the prior art based on large model enhancement.

[0006] In a first aspect, the present application provides a personalized recommendation method based on a large model, comprising: Obtaining a content data stream including a mixture of multi-modal features, configuring a multi-channel acquisition server, using a hardware separation unit of the multi-channel acquisition server to disassemble the content data stream into original data streams of different modal features, and generating independent feature packages according to the corresponding feature data and labels of the original data streams; The semantic relationship between the text features and the picture features, and the matching degree between the sound features and the video content among multiple independent feature packages are analyzed by a pre-established large model to establish a cross-modal similarity correspondence table; The historical behavior records among different modal contents in the content data stream are determined, and a dynamic compensation coefficient is generated by bias source analysis based on the cross-modal conversion records in the historical behavior records and in combination with the similarity correspondence table; A bipartite graph structure of content nodes and user nodes is constructed, the dynamic compensation coefficient is converted into an adjustment factor of connection strength, the connection strength of the content nodes and the user nodes is adjusted, and an expression vector containing associated features is generated by integrating the associated weight of adjacent nodes; User purpose feature values, content contact feature values, and time effectiveness feature values in the expression vector are extracted, a dynamic change weight model is constructed in combination with the preference change trajectory of the historical behavior records, a decision vector is generated by integrating the user purpose feature values, the content contact feature values, the time effectiveness feature values, and the change weight values output by the dynamic change weight model, and a cross-modal matching sorting is performed based on the decision vector to output a personalized content recommendation result.

[0007] Optionally, the extraction of the user purpose feature values, the content contact feature values, and the time effectiveness feature values in the expression vector, the construction of the dynamic change weight model in combination with the preference change trajectory of the historical behavior records, the generation of the decision vector by integrating the user purpose feature values, the content contact feature values, the time effectiveness feature values, and the change weight values output by the dynamic change weight model, and the cross-modal matching sorting based on the decision vector to output the personalized content recommendation result, comprises: The user purpose feature values, the content contact feature values, and the time effectiveness feature values are separated from the expression vector, and an operation time sequence is extracted based on the preference change trajectory of the historical behavior records, and the latest operation time and operation frequency are extracted from the operation time sequence; The dynamic change weight model is constructed based on the latest operation time and operation frequency, and the change weight value is calculated; For each content item, the user purpose feature values, the content contact feature values, the time effectiveness feature values, and the change weight values are calculated to generate decision values of the corresponding content items, and all decision values are organized into a decision vector according to the content item identifiers of different content items; The decision values of all content items in the decision vector are cross-modally uniformly sorted, and the content item identifier with the highest sorting is output as the personalized recommendation result.

[0008] Optionally, for each content item, the user purpose feature value, the content contact feature value, and the time effectiveness feature value are respectively calculated with the change weight value to generate a decision value of the corresponding content item, and all decision values are organized into a decision vector according to content item identifiers of different content items, including: extracting a user purpose feature value, a content contact feature value, and a time effectiveness feature value corresponding to the current content item; multiplying the user purpose feature value by the change weight value to obtain a purpose weighted value, multiplying the content contact feature value by the change weight value to obtain a contact weighted value, and multiplying the time effectiveness feature value by the change weight value to obtain a time effectiveness weighted value; adding the purpose weighted value, the contact weighted value, and the time effectiveness weighted value to generate a decision value of the current content item; associating the decision value with a content item identifier of the current content item, and arranging all decision values in a decision vector according to the content item identifiers after traversing all content items.

[0009] Optionally, the bipartite graph structure of the content node and the user node is constructed, the dynamic compensation coefficient is converted into an adjustment factor of connection strength, the connection strength of the content node and the user node is adjusted, and an expression vector containing association features is generated by integrating association weights of adjacent nodes, including: constructing a bipartite graph structure composed of content nodes and user nodes, and assigning an initial connection strength value to a connection line between each content node and the user node based on the number of historical operations of the user on the content node; converting the dynamic compensation coefficient into an adjustment factor of connection strength, and performing an operation on the initial connection strength value to obtain an adjusted connection strength value; for each user node, obtaining the adjusted connection strength value and the association weight of its adjacent content node, performing weighted processing on the connection strength value based on the association weight, generating an association feature value of the corresponding user node, and arranging the association feature values of all user nodes in a node order to form an expression vector.

[0010] Optionally, the historical behavior records between different modal contents in the content data stream are determined, and a dynamic compensation coefficient is generated by performing bias source analysis on the cross-modal conversion records in the historical behavior records in combination with the similarity corresponding table, including: reading historical behavior records between different modal contents in the content data stream from a storage system; statistically obtaining a conversion number ratio of text content to picture content as a text-picture conversion value, and statistically obtaining a conversion number ratio of sound content to video content as a sound-video conversion value; extracting a semantic relationship score and a matching degree score in the similarity correspondence table, and calculating a first deviation amount of the text-picture conversion value and the semantic relationship score and a second deviation amount of the sound-video conversion value and the matching degree score; generating a dynamic compensation coefficient positively related to the deviation amount according to the size range of the first deviation amount and the second deviation amount.

[0011] Optionally, the semantic relationship between the text features and the picture features and the matching degree between the sound features and the video content are analyzed among the multiple independent feature packages through the pre-established large model to establish a cross-modal similarity correspondence table, including: According to the independent feature package, the text features in the text feature package and the picture features in the picture feature package are input into the large model, the content consistency of the text features and the picture features is compared, and the semantic relationship score of each text feature and the corresponding picture feature is calculated; The sound features in the sound feature package and the video content in the video content feature package are input into the large model, the time alignment state of the sound features and the video content is detected, and the matching degree score of each sound feature and the corresponding video content is calculated; The semantic relationship score is recorded as a text-picture relationship item according to the combination of the text feature package identifier and the picture feature package identifier, and the matching degree score is recorded as a sound-video relationship item according to the combination of the sound feature package identifier and the video content feature package identifier, all the text-picture relationship items and the sound-video relationship items are integrated, and a similarity correspondence table containing identifier combinations and relationship scores is constructed.

[0012] Optionally, the content data stream including multi-modal feature mixing is obtained, a multi-channel acquisition server is configured, the content data stream is disassembled into original data streams of different modal features by using the hardware separation unit of the multi-channel acquisition server, and independent feature packages are generated according to the corresponding feature data and identifiers of the original data streams, including: Obtaining a content data stream containing text, pictures, sound and video; Activating multiple physical channels of the hardware separation unit of the multi-channel acquisition server to split the content data stream, and generating original data streams containing text stream, picture stream, sound stream and video stream; Extracting character sequences and position information from the text stream, extracting color distribution and contour point set from the picture stream, extracting frequency band and loudness sequence from the sound stream, and extracting motion trajectory and brightness change from the video stream to extract the corresponding feature data of each original data stream; The original data stream corresponding to the feature data is appended with an identifier containing a modal type code and a timestamp, and packaged to generate independent feature packages of corresponding text feature packages, picture feature packages, sound feature packages and video feature packages.

[0013] In a second aspect, the present application provides a personalized recommendation system based on a large model, comprising: The acquisition module is configured to acquire a content data stream including mixed multi-modal features, and to use a hardware separation unit of a multi-channel acquisition server to separate the content data stream into original data streams of different modal features, and to generate independent feature packages according to the feature data and identifiers corresponding to the original data streams. The analysis module is configured to analyze the semantic relationship between text features and picture features, and the matching degree between sound features and video content, between a plurality of independent feature packages, by using a pre-established large model, to establish a similarity correspondence table across modalities. The generation module is configured to determine historical behavior records between different modal content in the content data stream, and to analyze the bias source to generate a dynamic compensation coefficient in combination with the similarity correspondence table according to the cross-modal conversion records in the historical behavior records. The adjustment module is configured to construct a bipartite graph structure of content nodes and user nodes, to convert the dynamic compensation coefficient into an adjustment factor of connection strength, to adjust the connection strength of the content nodes and user nodes, and to integrate the correlation weight of adjacent nodes to generate an expression vector including correlation features. The output module is configured to extract user purpose feature values, content contact feature values and time efficiency feature values in the expression vector, to construct a dynamic change weight model in combination with the preference change trajectory of the historical behavior records, to integrate the user purpose feature values, content contact feature values, time efficiency feature values and change weight values output by the dynamic change weight model to generate a decision vector, and to perform cross-modal matching sorting based on the decision vector to output a personalized content recommendation result.

[0014] In a third aspect, the present application provides a computing device comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a personalized recommendation method based on a large model as described in the first aspect above.

[0015] In a fourth aspect, the present application provides a computer storage medium storing a computer program, which, when executed by a computer, implements a personalized recommendation method based on a large model as described in the first aspect.

[0016] In the examples of the present application, a content data stream including multi-modal feature mixing is obtained, a multi-channel acquisition server is configured, the content data stream is disassembled into original data streams of different modal features by using a hardware separation unit of the multi-channel acquisition server, and independent feature packages are generated according to corresponding feature data and labels of the original data streams; a large model established in advance is used to analyze the semantic relationship between text features and picture features and the matching degree between sound features and video content among a plurality of independent feature packages, so as to establish a similarity correspondence table across modalities; historical behavior records among different modal contents in the content data stream are determined, and a dynamic compensation coefficient is generated by combining the similarity correspondence table and performing bias source analysis according to cross-modal conversion records in the historical behavior records; a bipartite graph structure of content nodes and user nodes is constructed, the dynamic compensation coefficient is converted into an adjustment factor of connection strength, the connection strength of the content nodes and the user nodes is adjusted, and an associated weight of adjacent nodes is integrated to generate an expression vector containing associated features; user purpose feature values, content contact feature values and time effectiveness feature values in the expression vector are extracted, a dynamic change weight model is constructed by combining a preference change trajectory of the historical behavior records, a decision vector is generated by integrating the user purpose feature values, the content contact feature values, the time effectiveness feature values and change weight values output by the dynamic change weight model, and cross-modal matching sorting is performed based on the decision vector to output an individualized content recommendation result.

[0017] The technical solution of the present application has the following beneficial effects: The present application efficiently disassembles multi-modal content data streams by multi-channel hardware separation to generate independent feature packages, uses a large model to establish a similarity correspondence table of cross-modal semantic relationships, analyzes bias sources by combining cross-modal conversion records in historical behaviors to generate a dynamic compensation coefficient, and then drives adaptive adjustment of connection strength between content nodes and user nodes in a bipartite graph to construct an associated expression vector. Finally, the user purpose, content association, time effectiveness features and change weight values output by the dynamic weight model are integrated to generate a decision vector, high-precision matching sorting of cross-modal content and user dynamic interests is realized, and the accuracy and timeliness of individualized recommendation are significantly improved.

[0018] Further, after constructing the expression vector, the user purpose feature value, the content association feature value and the time-sensitive feature value are separated therefrom, and the operation time sequence is extracted based on the historical behavior record and the recent operation time and operation frequency are parsed; the dynamic change weight model is constructed using the recent operation time and operation frequency and the change weight value is generated; for each content item, the user purpose feature value, the content association feature value, the time-sensitive feature value are respectively weighted with the change weight value to generate the content item exclusive decision value, and the decision vector is organized according to the content identifier; finally, the decision vector is cross-modal unified sorting, and the optimal content item identifier is output as the recommendation result. The scheme dynamically captures the user's recent operation time and frequency to construct a weight model, integrates multi-dimensional features and dynamic weights to generate content item exclusive decision values, realizes unified quantitative sorting of cross-modal content, significantly improves the response speed and accuracy of the recommendation result to the user's instantaneous interest, and at the same time, alleviates the problem of insufficient exposure of long-tail content.

[0019] These aspects or other aspects of the present application will be more apparent in the following description of the embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0021] Figure 1 A flowchart of a personalized recommendation method based on a large model provided by the present application is shown; Figure 2 A scene diagram of a personalized recommendation method based on a large model provided by the present application is shown; Figure 3 A structural schematic diagram of a personalized recommendation system based on a large model provided by the present application is shown; Figure 4 A structural schematic diagram of a computing device provided by the present application is shown. DETAILED DESCRIPTION

[0022] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application.

[0023] In some of the flowcharts described in the description and claims of the present application and in the above-described figures, a plurality of operations are included which occur in a particular order, but it should be clearly understood that the operations can be performed in the order in which they appear in this document or in parallel, and the serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these flowcharts can include more or fewer operations, and the operations can be performed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do "first" and "second" represent different types.

[0024] Research shows that in the field of cross-modal large model recommendation, the existing large model recommendation scheme has three limitations: first, relying on offline generated pseudo data is easy to deviate from the real distribution, leading to distorted recommendations; second, the static processing mechanism cannot respond to changes in user behavior, causing recommendation lag when the user's instantaneous interest drifts; third, the shallow fusion of non-text modalities leads to a split of cross-modal semantics, and the value of multi-source information is not fully exploited. These defects are rooted in the lack of dynamic cross-modal alignment, interest tracking, and lightweight multi-modal collaboration capabilities.

[0025] To solve the above problems, the present application proposes a personalized recommendation method based on a large model, which separates and disassembles content data streams through hardware-level multi-channel to generate independent feature packages, and uses a large model to construct a cross-modal semantic association table; a dynamic compensation coefficient is generated by combining historical behavior to drive adaptive adjustment of the connection strength of the bipartite graph nodes; a decision vector is generated by fusing user purpose, content association, time effectiveness features, and dynamic weights to realize cross-modal unified sorting. This method eliminates data bias through hardware processing, dynamically compensates for the modal gap, and synchronously captures long-term and short-term interests through lightweight fusion, fundamentally solving the problems of distorted recommendations, lag, and modal splitting, and significantly improving accuracy and timeliness.

[0026] The technical solutions in the embodiments of the present application will be described clearly and completely in the embodiments of the present application in combination with the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0027] Figure 1 A flowchart of a personalized recommendation method based on a large model is provided for the embodiments of the present application, as shown in Figure 1 The method comprises the following steps. 101、Obtaining a content data stream including multi-modal feature mix, by configuring a multi-channel acquisition server, using a hardware separation unit of the multi-channel acquisition server to disassemble the content data stream into original data streams of different modal features, and generating independent feature packages according to the corresponding feature data and identification of the original data streams; Optionally, step 101 can specifically include the following steps: 1011、Obtaining a content data stream including text, pictures, sound and video; 1012、Activating a plurality of physical channels of the hardware separation unit of the configured multi-channel acquisition server to split the content data stream, and generating original data streams including text stream, picture stream, sound stream and video stream; 1013、Extracting character sequence and position information from the text stream, color distribution and contour point set from the picture stream, frequency band and loudness sequence from the sound stream, and motion trajectory and brightness change from the video stream to extract the corresponding feature data of each original data stream; 1014、Attaching the identification including modal type code and time stamp to the corresponding feature data of the original data stream, and packaging to generate independent feature packages of the corresponding text feature package, picture feature package, sound feature package and video feature package.

[0028] In the above scheme, the content data stream refers to a complex digital content carrier generated by user interaction behavior, including synchronous or asynchronous combination of text description, static image, audio waveform and dynamic video picture, which can be used to represent the complete semantic expression of cross-media information. The multi-channel acquisition server refers to a computing device configured with a dedicated hardware interface, which can be used to disassemble the mixed data stream by physical isolation. The hardware separation unit refers to a physical signal decoupling module integrated in the server, which can be used to disassemble the composite data stream into single modal basic stream according to the media type. The original data stream refers to the single modal basic data sequence generated by hardware separation, including pure text character stream, uncompressed image frame sequence, original sound wave sampling sequence or de-audio video frame sequence, which can be used to extract the essential features of a specific medium. The feature data refers to the quantitative index set extracted from the original data stream, including the position sequence of the text, the visual attributes of the image, the physical parameters of the sound wave and the dynamic change trajectory of the video, which can be used to represent the essential attributes of each modal content. The independent feature package refers to the structured encapsulated feature data set, including the feature data of a specific modal and its spatiotemporal identification metadata, which can be used to guarantee the data traceability and temporal consistency in cross-modal analysis.

[0029] In the embodiment of the application, first, the content data stream including text, pictures, sound and video submitted by the user is obtained through step 1011 at the platform data interface.

[0030] Subsequent step 1012 activates the four dedicated physical channels by the hardware separation unit of the multi-channel acquisition server to implement synchronous disassembly processing: the text channel uses a character signal filter based on regular matching to strip non-text noise and output a pure character sequence stream; the picture channel uses a trigger type capture circuit based on an interframe difference algorithm to identify key static frames and generate an image stream without dynamic redundancy; the sound channel uses a Fourier transform noise reduction chip to separate the target voiceprint and output a pure sound wave sampling sequence; and the video channel uses a motion vector detector to filter still pictures and generate a motion stream without background interference. This process achieves four-stream parallel processing at the hardware layer to ensure the integrity of each modality data in a physically isolated manner, and the time consumption is controlled within 80 milliseconds to complete the disassembly of a gigabit-level data stream.

[0031] Then, starting from the original data stream generated by hardware separation, step 1013 performs an OCR positioning algorithm on the text stream, scans the character sequence and records the space-time coordinates, for example, identifying the "limited-time offer" text at the [120, 80] position of the video at the 15th second; applying edge detection and color histogram analysis to the picture stream to extract object contour point sets and dominant color tone ratios, for example, calculating that the red area in the product screenshot accounts for 65% and marking 200 boundary points of the mobile phone frame; performing short-time Fourier transform on the sound stream to decompose the frequency band and draw the loudness change curve, for example, separating the 300-500Hz human voice frequency band and recording that the volume of "buy immediately" suddenly rises to 85dB; using a dense optical flow tracking algorithm on the video stream to capture the motion trajectory and interframe brightness difference, for example, tracking the coordinate sequence of the demonstration gesture movement path and the light brightness transition value when the product is unpacked; then, after normalization coding, all feature data is output as a structured feature set to ensure the spatiotemporal consistency of cross-modality features.

[0032] Finally, step 1014 adds a modality type code and a millisecond-level timestamp dual identifier to each modality feature data, marks the text feature as "TXT", the picture as "IMG", the sound as "AUD", and the video as "VID" and gives a timestamp accurate to one thousandth of a second, such as "20230815153025.456"; then, structured packaging is performed to bind the character sequence and position information of the text stream into a text feature package, integrate the color distribution and contour point set of the picture stream into an image feature package, package the frequency band and loudness sequence of the sound stream into an audio feature package, and construct the motion trajectory and brightness change of the video stream into a video feature package; the data and identifier in each feature package are associated and stored through a binary serialization protocol to generate four standardized independent file packages, ensuring that cross-modality data can be aligned at the millisecond level through the timestamp. For example, the motion trajectory data pointer in the video package points to the "VID_20230815104533.456" identifier.

[0033] In practical applications, an online education platform receives a mixed data stream of "Chemical Experiment Operation Guide" uploaded by a user, which includes experimental step text instructions containing the instruction "heat to 80°C and stop", a static photo of a reagent bottle, a safety voice prompt "wear goggles", and a solution color change process video; the multi-channel acquisition server activates the hardware separation unit, the text channel extracts the pure text stream by stripping image noise through the ASCII filter, outputs characters such as "heat to 80°C and stop", the picture channel generates an image stream by setting the frame difference threshold algorithm to 5% pixels when the continuous frame difference is greater than 5% pixels, the sound channel uses a 200-400Hz human voice range bandpass filter to eliminate environmental current sound and output a pure audio stream of the "goggles" sound wave, and the video channel filters the fixed background based on the motion vector detection to retain the dynamic color change stream of the solution; then feature extraction is performed, the text stream records "80°C" at the [300, 150] coordinate position of the 120th second of the video through OCR spatial positioning, the picture stream counts the blue pixel ratio of 65% in the reagent bottle screenshot through HSV color analysis, the sound stream measures the "goggles" vocabulary volume peak value of 75dB which is greater than the reference volume of 60dB by using the sound pressure calculation method, and the video stream captures the brightness jump of the solution color change moment by using the brightness difference method, the brightness average of the 135th second frame is 120 lumens, and the brightness average of the 136th second frame is 150 lumens, which is an increase of 30 lumens compared with the 135th second frame; finally, four independent feature packages are packaged: the text package binds the instruction characters + space-time coordinates + identifier "TXT_20230901103000.120", the picture package encapsulates the 65% blue ratio data + identifier "IMG_20230901103000.121", the sound package integrates the 75dB peak value + identifier "AUD_20230901103000.122", and the video package associates the 30-lumen brightness difference + identifier "VID_20230901103000.123", and the four independent feature packages are time-stamped at the millisecond level to ensure accurate alignment across modalities.

[0034] The overall scheme of 101 above converts complex mixed data into standardized independent feature packages through hardware-level disassembly and feature extraction, providing structured high-quality input for subsequent cross-modal analysis, while ensuring accurate alignment and traceability of different types of data.

[0035] 102. Analyze the semantic relationship between the text features and the picture features, and the matching degree between the sound features and the video content of the multiple independent feature packages through a pre-established large model, to establish a cross-modal similarity correspondence table; Optionally, step 102 can specifically include the following steps: 1021. According to the independent feature package, input the text features in the text feature package and the picture features in the picture feature package into the large model, compare the content consistency of the text features and the picture features, and calculate the semantic relationship score of each text feature and the corresponding picture feature; 1022. Input the sound features in the sound feature package and the video content in the video content feature package into the large model, detect the time alignment between the sound features and the video content, and calculate the matching score between each sound feature and the corresponding video content; 1023. Record the semantic relationship score as a text-image relationship item according to the combination of the text feature package identifier and the image feature package identifier, record the matching degree score as a sound-video relationship item according to the combination of the sound feature package identifier and the video content feature package identifier, integrate all the text-image relationship items and the sound-video relationship items, and construct a similarity correspondence table containing identifier combinations and relationship scores.

[0036] In the above scheme, the semantic relationship score is a quantitative indicator that reflects the degree of consistency between the text description and the image content. It contains the matching strength information between the text semantic vector and the visual feature vector, and can be used to evaluate the quality of semantic alignment between cross-modal content. The matching degree score is a measure that characterizes the spatiotemporal synchronization between sound features and video content. It contains the matching accuracy information between the audio event timestamp and the video action frame sequence, and can be used to detect the authenticity of the audio-visual collaborative performance. The similarity correspondence table is a global mapping database that integrates the correlation between multimodal features. It contains the spatial topological structure information of all text-image relationship items and audio-video relationship items, and can be used to drive the decision engine for cross-modal content collaborative analysis.

[0037] In this embodiment of the present application, first, step 1021 is used to input the text semantic vectors in the text feature package and the visual vectors in the corresponding image feature package into a pre-trained large model. The model then uses a cross-modal attention mechanism to align text and image regions on a segment-by-segment basis: it extracts key concepts from the text and calculates the semantic overlap between each text segment and the relevant image region. A cosine similarity weighted algorithm is then used to synthesize all segment alignment results to generate an overall semantic relationship score. For example, when the text describes a "red surfboard," the model detects the color features of the surfboard in the image. If the region's color value matches the red spectrum, such as RGB (220, 20, 60), it is assigned a high score, ultimately outputting a semantic relationship score of 0.93 for this image-text combination.

[0038] Secondly, the sound feature package and the video content package processed in step 1021 are input into the same large model through step 1022. The model aligns the time axis through the dynamic time warping algorithm, extracts the timestamp of the audio event and the video action frame, and calculates the overlap ratio of the two time windows. At the same time, the sound and picture consistency detection module is used to verify the content matching degree, such as whether the laughter sound corresponds to the crowd mouth opening action. Finally, the matching degree score is generated according to the time deviation and content coincidence degree. For example, when the video shows the frame sequence 5.1-5.4 seconds corresponding to the action of the person holding the cup, if the audio appears the glass collision sound at 5.15 seconds, the model determines that the time deviation is <0.2 seconds and the action sound source is consistent, and outputs the matching score 0.88.

[0039] Finally, the semantic relationship score generated in step 1021 and its corresponding text feature package identifier and picture feature package identifier are received through step 1023, and the three are bound as a structured relationship unit. For example, for the text package identifier T07 and the main picture package identifier P12 of the e-commerce platform, according to the semantic relationship score 0.93 calculated in step 1021, the "text-picture" relationship item is generated, and the complete record is "identifier combination T07_P12, semantic relationship score 0.93". The matching degree score and its associated sound feature package identifier and video content package identifier output by step 1022 are received synchronously, such as the matching score 0.88 of the product description audio package identifier S09 and the use demonstration video package identifier V15, and the "sound-video" relationship item "identifier combination S09_V15, matching degree score 0.88" is generated. Finally, through the data aggregation engine, all the above relationship items are integrated in a unified format: a unique index key is assigned to each relationship item, and a globally queryable similarity correspondence table is constructed, for example, the text picture item is stored with the "TP_" prefix number, and the sound video item is stored with the "SV_" prefix number.

[0040] In practical applications, when a certain short video platform recommends popular science content to users, the system obtains a mixed data stream of a wildlife documentary. The text feature bag contains the semantic vector of the commentary "Cheetah accelerates and the spine stretches to provide explosive power", and the corresponding picture feature bag is the visual vector of the continuous shooting pictures of the cheetah running. The large model decomposes the text into "cheetah subject", "spine stretching" and "explosive power" three segments, respectively matches the cheetah contour area, slow motion spine deformation frame and dust special effect area in the picture, and detects the matching degree as 0.97, 0.88 and 0.91 respectively. The weighted average calculation of the semantic relationship score is 0.92. Then process the audio and video of the same documentary: the sound feature bag contains the spectrum of cheetah panting and paw stepping sound, and the video content bag contains 20 seconds of key frames of cheetah sprinting. The model detects that the time deviation between the paw touching the ground picture at 5.3 seconds of the video and the impact sound at 5.28 seconds of the audio is 0.02 seconds, and the time deviation between the panting picture at 8.1 seconds and the breathing sound at 8.09 seconds is 0.01 seconds. The average deviation is 0.015 seconds, which is substituted into the matching formula to convert the matching degree score to 0.87. Finally, the semantic relationship score of the text feature bag and the picture feature bag 0.92 is bound as a relationship item, and the matching score of the sound feature bag and the video feature bag 0.87 is bound as another relationship item, which is integrated into the similarity corresponding table. When the user browses the "animal movement mechanism" picture text, the system associates the high matching degree video segment according to the table to trigger the personalized recommendation of the cheetah sprinting video.

[0041] The overall scheme of the above 102 converts the complex correlation between cross-modal content into a quantifiable similarity corresponding table by deeply analyzing the semantic consistency of text and pictures and the spatio-temporal synchronization of sound and video by a large model. This table accurately depicts the matching strength between multi-modal features, provides a key basis for subsequent dynamic compensation of cross-modal deviation, significantly improves the system's ability to understand the association between picture text and sound picture, lays a solid foundation for accurate matching of multi-source heterogeneous content in personalized recommendation, and ultimately drives the simultaneous optimization of recommendation accuracy and timeliness.

[0042] 103、Determine the historical behavior record between different modal content in the content data stream, and generate a dynamic compensation coefficient by analyzing the source of deviation according to the cross-modal conversion record in the historical behavior record and combining the similarity corresponding table; Optionally, step 103 can specifically include the following steps: 1031、Read the historical behavior record between different modal content in the content data stream from the storage system; 1032、Statistically count the proportion of the number of times of conversion from text content to picture content in the historical behavior record as the text picture conversion value, and statistically count the proportion of the number of times of conversion from sound content to video content as the sound video conversion value; 1033、extract the semantic relationship score and the matching degree score in the similarity correspondence table, calculate a first deviation amount of the text-picture conversion value and the semantic relationship score and a second deviation amount of the sound-video conversion value and the matching degree score; 1034、generate a dynamic compensation coefficient positively correlated with the deviation amount according to the size range of the first deviation amount and the second deviation amount.

[0043] In the above scheme, the historical behavior record refers to the interactive operation log of the user with different modal content such as text, picture, sound, and video, including browsing time, click frequency, and content switching behavior, which is used to restore the content consumption habits of the user. The cross-modal conversion record specifically refers to the statistical proportion of the user switching from one content modality to another, reflecting the user's cross-modal interest migration tendency. The similarity correspondence table is a correlation strength mapping table generated by a large model, recording the semantic correlation score of text and picture and the content matching degree score of sound and video, serving as a reference for cross-modal association relationship. The deviation source analysis is to compare the actual switching proportion of the user with the predicted association score of the large model to locate the deviation direction of the system's understanding of the user's interest. The dynamic compensation coefficient is a correction weight value calculated according to the deviation amount, and the larger the deviation, the higher the coefficient, which is used to enhance the underestimated cross-modal association relationship in subsequent recommendations.

[0044] In the embodiments of the present application, first, the historical behavior original log of the target user is extracted from the storage database through step 1031. These logs record the interactive events of the user with different modal content such as text, picture, sound, and video in chronological order. The structured query language is used to filter out the key fields, including content identifier, modality type, operation type, and timestamp, to form an operation sequence table sorted by time. For example, user A has generated 150 behavior records in the past week, of which the 5th record is "click text report ID123" and the 32nd record is "switch from text report ID123 to picture set ID456".

[0045] Next, based on the operation sequence table generated in step 1031, all adjacent operation records are traversed to identify the nodes of modality type change. For the text-to-picture conversion scenario, the number of "text content followed by operation picture content" is counted, and then divided by the total number of cross-modal switching to calculate the text-picture conversion value. For example, user A has 20 such conversions in the 150 records, and the total number of cross-modal switching is 50, so the text-picture conversion value is The sound-video conversion value is calculated in the same way. For example, user A has 10 such conversions in the 150 records, and the total number of cross-modal switching is 50, so the sound-video conversion value is Finally, the two key indicators, text-picture conversion value and sound-video conversion value, are output.

[0046] Then, the pre-generated similarity table of cross-modal correlation strength analyzed by the stored large model is called by step 1033. The text-picture semantic relationship score and the sound-video matching degree score are extracted. The output value of step 1032 is compared with the corresponding score to calculate the first deviation and the second deviation, and the calculation formula is as follows: For example, when the text-picture conversion value of user B is 0.3, and the text-picture semantic relationship score in the similarity table is 0.8, the system calculates as the first deviation; at the same time, the absolute difference value between the sound-video conversion value 0.2 and the matching degree score 0.5 is calculated as the second deviation. These two deviations directly quantify the error direction and amplitude of the system's understanding of the user's cross-modal interest, providing a correction basis for subsequent dynamic compensation.

[0047] Finally, the first deviation and the second deviation output by step 1033 are received by step 1034, and the two are added to obtain the total deviation. Then, a dynamic compensation coefficient is generated according to the pre-set compensation rule mapping table, which sets that the larger the total deviation, the higher the compensation coefficient. This is achieved by a piecewise function: when the total deviation is less than 0.4, a linear function is used: when the total deviation is in the interval of 0.4 to 0.6, the following is used: when the total deviation exceeds 0.6, an enhanced compensation function is used: This process ensures that the compensation coefficient is always in the range of 0.3~0.9, and is strictly positively related to the deviation between the user's actual cross-modal behavior and the system's prediction. The final generated coefficient will directly affect the adjustment proportion of the subsequent bipartite graph connection strength. For example, the text-image deviation of user C is 0.4 and the audio-video deviation is 0.3, which adds up to a total deviation of 0.7 greater than 0.6. Substituting into the enhanced compensation function, , which means that the system will increase the cross-modal content correlation weight underestimated in subsequent recommendations by 78%.

[0048] ​In practical applications, in the knowledge sharing platform E, the system reads 200 behavior records of the user F in the past 30 days from the storage log of the user F, including 80 cross-modal switching operations, traverses the operation sequence to identify specific conversion behaviors: statistics show that the switching of text to picture occurs 32 times, such as jumping from a programming tutorial text to a code diagram, and the switching of sound to video occurs 16 times, such as jumping from an algorithm explanation audio to a demonstration video, thus calculating the text picture conversion value as 32 divided by the total switching times 80 to obtain the text picture conversion value as 0.4, and the sound video conversion value as 16 divided by 80 to obtain the sound video conversion value as 0.2; then calling the pre-stored similarity corresponding table, extracting the text picture semantic relationship score 0.75 and the sound video matching degree score 0.55, calculating the first deviation as 0.4 minus 0.75 to obtain 0.35, and the second deviation as 0.2 minus 0.55 to obtain 0.35; finally, adding the two deviation amounts to obtain the total deviation amount 0.7, according to the preset rule "total deviation amount ≥ 0.6, compensation coefficient = 0.4 × total deviation amount + 0.5", substituting 0.4 × 0.7 + 0.5 = 0.78 into the calculation, to generate a dynamic compensation coefficient 0.78 for correcting the content recommendation weight of the user F in the future.

[0049] The overall scheme of the above 103 quantifies the difference between the actual cross-modal switching behavior of the user and the prediction correlation strength of the large model, dynamically generates a compensation coefficient, and corrects the understanding deviation of the recommendation system for the content correlation relationship of text, audio and video, etc. The compensation mechanism driven by real behavior data not only eliminates the distortion of the recommendation caused by the static prediction of the model, but also responds to the user interest shift in real time, simultaneously bridges the semantic gap between different modalities, and significantly improves the matching accuracy and timeliness of cross-modal content and user dynamic needs.

[0050] 104, build a bipartite graph structure of content nodes and user nodes, convert the dynamic compensation coefficient into an adjustment factor of connection strength, adjust the connection strength between the content nodes and the user nodes, and integrate the correlation weight of adjacent nodes to generate an expression vector containing correlation features; Optionally, step 104 can specifically include the following steps: 1041, build a bipartite graph structure composed of content nodes and user nodes, and assign an initial connection strength value to the connection line between each content node and the user node based on the historical operation times of the user on the content node; 1042, convert the dynamic compensation coefficient into an adjustment factor of connection strength, and perform operation on the initial connection strength value to obtain an adjusted connection strength value; 1043、For each of the user nodes, obtain the adjusted connection strength value and the association weight of its adjacent content nodes, weight process the connection strength value based on the association weight, generate the association feature value of the corresponding user node, and arrange the association feature values of all user nodes in node order to form an expression vector.

[0051] In the above scheme, the dynamic compensation coefficient refers to a quantitative value reflecting the cross-modal data deviation correction requirement, derived from the deviation analysis result of the cross-modal conversion record and the similarity correspondence table in the user historical behavior, used to adjust the association strength of the user node and the content node. The expression vector refers to a structured digital sequence representing the user's comprehensive interest distribution, composed of the association feature values of all user nodes arranged in order, used to drive the unified ranking decision of cross-modal content. The association feature value refers to the interest quantitative indicator of a single user node, generated by weighting and aggregating the adjusted connection strength of its adjacent content nodes, used to build a dynamic interest portrait in the user dimension. The adjustment factor refers to the dynamic scaling coefficient of the connection strength, directly transformed from the dynamic compensation coefficient, containing the cross-modal deviation correction direction and correction amplitude information, used to adaptively update the association relationship between users and content in the bipartite graph. The association weight refers to the inherent importance parameter of the content node, statically set based on the content attribute, containing the popularity decay coefficient and the long tail value gain factor, used to balance the contribution proportion of hot content and cold content in the interest representation.

[0052] In the embodiments of the present application, first, all users of the platform are mapped into independent user nodes by step 1041, and all content items are mapped into independent content nodes, forming the initial set of two types of nodes. When a user has historical operation behavior on a content item, the system establishes a connection line between the corresponding user node and the content node. Then, the total number of historical operations of the user on the content item corresponding to each connection line is counted, and the number is multiplied by the preset base weight coefficient 0.2 to obtain the initial connection strength value. Finally, the bipartite graph network structure containing user nodes, content nodes and connection lines with strength labels is constructed.

[0053] Next, the dynamic compensation coefficient generated by step 103 is received as an adjustment factor by step 1042, and all connection lines between user nodes and content nodes in the bipartite graph are traversed. For each connection line, the initial connection strength value calculated in step 1041 is read, and the value is scaled by multiplying it by the adjustment factor to generate an updated connection strength value and replace the original value. This process completes the correction of the association strength between all network nodes, making the bipartite graph relationship dynamically adapt to changes in user interest. For example, when the cross-modal behavior of user U1 triggers the dynamic compensation coefficient 1.1, the system traverses its connection lines: the original U1-V1 line strength 0.6 multiplied by 1.1 gives a new strength 0.66, and the original U1-V2 line strength 0.2 multiplied by 1.1 gives a new strength 0.22. The updated bipartite graph reflects the enhanced interest tendency of the user to the video content.

[0054] Finally, by step 1043, each user node is traversed to find all content nodes directly connected to it as adjacent nodes. For each adjacent node, the updated connection strength value after step 1042 and the content's predefined association weight value are obtained, and the two are multiplied to obtain a weighted strength value. The weighted strength values of all adjacent nodes of the user are accumulated to generate the association feature value of the user. Finally, all user association feature values are arranged in numerical sequence according to the user number order to form an expression vector output to the recommendation decision module. For example, the adjacent nodes of user U1 are video V1 and V2, where the adjusted strength and weight of V1 are 0.66 and 0.4 respectively, and the adjusted strength and weight of V2 are 0.22 and 0.6 respectively; the weighted cumulative value is calculated: 0.66 multiplied by 0.4 gives 0.264, and 0.22 multiplied by 0.6 gives 0.132, and the two are added to give an association feature value of 0.396. When the feature values of the three users in the system are 0.396, 0.51, and 0.23 respectively, the final output expression vector is [0.396, 0.51, 0.23].

[0055] In practical application, in a music streaming platform, user U1 collects song S1 a total of 2 times and reads an interview article of singer P1 5 times, the system maps U1 as a user node, S1 and P1 as content nodes respectively, and establishes two connection lines of U1 to S1 and U1 to P1; the initial connection strength is calculated based on the number of historical operations, the strength of U1-S1 line is equal to 2 times the basic weight coefficient 0.2, which is 0.4, and the strength of U1-P1 line is equal to 5 times 0.2, which is 1.0. Because the user recently changes from listening to songs to reading, a dynamic compensation coefficient 0.9 is generated, the initial strength of each connection line is multiplied by the coefficient to update the value, the new strength of U1-S1 is equal to 0.4 times 0.9, which is 0.36, and the new strength of U1-P1 is equal to 1.0 times 0.9, which is 0.9. Then, the adjacent nodes S1 and P1 of U1 are obtained, wherein the corresponding new strength and song class weight of S1 are 0.36 and 0.5 respectively, and the corresponding new strength and song class weight of P1 are 0.9 and 0.7 respectively, the weighted values are calculated as 0.36 times 0.5 equal to 0.18 and 0.9 times 0.7 equal to 0.63 respectively, and the sum is 0.81, which is the correlation characteristic value. Finally, the characteristic values of U1, U2 and U3 are arranged in ascending order of user ID, and the expression vector [0.81, 0.65, 0.42] is generated to input the recommendation module.

[0056] The overall scheme of the above 104, by constructing the user and content bipartite graph network, initializing the connection strength between nodes based on historical behavior, establishing a basic association model; adjust the connection strength value by using the dynamic compensation coefficient, make the network relationship adaptively respond to the user interest shift; finally, the expression vector is generated by weighted fusion of adjacent node information, realizing the quantitative representation and dynamic update of user interest. This process effectively solves the problems that static network cannot capture interest drift and cross-modal correlation is fragmented, and provides updated structured data basis for accurate recommendation.

[0057] 105, extract the user purpose characteristic value, content contact characteristic value and time efficiency characteristic value in the expression vector, construct a dynamic change weight model combined with the preference change trajectory of the historical behavior record, generate a decision vector by integrating the user purpose characteristic value, content contact characteristic value, time efficiency characteristic value and change weight value output by the dynamic change weight model, and perform cross-modal matching sorting based on the decision vector to output personalized content recommendation results.

[0058] Optionally, step 105 can specifically include the following steps: 1051, separate the user purpose characteristic value, content contact characteristic value and time efficiency characteristic value from the expression vector, and extract the operation time sequence based on the preference change trajectory of the historical behavior record, and extract the recent operation time and operation frequency from the operation time sequence; 1052. constructing a dynamic change weight model based on the latest operation time and operation frequency and calculating a change weight value; 1053. for each content item, calculating a decision value of the corresponding content item by multiplying the user purpose feature value, the content contact feature value and the time efficiency feature value respectively by the change weight value, and organizing all decision values into a decision vector according to the content item identifiers of different content items; The step 1053 can specifically include the following process: extracting the user purpose feature value, the content contact feature value and the time efficiency feature value corresponding to the current content item; multiplying the user purpose feature value by the change weight value to obtain a purpose weighted value, multiplying the content contact feature value by the change weight value to obtain a contact weighted value, and multiplying the time efficiency feature value by the change weight value to obtain a time efficiency weighted value; adding the purpose weighted value, the contact weighted value and the time efficiency weighted value to generate the decision value of the current content item; associating the decision value with the content item identifier of the current content item, and after traversing all content items, arranging all decision values in the order of the content item identifiers to form a decision vector.

[0059] 1054. performing cross-modal uniform sorting on the decision values of all content items in the decision vector, and outputting the content item identifier with the highest sorting as the personalized recommendation result.

[0060] In the above scheme, the user purpose feature value is a quantitative indicator reflecting the current core intention of the user, including behavior data such as search keyword intensity and page stay duration, which can be used to identify the dominant demand direction of the user. The content contact feature value is a measurement value representing the correlation degree between the content item and the user's historical preference, including theme similarity, author attention and interaction depth, etc., which can be used to evaluate the matching degree between the content and the user's interest. The time efficiency feature value is a dynamic value measuring the freshness of the content and the adaptability of the user's recent behavior, including time-sensitive factors such as publication time decay coefficient and interaction frequency, which can be used to capture the interest drift trend. The dynamic change weight model is a weight calculation function constructed based on user operation behavior, including the synergistic mechanism of the latest operation time weight factor and the operation frequency weight factor, which can be used to dynamically adjust the importance of features. The decision vector is a recommendation priority sequence formed by integrating weighted feature values, including the fusion decision values of all content items and their unique identifiers, which can be used to realize the fair sorting output of cross-modal content.

[0061] In the embodiments of the present application, the axis is separated from the expression vector by step 1051 to obtain the user purpose feature value, which is calculated by analyzing the user search keyword intensity and page stay duration and other behavior data, for quantifying the current core intention direction of the user; meanwhile, the content contact feature value is separated, which is based on the theme matching algorithm to evaluate the similarity between the content item and the user's historical preferences and the author's attention; and the timeliness feature value is separated, which is dynamically calculated by using the time decay function in combination with the content publishing time and the interaction frequency to calculate the freshness weight. At the same time, the system calls the operation time sequence in the user historical behavior record, that is, the complete interaction log sorted by timestamp, to parse the specific time point of the last operation and the operation frequency statistical value within the set time window.

[0062] Then, by step 1052, based on the recent operation time interval and operation frequency data provided by step 1051, a dynamic weight calculation engine is started for processing. For the recent operation time, an inverse proportion function is used to calculate the time weight factor, and the core principle is that the smaller the time interval, the higher the weight, for example, a coefficient of 1.2 is generated for an interval of 2 hours; for the operation frequency, a logarithmic function is used to calculate the frequency weight factor, and the characteristic is that the weight increases gradually converges when the frequency increases, for example, a coefficient of 1.4 is generated for 7 operations within 24 hours. Then, the time weight factor and the frequency weight factor are multiplied to obtain the comprehensive change weight value. For example, when it is detected that the user's recent operation occurred 1 hour ago and the user clicked the sports shoes category 12 times within the day, the time weight factor is calculated as 1.3 according to the inverse proportion principle, and the frequency weight factor is calculated as 1.5 according to the logarithmic function characteristics, and finally the change weight value is 1.95.

[0063] Then, by step 1053, for each content item to be recommended, its corresponding user purpose feature value, content contact feature value and timeliness feature value are extracted; then the three feature values are multiplied by the change weight value generated by step 1052 to obtain the purpose weighted value, the contact weighted value and the timeliness weighted value; then the three weighted values are added to calculate the final decision value of the content item; finally, the content item unique identifier is associated with the decision value, and when all the candidate content items are traversed, all the decision values are integrated to form a decision vector in the order of the identifiers. For example, for the running shoes frequently browsed by the user, the user purpose feature value 0.9, the content contact feature value 0.7 and the timeliness feature value 0.8 are extracted, and the weighted calculation is performed in combination with the change weight value 1.6 to obtain , , , the sum of the three is the decision value 3.84; after associating the product identifier ID_RUN001, the decision value is recorded in the vector, and finally occupies the first place in the decision vector containing 20 products.

[0064] Finally, after receiving the decision vector, the cross-modal unified ranking mechanism is started to globally compare the decision values of all content items through step 1054. The mechanism uses the extreme value retrieval algorithm to scan the entire decision vector, automatically locates the highest numerical decision value and locks its corresponding content item unique identifier. Then the content item represented by the identifier is directly output as the final personalized recommendation result. For example, when the decision value 4.37 of the hiking pole video in the decision vector surpasses the text guide 3.12, the system automatically pushes the video to the user's home page, realizing the optimal matching in the cross-modal scene.

[0065] In practical application, after user D searches "family camping equipment" on the comprehensive content platform, the system first parses three key numerical values from his behavior expression vector: the user purpose feature value 0.75 reflects a strong outdoor shopping intention, the content association feature value 0.65 indicates the association degree with the tent product purchased in history, and the timeliness feature value 0.85 corresponds to the newly listed portable grill product; at the same time, it is detected that the recent operation occurred 3 hours ago, and the user has browsed the camping category 6 times within 24 hours. Then the system starts dynamic weight calculation: based on the 3-hour interval, a time factor 1.15 is generated by an inverse proportional function, combined with the operation frequency of 6 times to generate a frequency factor 1.45 by a logarithmic function, and the two are multiplied to obtain a comprehensive change weight value 1.15 multiplied by 1.45 equal to 1.6675. Then for the portable grill product: the purpose feature value 0.75 multiplied by the weight value is 1.250625, the content association feature value 0.65 multiplied by the weight value is 1.083875, and the timeliness feature value 0.85 multiplied by the weight value is 1.417375, and the three are added to generate the final decision value 1.250625 plus 1.083875 plus 1.417375 equal to 3.751875 and associate the product identifier ID_Grill202. After the system traverses the 32 candidate products including picnic mat, folding chair, etc., the portable grill with a decision value of 3.751875 surpasses the picnic mat 2.98 and the folding chair 3.21, and finally wins in the cross-modal ranking, and the product video is pushed to the user's home page. At this time, the user is preparing for the weekend family camping, and the recommendation accurately matches his immediate purchase needs.

[0066] The overall scheme of the above 105 separates the user purpose, content association and timeliness features, constructs a weight model combining the recent behavior frequency and time dynamics, realizes the precise weighted fusion of multi-dimensional features; generates a content item exclusive decision value based on the weighted result and forms a globally comparable vector, and finally outputs the optimal recommendation result through cross-modal unified ranking. This mechanism effectively captures the instantaneous interest changes of users, synchronously coordinates long-term preferences and needs, breaks through the recommendation deviation caused by static data processing and modal barriers in traditional schemes, significantly improves the timeliness and accuracy of content matching, and ensures that the content of high-frequency strong needs of users is given priority to exposure.

[0067] The following is a complete example for steps 101-105, as shown in Figure 2 As shown, when user Li Hua searches for high-altitude camping equipment on the outdoor equipment platform, the multi-channel acquisition server captures a mixed data stream containing a high-altitude tent selection guide with text and images, a user-uploaded gas stove usage evaluation video, and a high-altitude cooking utensil voice Q&A. The hardware separation unit separates the text and images into text feature streams and image feature streams using a dedicated image processing chip. The text stream extracts text features such as titles and parameter descriptions, and the image stream analyzes tent structure diagrams and material close-ups. Meanwhile, the audio decoder separates the video's picture frame sequence and audio stream. The picture frame sequence captures the gas stove operation demonstration process, and the audio stream separates the commentary voice content. Finally, five independent feature packages are generated and labeled with a unified content identifier: the text feature package contains keywords such as wind resistance and warmth, the image feature package stores tent wind resistance test images, the video feature package records gas stove ignition demonstrations, and the audio feature package saves altitude usage points commentary. All feature packages are cross-modally associated through content identifiers.

[0068] Subsequently, the platform calls a pre-trained multi-modal large model to analyze the correlation between the feature packages. The model first calculates the semantic consistency between the wind resistance description in the text package and the tent deformation test images in the image package, obtaining a similarity score of 0.91 through attention matching algorithm. Then it detects the spatio-temporal alignment relationship between the blue flame close-up frame in the video package and the commentary on insufficient oxygen leading to insufficient combustion in the audio package, and after identifying a 0.8-second delay between the key frame and the voice, it gives a matching value of 0.87. Based on this, a cross-modal similarity correspondence table is constructed, establishing a mapping entry from the wind resistance parameters in the text and image to the test images, and generating an associated entry from the flame abnormal phenomenon in the video and audio to the insufficient combustion explanation.

[0069] The system traces back user Li Hua's behavior records within the past thirty days and finds that he frequently clicks on tent text and images and then watches test videos, with a text-to-video conversion rate of eighty-nine percent. However, the average completion rate of gas stove videos is only forty-five percent. Combined with the similarity table, the source of the deviation is analyzed in depth: the cross-modal matching degree between tent text and video is 0.91, which meets the expected value and does not need to be compensated; the gas stove content has a technical deviation of asynchronous pictures and commentary, as well as a content defect of missing high-altitude scenes. Accordingly, a dynamic compensation coefficient is generated: the tent content maintains a baseline coefficient of 1.0, the gas stove content adds a technical deviation compensation of 0.1 and a content missing compensation of 0.15, and the final compensation coefficient is determined to be 1.25.

[0070] Then, a bipartite graph structure of user Li Hua and content nodes is constructed, and the user nodes are connected to the tent text node and the gas stove video node. The initial edge strength is set to tent connection 0.85 and gas stove connection 0.75. The compensation coefficient is applied to dynamically adjust the connection relationship: the gas stove edge strength is updated to 0.75 multiplied by 1.25 to obtain 0.9375. The features of adjacent nodes are aggregated through the graph convolution network, and the material parameter characteristics of the tent node, the highland adaptation characteristics of the gas stove node, and the cold resistance demand characteristics in the user historical behavior are integrated, to finally generate an expression vector containing professional demand value 0.83, scene adaptation value 0.78, and time-sensitive value 0.95.

[0071] Finally, the user purpose feature value 0.83 is separated from the expression vector to represent the highland equipment procurement intention, the gas stove content contact feature value 0.78, and the time-sensitive feature value 0.95 corresponding to the newly listed highland special stove. The user dynamics are monitored synchronously: searching for highland stove four times in the last fifteen minutes. The time factor and the frequency factor are calculated by the dynamic weight model: the inverse proportional function processes the fifteen-minute interval to output 1.4, and the logarithmic function processes the four times frequency to output 1.35, and the comprehensive weight value is 1.4 multiplied by 1.35 equal to 1.89. The gas stove decision value is calculated step by step: the purpose weighted value 0.83 multiplied by 1.89 is 1.5687, the contact weighted value 0.78 multiplied by 1.89 is 1.4742, and the time-sensitive weighted value 0.95 multiplied by 1.89 is 1.7955, and the sum of the three is the decision value 4.8384. When the value exceeds the tent decision value 4.12, the system pushes the newly added highland gas stove operation detail video with 5000m altitude ignition real frame on the homepage during the user's packing for Tibet.

[0072] Figure 3 A structure diagram of a personalized recommendation system based on a large model is provided for the embodiments of the present application, as shown in Figure 3 The system comprises: An acquisition module 31 is configured to acquire a content data stream comprising a multi-modal feature mixture, and a hardware separation unit of a multi-channel acquisition server is used to separate the content data stream into original data streams of different modal features, and independent feature packages are generated according to the corresponding feature data and identifiers of the original data streams. An analysis module 32 is configured to analyze the semantic relationship between text features and picture features, and the matching degree between sound features and video content among a plurality of independent feature packages through a pre-established large model, to establish a similarity correspondence table across modalities. A generation module 33 is configured to determine historical behavior records among different modal content in the content data stream, and to generate a dynamic compensation coefficient by combining a bias source analysis with the similarity correspondence table according to the cross-modal conversion records in the historical behavior records. The adjustment module 34 is configured to construct a bipartite graph structure of the content nodes and the user nodes, convert the dynamic compensation coefficient into an adjustment factor of the connection strength, adjust the connection strength of the content nodes and the user nodes, and integrate the association weights of adjacent nodes to generate an expression vector containing association features. The output module 35 is configured to extract a user purpose feature value, a content association feature value and a time effectiveness feature value in the expression vector, construct a dynamic change weight model in combination with a preference change trajectory of the historical behavior record, integrate the user purpose feature value, the content association feature value, the time effectiveness feature value and a change weight value output by the dynamic change weight model to generate a decision vector, and perform cross-modal matching and sorting based on the decision vector to output a personalized content recommendation result.

[0073] Figure 3 The personalized recommendation system based on the large model can perform Figure 1 The personalized recommendation method based on the large model of the embodiment shown in the figure has been described above. The specific implementation of the operations of each module and unit of the personalized recommendation system based on the large model in the above embodiment has been described in detail in the embodiment related to the method, and will not be described in detail here.

[0074] In one possible design, Figure 3 The personalized recommendation system based on the large model of the embodiment shown in the figure can be implemented as a computing device, such as a server. Figure 4 As shown in the figure, the computing device can include a storage component 41 and a processing component 42. The storage component 41 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 42.

[0075] The processing component 42 is configured to perform the above Figure 1 The personalized recommendation method based on the large model of the embodiment.

[0076] The processing component 42 can include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component can also be one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components, for executing the above method.

[0077] The storage component 41 is configured to store various types of data to support the operation of the terminal. The storage component can be implemented by any type of volatile or nonvolatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic or optical disk.

[0078] Of course, the computing device can also necessarily include other components, such as an input / output interface, a display component, a communication component, etc.

[0079] The input / output interface provides an interface between the processing component and peripheral interface modules, which can be output devices, input devices, etc.

[0080] The communication component is configured to facilitate wired or wireless communication between the computing device and other devices, etc.

[0081] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, at which time the computing device can refer to a cloud server, and the processing component, the storage component, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0082] The embodiment of the application also provides a computer storage medium, which stores a computer program, and the computer program can implement the above-mentioned Figure 1 The embodiment of the application also provides a computer storage medium, which stores a computer program, and the computer program can implement the above-mentioned

[0083] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-mentioned system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

[0084] The device embodiment described above is only schematic, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment scheme. Those skilled in the art can understand and implement it without creative labor.

[0085] Those skilled in the art can clearly understand the implementation of the various embodiments by means of software and the necessary general hardware platform from the above description of the embodiments, and of course, the embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that contributes to the technical solutions can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for causing a computer device (which can be a personal computer, a server, etc.) to execute the methods described in the various embodiments or some parts of the methods.

[0086] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A personalized recommendation method based on a large model, characterized in that: include: Acquire a content data stream including a mixture of multimodal features, configure a multi-channel acquisition server, use a hardware separation unit of the multi-channel acquisition server to decompose the content data stream into raw data streams of different modal features, and generate independent feature packets based on feature data and identifiers corresponding to the raw data streams; By using a pre-established large model, the semantic relationship between text features and image features, and the matching degree between sound features and video content between multiple independent feature packages are analyzed to establish a cross-modal similarity correspondence table; Determining historical behavior records between different modal contents in the content data stream, and performing deviation source analysis based on cross-modal conversion records in the historical behavior records in combination with the similarity correspondence table to generate a dynamic compensation coefficient; Constructing a bipartite graph structure of content nodes and user nodes, converting the dynamic compensation coefficient into an adjustment factor of connection strength to adjust the connection strength between the content nodes and the user nodes, and integrating the association weights of adjacent nodes to generate an expression vector containing association features; Extract the user purpose feature value, content connection feature value and timeliness feature value from the expression vector, construct a dynamic change weight model based on the preference change trajectory of the historical behavior record, integrate the user purpose feature value, content connection feature value, timeliness feature value and the change weight value output by the dynamic change weight model to generate a decision vector, perform cross-modal matching and sorting based on the decision vector, and output personalized content recommendation results.

2. The method according to claim 1, characterized in that The method extracts the user purpose feature value, content connection feature value, and timeliness feature value from the expression vector, constructs a dynamic change weight model based on the preference change trajectory of the historical behavior record, integrates the user purpose feature value, content connection feature value, timeliness feature value, and the change weight value output by the dynamic change weight model to generate a decision vector, and performs cross-modal matching and sorting based on the decision vector to output personalized content recommendation results, including: Separating the user purpose feature value, content connection feature value, and timeliness feature value from the expression vector, extracting an operation time series based on the preference change trajectory of historical behavior records, and extracting the most recent operation time and operation frequency from the operation time series; Constructing a dynamic change weight model based on the most recent operation time and operation frequency and calculating a change weight value; For each content item, the user purpose feature value, the content connection feature value, and the timeliness feature value are respectively calculated with the change weight value to generate a decision value for the corresponding content item, and all decision values ​​are organized according to the content item identifiers corresponding to different content items to form a decision vector; The decision values ​​of all content items in the decision vector are uniformly ranked across modalities, and the identifier of the content item with the highest ranking is output as the personalized recommendation result.

3. The method according to claim 2, characterized in that For each content item, the user purpose feature value, the content connection feature value, and the timeliness feature value are respectively calculated with the change weight value to generate a decision value for the corresponding content item, and all decision values ​​are organized according to the content item identifiers corresponding to different content items to form a decision vector, including: Extract the user purpose feature value, content connection feature value and timeliness feature value corresponding to the current content item; Multiplying the user purpose feature value by the change weight value to obtain a purpose weight value, multiplying the content connection feature value by the change weight value to obtain a connection weight value, and multiplying the timeliness feature value by the change weight value to obtain a timeliness weight value; Adding the purpose weighted value, the connection weighted value, and the timeliness weighted value to generate a decision value for the current content item; The decision value is associated with the content item identifier of the current content item, and after traversing all content items, all the decision values ​​are arranged in the order of the content item identifiers to form a decision vector.

4. The method according to claim 1, wherein The bipartite graph structure of the content nodes and the user nodes is constructed, the dynamic compensation coefficient is converted into an adjustment factor of the connection strength to adjust the connection strength between the content nodes and the user nodes, and the association weights of adjacent nodes are integrated to generate an expression vector containing association features, including: Constructing a bipartite graph structure consisting of content nodes and user nodes, and assigning an initial connection strength value to each connection line between the content node and the user node based on the number of historical operations performed by the user on the content node; Converting the dynamic compensation coefficient into an adjustment factor for connection strength, and performing an operation on the initial connection strength value to obtain an adjusted connection strength value; For each of the user nodes, the adjusted connection strength value and association weight of its adjacent content nodes are obtained, the connection strength value is weighted based on the association weight to generate the association feature value of the corresponding user node, and the association feature values ​​of all user nodes are arranged in node order to form an expression vector.

5. The method according to claim 1, wherein Determining the historical behavior records between different modal contents in the content data stream, and performing deviation source analysis based on the cross-modal conversion records in the historical behavior records in combination with the similarity correspondence table to generate a dynamic compensation coefficient includes: Reading historical behavior records between different modal contents in the content data stream from a storage system; Counting the ratio of the number of conversions from text content to image content in the historical behavior record as a text-to-image conversion value, and counting the ratio of the number of conversions from sound content to video content as a sound-to-video conversion value; Extracting the semantic relationship score and the matching degree score from the similarity correspondence table, and calculating a first deviation between the text-to-picture conversion value and the semantic relationship score and a second deviation between the audio-to-video conversion value and the matching degree score; A dynamic compensation coefficient positively correlated with the deviation is generated according to the size range of the first deviation and the second deviation.

6. The method according to claim 1, characterized in that The pre-established large model is used to analyze the semantic relationship between text features and image features, and the matching degree between sound features and video content between multiple independent feature packages to establish a cross-modal similarity correspondence table, including: According to the independent feature package, the text features in the text feature package and the image features in the image feature package are input into the large model, the content consistency of the text features and the image features is compared, and the semantic relationship score between each text feature and the corresponding image feature is calculated; Input the sound features in the sound feature package and the video content in the video content feature package into the large model, detect the time alignment status of the sound features and the video content, and calculate the matching degree score between each sound feature and the corresponding video content; The semantic relationship score is recorded as a text-image relationship item according to the combination of the text feature package identifier and the image feature package identifier, and the matching degree score is recorded as a sound-video relationship item according to the combination of the sound feature package identifier and the video content feature package identifier. All the text-image relationship items and the sound-video relationship items are integrated to construct a similarity correspondence table containing identifier combinations and relationship scores.

7. The method according to claim 1, characterized in that The method of acquiring a content data stream including a mixture of multimodal features, configuring a multi-channel acquisition server, utilizing a hardware separation unit of the multi-channel acquisition server to decompose the content data stream into original data streams of different modal features, and generating independent feature packages based on feature data and identifiers corresponding to the original data streams, includes: Obtain content data streams containing text, images, sounds, and videos; Activating multiple physical channels of a hardware separation unit by configuring a multi-channel acquisition server to split the content data stream, thereby generating an original data stream including a text stream, a picture stream, a sound stream, and a video stream; Extracting character sequences and position information from the text stream, extracting color distribution and contour point sets from the image stream, extracting frequency bands and loudness sequences from the sound stream, and extracting motion trajectories and brightness changes from the video stream, so as to extract feature data corresponding to each of the original data streams; An identifier including a modality type code and a timestamp is attached to the feature data corresponding to the original data stream, and the data is packaged to generate independent feature packages corresponding to a text feature package, a picture feature package, a sound feature package, and a video feature package.

8. A personalized recommendation system based on a large model, characterized by: include: An acquisition module is configured to acquire a content data stream including a mixture of multimodal features, configure a multi-channel acquisition server, utilize a hardware separation unit of the multi-channel acquisition server to decompose the content data stream into raw data streams of different modal features, and generate independent feature packets based on the feature data and identifiers corresponding to the raw data streams; An analysis module is used to analyze the semantic relationship between text features and image features, and the matching degree between sound features and video content between the plurality of independent feature packages using a pre-established large model, so as to establish a cross-modal similarity correspondence table; a generation module configured to determine historical behavior records between different modal contents in the content data stream, and to generate a dynamic compensation coefficient by performing a deviation source analysis based on cross-modal conversion records in the historical behavior records and in combination with the similarity correspondence table; An adjustment module is configured to construct a bipartite graph structure of content nodes and user nodes, convert the dynamic compensation coefficient into an adjustment factor of connection strength to adjust the connection strength between the content nodes and the user nodes, and integrate the association weights of adjacent nodes to generate an expression vector containing association features; The output module is used to extract the user purpose feature value, content connection feature value and timeliness feature value from the expression vector, construct a dynamic change weight model based on the preference change trajectory of the historical behavior record, integrate the user purpose feature value, content connection feature value, timeliness feature value and the change weight value output by the dynamic change weight model to generate a decision vector, and perform cross-modal matching and sorting based on the decision vector to output personalized content recommendation results.

9. A computing device, characterized in that It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a large model-based personalized recommendation method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that A computer program is stored, and when the computer program is executed by a computer, the method for personalized recommendation based on a large model as claimed in any one of claims 1 to 7 is implemented.

Images