AI intelligent recommendation method and system
Through feature extraction and angle calculation, the fusion of text and image data is achieved, which solves the problem of multimodal data processing and improves the accuracy and flexibility of the recommendation system.
Patent Information
- Application Number
- CN202411419805.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-12
- Publication Date
- 2024-12-24
- Estimated Expiration
- 2044-10-12
AI Technical Summary
Existing recommendation systems are difficult to effectively process multimodal data, especially the fusion of text and image data, resulting in inaccurate recommendations.
Through feature extraction, the text and image data are converted into feature vectors, and the angle between feature vectors is calculated to determine whether to fusion is performed, ensuring that different modal data are compared and fused in the same feature space, and flexibly deal with missing data.
It improves the accuracy and robustness of the recommendation system, generates more comprehensive and accurate recommendation content, and enhances the flexibility and performance of the system.
Smart Images
Figure CN118964647B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of AI data processing technology, and in particular to an AI intelligent recommendation method and system. Background Art
[0002] One of the challenges facing existing recommendation systems and content generation systems is how to effectively handle multimodal data. Multimodal data refers to data from different sensory or information sources, such as text, images, audio, etc. Specifically in this case, the main issue is how to handle data that has both text and images.
[0003] Since text and images belong to different data types, they have different structures and semantic information. It is not easy to directly merge these two types of data because their feature representation methods are different. Moreover, the feature extraction methods of text and images are different. Text feature extraction usually relies on natural language processing technology (such as TF-IDF, Word2Vec, etc.), while image feature extraction relies on computer vision technology (such as convolutional neural network CNN, feature point detection, etc.). The information carried by text and images may be complementary, but how to effectively combine this information is a challenge. For example, a picture may contain a lot of visual information, but lack specific descriptive information, while text can supplement this information. How to align the feature vectors of different modalities and make accurate recommendations based on the multimodal data input by the user is a difficult problem that needs to be solved urgently. Summary of the invention
[0004] The embodiments of the present invention provide an AI intelligent recommendation method and system, which generate more accurate recommendation solutions based on user behavior data and preferences.
[0005] A first aspect of an embodiment of the present application provides an AI intelligent recommendation method, including:
[0006] Get input text and / or input image;
[0007] After obtaining the input text, extracting features of the input text to obtain a plurality of text feature words, and combining the plurality of text feature words in pairs to obtain a plurality of text feature vectors;
[0008] After obtaining the input image, extracting features of the input image to obtain a plurality of image feature words, and combining all the image feature words to obtain an image feature vector;
[0009] Calculate the angles between all the text feature vectors and the preset center vector as the text angle; if the angle between one of the text feature vectors and the picture feature vector is less than or equal to the corresponding text angle, fuse the text feature vector with the picture feature vector to obtain a picture-text feature vector; if the angle between one of the text feature vectors and the picture feature vector is greater than the corresponding text angle, recombine the picture feature vector until the angle between the text feature vector and the picture feature vector is less than or equal to the corresponding text angle, fuse the text feature vector with the picture feature vector to obtain a picture-text feature vector; if the input text is not obtained, use the picture feature vector as the picture-text feature vector; if the input picture is not obtained, use the fusion vector of the several text feature vectors as the picture-text feature vector;
[0010] Determine in the database several label vectors whose similarity with the image-text feature vector is greater than a preset threshold, input the material resources corresponding to the several label vectors into a set text generation model to obtain a recommendation scheme; the label vector is generated after feature extraction of the corresponding material resources.
[0011] In a possible implementation of the first aspect, after acquiring the input text, extracting features of the input text to obtain a plurality of text feature words, and combining the plurality of text feature words in pairs to obtain a plurality of text feature vectors, specifically includes:
[0012] Segmenting the input text according to a preset rule to obtain a plurality of words;
[0013] Using the TextRank algorithm to extract the several words to obtain several feature words;
[0014] The plurality of feature words are combined in pairs to form a plurality of feature vectors.
[0015] In a possible implementation manner of the first aspect, after acquiring the input picture, performing feature extraction on the input picture to obtain a plurality of picture feature words, and combining all the picture feature words to obtain a picture feature vector, specifically includes:
[0016] Separating text areas from other non-text areas according to the connected area recognition result of the input image;
[0017] In the text area, the accumulated value of each column of pixels is calculated, and the positions of several text lines are determined according to the peak value of the vertical projection;
[0018] According to the position of the text line, the accumulated value of the pixels of each line of the text line is calculated, and segmentation is performed according to the peak value of the horizontal projection to obtain a plurality of words;
[0019] Classifying the plurality of words using a machine learning model or a deep learning model, comparing each classification with a preset word template, and obtaining a plurality of matching first image feature words according to the comparison result;
[0020] Clustering all the other non-text areas, comparing each cluster with a preset picture template, and obtaining a plurality of matching second picture feature words according to the comparison result;
[0021] The plurality of first picture feature words and the plurality of second picture feature words are combined to obtain a picture feature vector.
[0022] In a possible implementation manner of the first aspect, in the text area, calculating the accumulated value of each column of pixels and determining the positions of a plurality of text rows according to a peak value of a vertical projection specifically includes:
[0023] Create a first array with length equal to the image width;
[0024] Traverse each column of pixels, and use the number of preset color pixels in each column as the array value of the first array;
[0025] The local maximum value of the first array is found through the sliding window method to determine the starting position and the ending position of the text line.
[0026] In a possible implementation manner of the first aspect, calculating the accumulated value of pixels of each line of the text line and segmenting the text line according to the peak value of the horizontal projection to obtain a plurality of words specifically includes:
[0027] Initialize a second array with a length equal to the height of the text line;
[0028] Traversing each text line, and calculating the number of pixels of a preset color in the line as the array value of the second array;
[0029] The local maximum interval of the second array is found by a sliding window method, and the text line is segmented according to the local maximum interval of the second array to obtain a plurality of words.
[0030] In a possible implementation manner of the first aspect, separating the text area from other non-text areas according to the connected area recognition result of the input image specifically includes:
[0031] Detecting connected regions in the input image after binarization;
[0032] Calculate the area, aspect ratio and appearance contour of each of the connected regions;
[0033] The connected regions with an outer contour of a long strip or a circle are excluded, and the text regions are screened out according to the area and aspect ratio of the remaining connected regions.
[0034] In a possible implementation manner of the first aspect, before calculating the angles between all the character feature vectors and a preset center vector as the character angle, the method includes:
[0035] Obtain user behavior data for all recommended solutions in historical data;
[0036] Obtain several label vectors corresponding to several recommendation solutions whose user click rate and dwell time meet preset conditions;
[0037] A clustering algorithm is used to cluster the plurality of label vectors to obtain a plurality of clusters; and an average vector of all the clusters is selected as a preset central vector.
[0038] In a possible implementation manner of the first aspect, if the angle between one of the text feature vectors and the picture feature vector is greater than the corresponding text angle, recombining the picture feature vectors until the angle between the text feature vector and the picture feature vector is less than or equal to the corresponding text angle specifically includes:
[0039] Determine a first feature vector for each of the first picture feature words and a second feature vector for each of the second picture feature words;
[0040] Determine the first picture feature word of the first feature vector which is farthest from the preset center vector as the first target feature word;
[0041] Determine the second picture feature word of the second feature vector which is farthest from the preset center vector as the second target feature word;
[0042] The first target feature word and the second target feature word are deleted, and the remaining first picture feature words and the remaining second picture feature words are recombined to obtain a new picture feature vector, until the angle between the text feature vector and the picture feature vector is less than or equal to the corresponding text angle.
[0043] In a possible implementation manner of the first aspect, before inputting the material resources corresponding to the plurality of label vectors into a set text generation model to obtain a recommendation solution, the method includes:
[0044] According to the direction of the preset center vector, increasing the learning rate of the text generation model in the direction;
[0045] According to the distribution characteristics of the preset center vector, adjusting the batch size of the text generation model;
[0046] According to the number of clusters in the clustering, the text generation model is enlarged by a multiple equal to the number.
[0047] A second aspect of an embodiment of the present application provides an AI intelligent recommendation system, including:
[0048] An acquisition module, used to acquire input text and / or input image;
[0049] A text extraction module is used to extract features of the input text to obtain a plurality of text feature words after obtaining the input text, and to combine the plurality of text feature words in pairs to obtain a plurality of text feature vectors;
[0050] A picture extraction module, configured to extract features of the input picture after acquiring the input picture to obtain a plurality of picture feature words, and combine all of the plurality of picture feature words to obtain a picture feature vector;
[0051] A vector module, used for calculating the angles between all the text feature vectors and a preset center vector as the text angle; if the angle between one of the text feature vectors and the picture feature vector is less than or equal to the corresponding text angle, the text feature vector is merged with the picture feature vector to obtain a picture-text feature vector; if the angle between one of the text feature vectors and the picture feature vector is greater than the corresponding text angle, the picture feature vector is recombined until the angle between the text feature vector and the picture feature vector is less than or equal to the corresponding text angle, and the text feature vector is merged with the picture feature vector to obtain a picture-text feature vector; if the input text is not obtained, the picture feature vector is used as the picture-text feature vector; if the input picture is not obtained, the fusion vector of the several text feature vectors is used as the picture-text feature vector;
[0052] The recommendation module is used to determine in the database several label vectors whose similarity with the image and text feature vector is greater than a preset threshold, input the material resources corresponding to the several label vectors into the set text generation model, and obtain a recommendation scheme; the label vector is generated after feature extraction of the corresponding material resources.
[0053] Compared with the prior art, the embodiments of the present invention provide an AI intelligent recommendation method and system, which realizes the unified representation of data of different modalities by converting text features and image features into feature vectors respectively, so that data of different modalities can be compared and fused in the same feature space. The method of calculating the angle enables the system to quantify the similarity between different feature vectors and decide whether to fuse them accordingly. This alignment method ensures the consistency of features of different modalities and improves the accuracy of fusion. In addition, in the absence of data of a certain modality, the system can flexibly use data of another modality to generate recommendation solutions to ensure the robustness of the recommendation system.
[0054] In summary, through feature extraction, feature alignment and flexible handling of missing data, we ensured the effective integration of different modal data, thereby generating more comprehensive and accurate recommendations. This method not only improves the performance of the recommendation system, but also enhances the flexibility and robustness of the system. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 It is a flowchart of an AI intelligent recommendation method provided by an embodiment of the present invention;
[0056] Figure 2 It is a structural diagram of an AI intelligent recommendation system provided by one embodiment of the present invention. DETAILED DESCRIPTION
[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0058] Please refer to Figure 1 , an embodiment of the present invention provides an AI intelligent recommendation method, comprising:
[0059] S10: Obtain input text and / or input image.
[0060] S11. After obtaining the input text, extract the features of the input text to obtain a plurality of text feature words, and combine the plurality of text feature words in pairs to obtain a plurality of text feature vectors.
[0061] S12: After acquiring the input image, extract features of the input image to obtain a number of image feature words, and combine all the image feature words to obtain a image feature vector.
[0062] S13, calculate the angles between all the text feature vectors and the preset center vector as the text angle; if the angle between one of the text feature vectors and the image feature vector is less than or equal to the corresponding text angle, fuse the text feature vector with the image feature vector to obtain a text-image feature vector; if the angle between one of the text feature vectors and the image feature vector is greater than the corresponding text angle, recombine the image feature vectors until the angle between the text feature vector and the image feature vector is less than or equal to the corresponding text angle, fuse the text feature vector with the image feature vector to obtain a text-image feature vector; if the input text is not obtained, use the image feature vector as the text-image feature vector; if the input image is not obtained, use the fusion vector of the several text feature vectors as the text-image feature vector.
[0063] S14, determining in the database several label vectors whose similarity with the image-text feature vector is greater than a preset threshold, inputting the material resources corresponding to the several label vectors into a set text generation model to obtain a recommendation scheme; the label vector is generated after feature extraction of the corresponding material resources.
[0064] S10 needs to ensure that the system can handle different types of input data, such as pure text, pure images, or both. S11 converts text into a form that can be processed by computers through feature extraction; through feature combination, the semantic relationship in the text can be better captured to prepare for subsequent fusion. Then in S12, the image is converted into a feature vector for subsequent processing. By combining feature words, a comprehensive image feature vector is formed, which is convenient for comparison and fusion with the text feature vector.
[0065] S13 determines the consistency of feature vectors by calculating the angle to ensure the rationality of fusion. Fusion is performed only when the consistency between feature vectors is high enough, otherwise they are recombined to ensure the accuracy and reliability of the results. Even if there is only one modality of data input, a valid feature vector can be generated to ensure the flexibility of the recommendation system.
[0066] Finally, in S14, the database is searched for label vectors whose similarity with the image and text feature vector is greater than a preset threshold. The material resources corresponding to the matched label vectors are input into the text generation model. The model is used to generate recommendation solutions; the label vectors are matched by calculating similarity to ensure the relevance of the recommended content. The material resources corresponding to the matched label vectors are used to improve the quality of the recommended content; the recommendation solutions are generated through the text generation model to ensure the diversity and novelty of the recommended content.
[0067] The above method can handle different types of input data, such as pure text, pure images, or both. Through feature extraction and combination, data of different modalities are converted into a processable form and the semantic relationship between them is captured; the consistency of feature vectors is judged by calculating the angle, ensuring that fusion is performed only when the consistency between feature vectors is high enough, thereby improving the accuracy and reliability of recommendation results. The most relevant label vectors are found through similarity matching, and these label vectors are used to generate high-quality recommendation solutions. This method combines natural language processing, computer vision, and machine learning technologies to generate more accurate and diverse recommendation content based on user behavior data and preferences.
[0068] Compared with the prior art, an embodiment of the present invention provides an AI intelligent recommendation method, which realizes the unified representation of data of different modalities by converting text features and image features into feature vectors respectively, so that data of different modalities can be compared and fused in the same feature space. The method of calculating the angle enables the system to quantify the similarity between different feature vectors and decide whether to fuse them accordingly. This alignment method ensures the consistency of features of different modalities and improves the accuracy of fusion. In addition, in the absence of data of a certain modality, the system can flexibly use data of another modality to generate recommendation solutions to ensure the robustness of the recommendation system.
[0069] In summary, through feature extraction, feature alignment and flexible handling of missing data, we ensured the effective integration of different modal data, thereby generating more comprehensive and accurate recommendations. This method not only improves the performance of the recommendation system, but also enhances the flexibility and robustness of the system.
[0070] Exemplarily, after obtaining the input text, feature extraction is performed on the input text to obtain a plurality of text feature words, and the plurality of text feature words are combined in pairs to obtain a plurality of text feature vectors, specifically including:
[0071] Segmenting the input text according to a preset rule to obtain a plurality of words;
[0072] Using the TextRank algorithm to extract the several words to obtain several feature words;
[0073] The plurality of feature words are combined in pairs to form a plurality of feature vectors.
[0074] First, you can preprocess the input text, including removing non-text characters such as punctuation and numbers. Convert to lowercase (if applicable) to avoid differences caused by uppercase and lowercase. Then use a word segmentation tool (such as Jieba word segmentation or other natural language processing tools) to split the input text into individual words.
[0075] The importance of each node is calculated using the TextRank algorithm. The TextRank algorithm iteratively calculates the weight of each node, similar to the PageRank algorithm in web page ranking. The weight of each node depends on its connection relationship with other nodes and the importance of these nodes. According to the node weight sorting, several words with the highest weight are selected as feature words. Feature words are usually those words that are more important and highly representative in the text.
[0076] The extracted feature words are combined in pairs to form feature vectors. Each feature vector consists of two feature words, representing the relationship between a pair of words. For example, if there are feature words "A", "B", and "C", feature vectors [A, B], [B, C], etc. can be formed.
[0077] Assume the input text is: "This is a sample text used to demonstrate the process of feature extraction."
[0078] During preprocessing, punctuation marks are removed and converted to lowercase. The result is: "This is a sample text used to demonstrate the feature extraction process";
[0079] Use the word segmentation tool to segment words. The result is: ["this is", "a", "example", "text", "used for", "demonstration", "feature", "extraction", "process"];
[0080] Use the TextRank algorithm to calculate the importance of each word. Assume the result is: "example", "feature", "extraction", and combine the feature words in pairs. The result is: [["example", "feature"], ["feature", "extraction"]].
[0081] Through the above steps, meaningful feature words are extracted from the input text, and these feature words are combined in pairs to form feature vectors. These feature vectors can be used for subsequent fusion and other processing steps to generate recommendations that better meet user preferences.
[0082] Exemplarily, after acquiring the input image, extracting features of the input image to obtain a plurality of image feature words, and combining all the image feature words to obtain an image feature vector, specifically includes:
[0083] Separating text areas from other non-text areas according to the connected area recognition result of the input image;
[0084] In the text area, the accumulated value of each column of pixels is calculated, and the positions of several text lines are determined according to the peak value of the vertical projection;
[0085] According to the position of the text line, the accumulated value of the pixels of each line of the text line is calculated, and segmentation is performed according to the peak value of the horizontal projection to obtain a plurality of words;
[0086] Classifying the plurality of words using a machine learning model or a deep learning model, comparing each classification with a preset word template, and obtaining a plurality of matching first image feature words according to the comparison result;
[0087] Clustering all the other non-text areas, comparing each cluster with a preset picture template, and obtaining a plurality of matching second picture feature words according to the comparison result;
[0088] The plurality of first picture feature words and the plurality of second picture feature words are combined to obtain a picture feature vector.
[0089] Assuming there is an input picture, the specific steps are as follows:
[0090] During image preprocessing, the input image can be converted into a grayscale image. The grayscale image is binarized to enhance the contrast between the text and the background.
[0091] Then proceed to connected region identification: Use the connected region detection algorithm to detect connected regions in the binary image. Calculate the basic features of each connected region, such as area, aspect ratio, etc. Exclude non-text regions and filter out text regions.
[0092] When calculating the vertical projection, create a first array with a length equal to the width of the image. Go through each column of pixels and count the number of non-zero pixels in each column. Use the number of non-zero pixels in each column as the value of the array. Use the sliding window method to find the local maximum in the array and determine the location of the text line.
[0093] During the calculation of the horizontal projection, for each line of text, initialize a second array with a length equal to the height of the line. Traverse each line of text and count the number of non-zero pixels in each line. The number of non-zero pixels in each line is used as the value of the array. Use the sliding window method to find the local maximum interval in the array, and divide the text line into several words according to the local maximum interval.
[0094] During the word classification process, a pre-trained machine learning model or deep learning model can be used to classify the segmented words. The model outputs the category of each word. Each classification result is compared with a preset word template to obtain a number of matching first image feature words.
[0095] Perform cluster analysis on all non-text areas. Each cluster represents the characteristics of a type of non-text area. Compare each cluster with a preset image template to obtain a number of matching second image feature words.
[0096] The first picture feature word and the second picture feature word are combined into a picture feature vector.
[0097] Through the above steps, meaningful feature words can be extracted from the input image and combined into a comprehensive image feature vector. This feature vector can be used for subsequent fusion and other processing steps to generate a recommendation solution that better meets the user's preferences.
[0098] Exemplarily, in the text area, calculating the accumulated value of each column of pixels and determining the positions of several text rows according to the peak value of the vertical projection specifically includes:
[0099] Create a first array with length equal to the image width;
[0100] Traverse each column of pixels, and use the number of preset color pixels in each column as the array value of the first array;
[0101] The local maximum value of the first array is found through the sliding window method to determine the starting position and the ending position of the text line.
[0102] Define a sliding window. The size of the window can be set according to the actual situation (for example, it can be set to 3 or 5). The sliding window moves from left to right, one pixel column at a time. In each sliding window, find the maximum value in the window. If the value of the current column is greater than the values on its left and right sides, the column is considered to be a local maximum. When the local maximum in the sliding window exceeds a certain threshold, it is considered that the position may be the starting position of the text line.
[0103] Similarly, when the local maximum value in the sliding window begins to decrease and is lower than the threshold, it is considered that the position may be the end position of the text line. In this way, the position of each line of text can be determined.
[0104] Through the above steps, the position of the text line can be determined from the text area. This method counts the pixel accumulation values of each column and finds the local maximum value through the sliding window method to determine the starting and ending positions of the text line. This position information can be used for further text segmentation and processing.
[0105] Exemplarily, the step of calculating the accumulated value of pixels of each line of the text and segmenting the text according to the peak value of the horizontal projection to obtain a plurality of words specifically includes:
[0106] Initialize a second array with a length equal to the height of the text line;
[0107] Traversing each text line, and calculating the number of pixels of a preset color in the line as the array value of the second array;
[0108] The local maximum interval of the second array is found by a sliding window method, and the text line is segmented according to the local maximum interval of the second array to obtain a plurality of words.
[0109] Exemplarily, the separating the text area from other non-text areas according to the connected area recognition result of the input image specifically includes:
[0110] Detecting connected regions in the input image after binarization;
[0111] Calculate the area, aspect ratio and appearance contour of each of the connected regions;
[0112] The connected regions with an outer contour of a long strip or a circle are excluded, and the text regions are screened out according to the area and aspect ratio of the remaining connected regions.
[0113] You can use connected region detection algorithms (such as OpenCV's findContours function) to detect connected regions in binary images. A connected region is a set of pixels in an image that have the same pixel value and are connected to each other.
[0114] For each connected region, calculate its area (that is, the number of pixels in the region). Calculate the aspect ratio:
[0115] For each connected region, calculate its aspect ratio. You can get the aspect ratio by calculating the width and height of the enclosing rectangle. Aspect ratio = width of enclosing rectangle / height of enclosing rectangle.
[0116] For each connected region, calculate its outer contour. The outer contour refers to the set of edge pixels of the connected region. You can use OpenCV's cv2.findContours() function to get the contour of the connected region.
[0117] It should be noted that a long connected region usually has a very large aspect ratio (for example, the aspect ratio is greater than a certain threshold, such as 10). A circular connected region usually has a large area but an aspect ratio close to 1. These connected regions that do not meet the text characteristics are excluded.
[0118] The text region is selected based on the area and aspect ratio of the remaining connected regions. Usually, the text region has a moderate area and aspect ratio.
[0119] Through the above steps, connected regions can be detected from the input image, and the area, aspect ratio and outer contour of each connected region can be calculated. By excluding long strip or circular connected regions and filtering out text regions based on area and aspect ratio, text regions and other non-text regions can be effectively distinguished. This method can improve the accuracy of text recognition and reduce the possibility of false detection and missed detection.
[0120] Exemplarily, before calculating the angles between all the character feature vectors and the preset center vector as the character angle, the method includes:
[0121] Obtain user behavior data for all recommended solutions in historical data;
[0122] Obtain several label vectors corresponding to several recommendation solutions whose user click rate and dwell time meet preset conditions;
[0123] A clustering algorithm is used to cluster the plurality of label vectors to obtain a plurality of clusters; and an average vector of all the clusters is selected as a preset central vector.
[0124] According to preset conditions (such as click-through rate higher than a certain threshold, longer stay time, etc.), filter out qualified recommendation solutions. These conditions can be adjusted according to business needs. For each qualified recommendation solution, extract its corresponding label vector. The label vector can be a vector representation obtained by feature extraction algorithm (such as TF-IDF, Word2Vec, etc.).
[0125] Assuming that there is user behavior data for all recommended solutions in historical data, the specific steps are as follows:
[0126] Collect user behavior data from historical data, including click records, dwell time, etc. The data format may be a CSV file or a database table.
[0127] Get several label vectors corresponding to several recommended solutions whose user click rate and dwell time meet the preset conditions: filter out the recommended solutions with a click rate higher than 0.5 and a dwell time greater than 5 seconds. Extract the label vectors corresponding to these recommended solutions.
[0128] Use a clustering algorithm to cluster the label vectors to obtain a number of clusters: Use the K-Means algorithm to cluster the label vectors. Set the number of clusters to 5 (adjust according to the specific situation). Select the average vector of all clusters as the preset center vector: Calculate the average vector for each cluster. Select the average vector of all clusters as the preset center vector.
[0129] Exemplarily, if the angle between one of the text feature vectors and the picture feature vector is greater than the corresponding text angle, recombining the picture feature vectors until the angle between the text feature vector and the picture feature vector is less than or equal to the corresponding text angle specifically includes:
[0130] Determine a first feature vector for each of the first picture feature words and a second feature vector for each of the second picture feature words;
[0131] Determine the first picture feature word of the first feature vector which is farthest from the preset center vector as the first target feature word;
[0132] Determine the second picture feature word of the second feature vector which is farthest from the preset center vector as the second target feature word;
[0133] The first target feature word and the second target feature word are deleted, and the remaining first picture feature words and the remaining second picture feature words are recombined to obtain a new picture feature vector, until the angle between the text feature vector and the picture feature vector is less than or equal to the corresponding text angle.
[0134] Assume that a number of first image feature words (feature words of text areas) and second image feature words (feature words of non-text areas) have been obtained, and the preset center vector has been calculated. The specific steps are as follows:
[0135] 1. Obtain feature vectors: obtain the first feature vector of each first image feature word; obtain the second feature vector of each second image feature word.
[0136] 2. Calculate the distance between the first feature vector and the preset center vector: Calculate the distance between each first feature vector and the preset center vector. Select the first image feature word corresponding to the first feature vector with the farthest distance as the first target feature word.
[0137] 3. Calculate the distance between the second feature vector and the preset center vector: Calculate the distance between each second feature vector and the preset center vector. Select the second image feature word corresponding to the second feature vector with the farthest distance as the second target feature word.
[0138] 4. Delete the target feature words and recombine: Delete the first target feature words and the second target feature words, and recombine the remaining first picture feature words and second picture feature words to form a new picture feature vector.
[0139] Repeat steps 2 to 4 until the angle between the text feature vector and the image feature vector is less than or equal to the corresponding text angle.
[0140] Through the above steps, the image feature vector can be effectively adjusted so that the angle between it and the text feature vector is less than or equal to the preset text angle. This method ensures the consistency between the image feature vector and the text feature vector, thereby improving the accuracy and reliability of the recommendation system.
[0141] Exemplarily, before inputting the material resources corresponding to the plurality of label vectors into a set text generation model to obtain a recommendation solution, the method includes:
[0142] According to the direction of the preset center vector, increasing the learning rate of the text generation model in the direction;
[0143] According to the distribution characteristics of the preset center vector, adjusting the batch size of the text generation model;
[0144] According to the number of clusters in the clustering, the text generation model is enlarged by a multiple equal to the number.
[0145] The preset center vector is the average of the feature vectors extracted from the historical user behavior data, which represents the central trend of user preferences. The direction can be determined by calculating the angle between the preset center vector and the coordinate axis, or by observing the main components of the vector to determine the main direction.
[0146] The learning rate of the text generation model is adjusted according to the main direction of the preset center vector. This means that during the model training process, features that are consistent with the direction of the preset center vector are given higher weights.
[0147] For example, if the preset center vector has a higher value in a certain dimension, the learning rate in that dimension can be appropriately increased to enhance the model's ability to learn the features of that dimension.
[0148] Observe the distribution characteristics of the preset center vector, such as concentration, variance, etc. The distribution characteristics of the preset center vector reflect the concentration of user preferences. If the distribution of the preset center vector is relatively concentrated (small variance), it means that the user preferences are relatively single. You can consider reducing the batch size to let the model focus on learning these concentrated features. If the distribution of the preset center vector is relatively dispersed (large variance), it means that the user preferences are diverse. You can consider increasing the batch size so that the model can handle more diverse features.
[0149] This embodiment needs to determine the number of clusters in the clustering. This number reflects the diversity or complexity of the data. The scale of the text generation model is enlarged according to the number of clusters. The enlargement of the model scale can be reflected in the increase in the number of model layers, the increase in the number of hidden units, etc. The enlargement multiple is proportional to the number of clusters.
[0150] Through the above steps, we can adjust the learning rate according to the direction of the preset center vector, adjust the batch size according to its distribution characteristics, and scale the model according to the number of clusters in the cluster. These adjustments help improve the effect of the text generation model and make the generated content more in line with user needs and preferences.
[0151] Compared with the prior art, an embodiment of the present invention provides an AI intelligent recommendation method, which realizes the unified representation of data of different modalities by converting text features and image features into feature vectors respectively, so that data of different modalities can be compared and fused in the same feature space. The method of calculating the angle enables the system to quantify the similarity between different feature vectors and decide whether to fuse them accordingly. This alignment method ensures the consistency of features of different modalities and improves the accuracy of fusion. In addition, in the absence of data of a certain modality, the system can flexibly use data of another modality to generate recommendation solutions to ensure the robustness of the recommendation system.
[0152] In summary, through feature extraction, feature alignment and flexible handling of missing data, we ensured the effective integration of different modal data, thereby generating more comprehensive and accurate recommendations. This method not only improves the performance of the recommendation system, but also enhances the flexibility and robustness of the system.
[0153] An embodiment of the present application provides an AI intelligent recommendation system, including: an acquisition module 20, a text extraction module 21, an image extraction module 22, a vector module 23 and a recommendation module 24.
[0154] The acquisition module 20 is used to acquire input text and / or input image.
[0155] The text extraction module 21 is used for obtaining the input text, performing feature extraction on the input text to obtain a plurality of text feature words, and combining the plurality of text feature words in pairs to obtain a plurality of text feature vectors.
[0156] The picture extraction module 22 is used to obtain the input picture, perform feature extraction on the input picture to obtain a plurality of picture feature words, and combine all of the plurality of picture feature words to obtain a picture feature vector.
[0157] The vector module 23 is used to calculate the angles between all the text feature vectors and the preset center vector as the text angle; if the angle between one of the text feature vectors and the image feature vector is less than or equal to the corresponding text angle, the text feature vector is merged with the image feature vector to obtain a text-image feature vector; if the angle between one of the text feature vectors and the image feature vector is greater than the corresponding text angle, the image feature vectors are recombined until the angle between the text feature vector and the image feature vector is less than or equal to the corresponding text angle, and the text feature vector is merged with the image feature vector to obtain a text-image feature vector; if the input text is not obtained, the image feature vector is used as the text-image feature vector; if the input image is not obtained, the fusion vector of the several text feature vectors is used as the text-image feature vector.
[0158] The recommendation module 24 is used to determine in the database several label vectors whose similarity with the image and text feature vector is greater than a preset threshold, input the material resources corresponding to the several label vectors into the set text generation model, and obtain a recommendation scheme; the label vector is generated after feature extraction of the corresponding material resources.
[0159] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the identification system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0160] Compared with the prior art, an embodiment of the present invention provides an AI intelligent recommendation system, which realizes the unified representation of data of different modalities by converting text features and image features into feature vectors respectively, so that data of different modalities can be compared and fused in the same feature space. The method of calculating the angle enables the system to quantify the similarity between different feature vectors and decide whether to fuse them accordingly. This alignment method ensures the consistency of features of different modalities and improves the accuracy of fusion. In addition, in the absence of data of a certain modality, the system can flexibly use data of another modality to generate recommendation solutions to ensure the robustness of the recommendation system.
[0161] In summary, through feature extraction, feature alignment and flexible handling of missing data, we ensured the effective integration of different modal data, thereby generating more comprehensive and accurate recommendations. This method not only improves the performance of the recommendation system, but also enhances the flexibility and robustness of the system.
[0162] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.
Claims
1. An AI intelligent recommendation method, characterized in that: include: Get input text and / or input image; After obtaining the input text, extracting features of the input text to obtain a plurality of text feature words, and combining the plurality of text feature words in pairs to obtain a plurality of text feature vectors; After obtaining the input image, extracting features of the input image to obtain a plurality of image feature words, and combining all the image feature words to obtain an image feature vector; Calculate the angles between all the text feature vectors and the preset center vector as the text angle; if the angle between one of the text feature vectors and the picture feature vector is less than or equal to the corresponding text angle, fuse the text feature vector with the picture feature vector to obtain a picture-text feature vector; if the angle between one of the text feature vectors and the picture feature vector is greater than the corresponding text angle, recombine the picture feature vector until the angle between the text feature vector and the picture feature vector is less than or equal to the corresponding text angle, fuse the text feature vector with the picture feature vector to obtain a picture-text feature vector; if the input text is not obtained, use the picture feature vector as the picture-text feature vector; if the input picture is not obtained, use the fusion vector of the several text feature vectors as the picture-text feature vector; Determine in the database a number of label vectors whose similarity with the image and text feature vector is greater than a preset threshold, input the material resources corresponding to the number of label vectors into a set text generation model, and obtain a recommendation plan; The label vector is generated after feature extraction of the corresponding material resource; After obtaining the input image, extracting features of the input image to obtain a plurality of image feature words, and combining all the image feature words to obtain an image feature vector, specifically including: Separating text areas from other non-text areas according to the connected area recognition result of the input image; In the text area, the accumulated value of each column of pixels is calculated, and the positions of several text lines are determined according to the peak value of the vertical projection; According to the position of the text line, the accumulated value of the pixels of each line of the text line is calculated, and segmentation is performed according to the peak value of the horizontal projection to obtain a plurality of words; Classifying the plurality of words using a machine learning model or a deep learning model, comparing each classification with a preset word template, and obtaining a plurality of matching first image feature words according to the comparison result; Clustering all the other non-text areas, comparing each cluster with a preset picture template, and obtaining a plurality of matching second picture feature words according to the comparison result; Combining the plurality of first image feature words and the plurality of second image feature words to obtain an image feature vector; Before calculating the angles between all the character feature vectors and the preset center vector as the character angle, the method includes: Obtain user behavior data for all recommended solutions in historical data; Obtain several label vectors corresponding to several recommendation solutions whose user click rate and dwell time meet preset conditions; Clustering the plurality of label vectors using a clustering algorithm to obtain a plurality of clusters; selecting an average vector of all clusters as a preset center vector; If the angle between one of the text feature vectors and the picture feature vector is greater than the corresponding text angle, the picture feature vectors are recombined until the angle between the text feature vector and the picture feature vector is less than or equal to the corresponding text angle, specifically including: Determine a first feature vector for each of the first picture feature words and a second feature vector for each of the second picture feature words; Determine the first picture feature word of the first feature vector which is farthest from the preset center vector as the first target feature word; Determine the second picture feature word of the second feature vector which is farthest from the preset center vector as the second target feature word; The first target feature word and the second target feature word are deleted, and the remaining first picture feature words and the remaining second picture feature words are recombined to obtain a new picture feature vector, until the angle between the text feature vector and the picture feature vector is less than or equal to the corresponding text angle.
2. An AI intelligent recommendation method as claimed in claim 1, characterized in that: After obtaining the input text, extracting features of the input text to obtain a plurality of text feature words, and combining the plurality of text feature words in pairs to obtain a plurality of text feature vectors, specifically includes: Segmenting the input text according to a preset rule to obtain a plurality of words; Using the TextRank algorithm to extract the several words to obtain several feature words; The plurality of feature words are combined in pairs to form a plurality of feature vectors.
3. The AI intelligent recommendation method according to claim 1, characterized in that: The step of calculating the accumulated value of each column of pixels in the text area and determining the positions of several text lines according to the peak value of the vertical projection specifically includes: Create a first array with length equal to the image width; Traverse each column of pixels, and use the number of preset color pixels in each column as the array value of the first array; The local maximum value of the first array is found through the sliding window method to determine the starting position and the ending position of the text line.
4. The AI intelligent recommendation method according to claim 1, characterized in that: The calculation of the accumulated value of the pixels of each line of the text line and segmentation according to the peak value of the horizontal projection to obtain a plurality of words specifically includes: Initialize a second array with a length equal to the height of the text line; Traversing each text line, and calculating the number of pixels of a preset color in the line as the array value of the second array; The local maximum interval of the second array is found by a sliding window method, and the text line is segmented according to the local maximum interval of the second array to obtain a plurality of words.
5. The AI intelligent recommendation method according to claim 1, characterized in that: The step of separating the text area from other non-text areas according to the connected area recognition result of the input image specifically includes: Detecting connected regions in the input image after binarization; Calculate the area, aspect ratio and appearance contour of each of the connected regions; The connected regions with an outer contour of a long strip or a circle are excluded, and the text regions are screened out according to the area and aspect ratio of the remaining connected regions.
6. The AI intelligent recommendation method according to claim 1, characterized in that: Before inputting the material resources corresponding to the plurality of label vectors into the set text generation model to obtain the recommendation scheme, the method includes: According to the direction of the preset center vector, increasing the learning rate of the text generation model in the direction; According to the distribution characteristics of the preset center vector, adjusting the batch size of the text generation model; According to the number of clusters in the clustering, the text generation model is enlarged by a multiple equal to the number.
7. An AI intelligent recommendation system, characterized in that: include: An acquisition module, used to acquire input text and / or input image; A text extraction module is used to extract features of the input text to obtain a plurality of text feature words after obtaining the input text, and to combine the plurality of text feature words in pairs to obtain a plurality of text feature vectors; A picture extraction module, configured to extract features of the input picture after acquiring the input picture to obtain a plurality of picture feature words, and combine all of the plurality of picture feature words to obtain a picture feature vector; A vector module, used for calculating the angles between all the text feature vectors and a preset center vector as the text angle; if the angle between one of the text feature vectors and the picture feature vector is less than or equal to the corresponding text angle, the text feature vector is merged with the picture feature vector to obtain a picture-text feature vector; if the angle between one of the text feature vectors and the picture feature vector is greater than the corresponding text angle, the picture feature vector is recombined until the angle between the text feature vector and the picture feature vector is less than or equal to the corresponding text angle, and the text feature vector is merged with the picture feature vector to obtain a picture-text feature vector; if the input text is not obtained, the picture feature vector is used as the picture-text feature vector; if the input picture is not obtained, the fusion vector of the several text feature vectors is used as the picture-text feature vector; A recommendation module, used to determine in a database a number of label vectors whose similarity with the image-text feature vector is greater than a preset threshold, and input the material resources corresponding to the number of label vectors into a set text generation model to obtain a recommendation plan; The label vector is generated after feature extraction of the corresponding material resource; After obtaining the input image, extracting features of the input image to obtain a plurality of image feature words, and combining all the image feature words to obtain an image feature vector, specifically including: Separating text areas from other non-text areas according to the connected area recognition result of the input image; In the text area, the accumulated value of each column of pixels is calculated, and the positions of several text lines are determined according to the peak value of the vertical projection; According to the position of the text line, the accumulated value of the pixels of each line of the text line is calculated, and segmentation is performed according to the peak value of the horizontal projection to obtain a plurality of words; Classifying the plurality of words using a machine learning model or a deep learning model, comparing each classification with a preset word template, and obtaining a plurality of matching first image feature words according to the comparison result; Clustering all the other non-text areas, comparing each cluster with a preset picture template, and obtaining a plurality of matching second picture feature words according to the comparison result; Combining the plurality of first image feature words and the plurality of second image feature words to obtain an image feature vector; Before calculating the angles between all the character feature vectors and the preset center vector as the character angle, the method includes: Obtain user behavior data for all recommended solutions in historical data; Obtain several label vectors corresponding to several recommendation solutions whose user click rate and dwell time meet preset conditions; Clustering the plurality of label vectors using a clustering algorithm to obtain a plurality of clusters; selecting an average vector of all clusters as a preset center vector; If the angle between one of the text feature vectors and the picture feature vector is greater than the corresponding text angle, the picture feature vectors are recombined until the angle between the text feature vector and the picture feature vector is less than or equal to the corresponding text angle, specifically including: Determine a first feature vector for each of the first picture feature words and a second feature vector for each of the second picture feature words; Determine the first picture feature word of the first feature vector which is farthest from the preset center vector as the first target feature word; Determine the second picture feature word of the second feature vector which is farthest from the preset center vector as the second target feature word; The first target feature word and the second target feature word are deleted, and the remaining first picture feature words and the remaining second picture feature words are recombined to obtain a new picture feature vector, until the angle between the text feature vector and the picture feature vector is less than or equal to the corresponding text angle.